Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Aquarium / Lco learning and production
- - By cma6 (****) Date 2019-04-16 21:27
My main analysis system is a dual-Xeon system with 2 X Xeon-E5-2686-V3 running at 2.0 GHz for 2 X 18 = 36 cores.
With no graphics card, I am using onboard video.
I am considering adding a graphics card with two ideas in mind--not mutually exclusive. Hopefully Pawnslinger and others who are knowledgeable can give me tips.
  1) Pawnslinger has mentioned that he uses his GPU with Lc0 to feed positions into Aquarium trees. What GPU do you use and does it provide enough useful input (tasks) so that IDeA output is significantly improved with the Lco input?
Or is this mostly a learning experience in the NN world?
Put another way, is the game worth the candle: for the time/expense of a mid-high GPU like the RTX 2070, will one get enough useful input to improve IDeA output?
  My impression is that one needs a superfast GPU setup to get anything worthwhile out of Lc0, especially if one already has a high-end CPU infrastructure on the master system and over one's lan (56 cores total).
  2) If there is little value added from GPU/Lc0 into IDeA, then is it still worthwhile to learn now about the Lc0/nn world with a perhaps lower end GPU?
  So if one is not interested in an academic exercise for its own sake, how should one proceed with respect to a new or used GPU?
  a) higher end GPU for input into IDeA;
  b) lower end GPU to learn the nn system;
  c) Do nothing until GPU prices finally break (sometime in our lifetime) and/or a big increase in value is provided to the GPU consumer?
Parent - By pawnslinger (****) Date 2019-04-16 23:01
I have been puttering around with lc0 just to get my feet wet, in a manner of speaking.  Nothing serious.  What I have learned is that lc0 needs the best gpu you can afford AND the more vram the better.  My 3gb card is really not enough to get a serious test of lc0.

If I had the funds, I would opt for one of the new RTX cards from Nvidia, the more vram, the happier lc0 will be.  It all depends on your budget and what card will work in your workstation.  Does your xeon system have a uefi bios?  That will probably be necessary for an RTX card.  One thing, the last I looked (a few moths ago) AMD gpu's were a no go.  So you would need to be considering gpu's from Nvidia that will work on your machine with the most vram you can find.

I don't know how they did it, but in the last tournament I saw lc0 play in they used dual GTX 1080's.  And it was doing fine.  When cut back to one 1080, the quality of play dropped off a lot.

My xeon system has a legacy bios, so it is a no go for newer cards.  My system has 12 cores at 3ghz, so fewer than yours, but a little faster cores.  My 3gb card is in my personal system running a uefi bios and Ryzen 1700x.  It is the GTX 1060 3gb model, the best I could afford at the time.  I purchased it before lc0 became a thing, I needed a gpu to run some games (not Chess) at 1080p and the 3gb vram is fine for that.
Parent - - By Ghengis-Kann (***) Date 2019-04-16 23:36
Hi Cma6.

I have experimented extensively with LC0 and Aquarium over the past month and can share my experience.
First of all you want the right network. 32930 has been the tournament network for awhile, but I think it has recently been superceded by 41800.

The video card quality makes an enormous difference.
Don't even bother with AMD or any Nvidia card prior to the current RTX series.

The KN/s numbers are way lower for Monte Carlo Tree search engines than for a/b so don't be deceived by the magnitude of the numbers.
My GTX980 card gets maybe 1 KN/s, the GTX1080 G1 gets about 2 or 3 KN/s, and the 2080Ti card is more like 38 KN/s.
So the $1200 card is 15 times faster than the $600 card.

The reason for this is the inclusion of Tensor Cores in the RTX (Turing) series cards.
The purpose of this is to run a video enhancing neural network called Deep Learning Super Sampling, but the hardware is also a great place for Leela to live.
The Turing series cards can also run FP16 Tensor operations, which greatly accelerate LC0 compared to FP32.

Leela is self-taught, and tends to pick certain openings to focus on during the training process, so it is not uniformly strong across all types of positions.
It is great at positional chess but lacking in tactical execution, and tends to dither about in an infuriating manner.
The problem is that it is unable to choose between multiple winning moves. For example, in a queen and king versus lone king endgame any move that doesn't stalemate or hang the queen gets a winning evaluation. It doesn't care at all how quickly it wins and will move the queen around randomly until the 50 move limit approaches, at which point it will find mate.

Endgame tablebases will solve that particular problem, but I have seen it play rook and pawn endings for hundreds of moves, most of which were meaningless checks that chase the enemy king in circles. Komodo MCTS solves this problem by switching to a/b search when the evaluation exceeds +/- 7, but LC0 does not have this option.

If you are serious about it go for the 2080Ti card, but a decent card to experiment with at a much more reasonable price is the RTX2060.
Attachment: RTXComparison.JPG (90k)
Parent - - By cma6 (****) Date 2019-04-17 00:39
Pawnslinger, thanks for detailed responses. I will take them to mean "yes" to question 2) as to whether I should begin learning to use Lc0.
But as 1) whether your IDeA output is noticeably improved because of lc0 input, I'm not clear on that.

My Xeon does have uefi BIOS, so I should probably go ahead with RTX 2070, and thanks for very interesting URL for RTX comparison.
Parent - By pawnslinger (****) Date 2019-04-17 02:02
Well, my experience is not complete due to the lack of powerful gpu.  Since your system has uefi bios, I think the RTX line of cards is the place to buy.  I would go with the card that has the most vram, that is still within your budget.

Best of luck!  Let us know here about your experience!!

One thing to think about... the upcoming Navi cards from AMD are rumored to have ray tracing, similar to RTX cards.  I do not know how they will fare against the RTX cards, nor do I know if they will have tensor cores (a very nice feature of RTX).  But I think they will give RTX some competition.
Parent - By pawnslinger (****) Date 2019-04-17 02:17
In competitions that I have held privately on my own system, I have seen lc0 unable to win simple R v P endings.  Even R v K endings seem hard for it to comprehend.  I always thought this was due to my poor gpu.  But maybe the NN technology has not progressed far enough that lc0 has not learned the basic techniques.  I recall complaining in the lc0 forums that this type of ending is something that I learned very early in my Chess career.  I found it amazing that lc0 could not learn the simple method.

I think this is a weakness of the "self taught" NN approach.  Perhaps lc0 should read "Basic Chess Endings" or "Pawn Power" or even  Nimzo's tome "My System".  Give lc0 a leg up.  But I doubt that the self taught NN can read and comprehend.  Maybe IBM's AI can do that...  what does IBM call its AI?  I forget.  But they claim it can read.
Parent - - By ChiefPushesWood (**) Date 2019-04-17 13:01
I'm nowhere near the caliber of Pawnslinger or Ghengis. However, I did just recently buy an RTX 2080 and I've been playing with Leela.

Here's my take on your question. From the perspective of correspondence chess, is it worth having the card and Leela? In short, yes. (In my opinion). With a couple caveats.

#1. Leela isn't going to (at least currently) revolutionize your games, analysis or processes. As already mentioned, she struggles in certain areas (tactics, endgames, etc)

#2. It's useful as another tool in your toolbox. She hasn't replaced anything in my arsenal. And I don't see her replacing anything for the foreseeable future.

So why is it worth it? Well... 2 reasons (again, IMO). Leela continues to get stronger from a different perspective and methodology than the AB engines. This has made things interesting for me again (see my 'boredom' post from a while back). I watched the recent TCEC where Leela gave SF all it wanted in the SUFI. Many, including myself, believe that Leela outplayed SF and really only lost due to some of the technical shenanigans that went on. To boot, the games were some of the most interesting engine games I've ever seen from a purely chess perspective. Leela's positional play is incredibly fun to watch. And even sensible from a human perspective.
Finally, I think that the greatest benefit of Leela for the Corr player is idea generation (not Aq IDEA). In quiet and positional games, especially where AQ IDEA is giving your top 10 moves all 0.00 for example, Leela comes up with different, interesting and - dare I say - human style plans against the obvious high use of SF in these games.

Let me say this... I can't spend your money. Nor would I want to. The investment is steep for sure. These cards are stupid expensive. Only you can make that decision for yourself. I've enjoyed playing with Leela and I'm glad I made the decision to buy the card. I have benefited from having Leela in my arsenal. But, again, NOTHING GAME-CHANGING, from a correspondence game perspective, at all.

Just another tool.

Parent - - By cma6 (****) Date 2019-04-17 14:09 Edited 2019-04-17 17:17
Thanks to CPW and Ghengis for the very useful commentary; and a correction: thanks to Ghengis for the RTX comparison url.
"32930 has been the tournament network for awhile, but I think it has recently been superceded by 41800."
  I am confused as to the frequent references to a network. A correspondence player won't be getting the client; just regularly downloading the weights file, isn't that right? So the reference to "network"-- doesn't that really mean "the url where you will get updates (how frequent?) for the weights file?
  I have been focusing (as a compromise on price and capability) on the EVGA GeForce ultra RTX 2070 XC Gaming, 8GB GDDR6. However, with respect to all the high end GPUs, I often see the following: if your GPU is much faster than your cpu, then your cpu(s) will bottleneck the GPU performance.
With my 2.0 GHz Xeon, would a RTX 2070 suffer from this problem because of a too slow CPU?
Parent - - By ChiefPushesWood (**) Date 2019-04-17 14:41 Edited 2019-04-17 14:46
Referencing the network is referencing the 'weights' file.
41800 is currently the 'tournament' network that would be used by the developers. 32930 was the network used by Leela in the last TCEC SUFI.
You can keep track of all of this here:

Changing out the network is SUPER easy (just downloading, renaming and placing the NN weights file into the appropriate folder)

Personally, I think you would be perfectly fine with that CPU/GPU combo. For chess purposes at least.. If you were gaming, that would be a different story.


Edit: For entertainment purposes: I just realized that they 'noted' on line 17 of that spreadsheet that 41800 is the first tested network to beat AlphaZero.
Parent - - By Ghengis-Kann (***) Date 2019-04-17 16:30
The only thing I would add to this is that if you really want to know what's going on and keep up to date you need to join the Discord channel.

Click the link on the left of this page that says Chat:
Parent - By ChiefPushesWood (**) Date 2019-04-17 18:58
Indeed. The discord is THE place for info.

Parent - - By CumnorChessClub (***) Date 2019-04-17 19:31
When and where did they play AlphaZero?

I think they mean something different to actually playing?
Parent - - By ChiefPushesWood (**) Date 2019-04-17 19:37 Edited 2019-04-17 19:46
I have no idea. That's why I posted the image. I'm just relaying what I'm reading. Hard to BEAT AZ without PLAYING AZ.
I'm curious myself.

CLARIFICATION: They are comparing AZ's ELO vs SF. This is the first net to better AZ's ELO.

Parent - By CumnorChessClub (***) Date 2019-04-18 08:24
Thanks for the info.
Up Topic Rybka Support & Discussion / Aquarium / Lco learning and production

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill