Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Aquarium / Odd behavior from IDeA
- - By cma6 (****) Date 2020-08-15 13:14 Edited 2020-08-15 16:26
I just recently implemented the use of SF NNUE in IDeA, running overnight, as usual, for 10 hours at least on all systems: the master and 4 slave systems. On each system SF is running with NNUE checked. Naturally each system has the file "\eval\nn.bin" (referenced as an SF parameter).

Here is the unsettling behavior change I have seen in IDeA. Instead of using 52 physical cores as it always has, IDeA cycles between using the 52 engines enabled down to as few as 22 engines; or sometimes a number in between like 38 engines. This behavior is naturally very detrimental to IDeA throughput. Over a lengthy IDeA run, the average number of engines working might work out to be 29.23, for example, instead of 52.

Has anyone else noticed this "throttling down" by IDeA when using SF NNUE; and if so, is there a fix?
Parent - - By dickie (**) Date 2020-08-15 14:51 Upvotes 1
No problem at all here. Can I ask which net are using and which SF compile?
Parent - By cma6 (****) Date 2020-08-15 16:30
I use the latest SF compiles and recommended net from https://abrok.eu/stockfish/ , e.g.,
Date: Fri Aug 14 16:39:52 2020 +0200    Timestamp: 1597415992
  and nn-82215d0fd0df.nnue
Parent - - By pawnslinger (****) Date 2020-08-15 16:06 Upvotes 1
I am not having this problem.

It sounds like some sort of resource bottleneck.  Off-hand I would suspect either video ram or system ram.  I am sure each instance of SF-NNUE is using quite a bit of ram.
Parent - - By cma6 (****) Date 2020-08-15 16:41
PS, you may be onto something re RAM (though not video RAM.)
Here is the relevant section of    RTHomeServer.ini

The remote engine settings
Hash=1024
Threads=1
Contempt=0
MultiPV=1
Move Overhead=30
Analysis Contempt=Off
Contempt=0
Use NNUE=true
EvalFile=c:\Engines\eval\nn.bin
Hash=1024
Syzygy50MoveRule=false
SyzygyPath=C:\345TBs;E:\TB6
Threads=1

Hash size of 1024 is unchanged from pre-NNUE days.
However, there are two major changes from pre-NNUE:
   1) Each engine (physical core) on every system is constantly accessing the 20 MB file nn.bin
   2) Although I always had a SyzygyPath, I believe that the paths on the slave systems were incorrect, so that the remote engines were not in fact accessing the TB files, which in toto amount to 7 GB + 150 GB = 157 GB.
Parent - - By dickie (**) Date 2020-08-15 17:15
This may be an Intel issue. On my Intel I see physical memory utilisation steadily growing to 100% with NNUE, though I have not noticed any engines dropping out. On my AMD, physical memory use remains constant at approx the level of the combined hash files. Something to do with the Intel architecture perhaps, but well beyond my knowledge I'm afraid.
Parent - By cma6 (****) Date 2020-08-15 17:46 Edited 2020-08-16 17:00
Although I haven't used lc0 and don't own a high end GPU, I have often read that advanced lco users run into a similar issue, where they can run i.a. for a limited period of time, depending on system RAM, at which point lco practically shuts down.

  To the extent that SF-NNUE is filling up RAM on each system by constantly and repeatedly reading nn.bin, to that extent I may be seeing a similar effect in SF-NNUE. I will see if I can increase RAM allocation on one system to start.

When went back to using SF, instead of SF-NNUE, in IDeA, I had no problems. That means that the issue is the hash fills up pretty quickly. This is not an issue with i.a., since one can use much more of the available system RAM for i.a., than for each engine (core). I don't know why you fellows have escaped this problem, unless you don't run IDeA for long periods.
Up Topic Rybka Support & Discussion / Aquarium / Odd behavior from IDeA

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill