Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka 3 under Linux crashes (also on Windows !)
- - By pgeorges (*) Date 2008-08-09 21:30
Hi,

I just bought Rybka 3 and cannot run it under Linux (my goal is to use it with Scid). The engine crashes as soon as it gets the input "go infinite ponder".
I tried with various versions of wine and it crashes after those commands :
uci
isready
go infinite ponder

then, I tried under Windows XP, and the same commands make Rybka crash.

All other UCI engines I am aware of don't crash after those commands. So maybe it is a bug in Rybka ?

Pascal Georges
Parent - - By pgeorges (*) Date 2008-08-09 23:05
I found the solution and now Rybka 3 works with Scid. Simply Rybka 3 dislikes
   go infinite ponder
but
   go ponder infinite
is ok for him !

So the workaround is easy. But Rybka's command line parser needs a small fix ;-)

Pascal Georges
Parent - - By brunjes (**) Date 2008-08-10 05:28
Pascal,

So what is the fix to make Rybka 3 work with SCID? Is there a change being made to SCID to accommodate Rybka 3's inability to handle "go infinite ponder"?

Thanks,

Roy
Parent - - By pgeorges (*) Date 2008-08-10 06:41
It is sufficient to look for "go infinite ponder" occurences in Scid and change it for "go ponder infinite".
I will release a Scid 3.6.25 release candidate soon with this small change. In case it is useful for you, Scid's CVS at Sourceforge is updated also.

Pascal Georges
Parent - By Felix Kling (Gold) Date 2008-08-10 08:58
Ah, so you are the guy behind Scid :) welcome to our forum! :)
Parent - - By giorgio Date 2008-08-10 09:55
Hello Pascal,
I'm afraid that it's not enough.
I don't have Rybka 3 so I can't check but yesterday Roy sent me a log file where "go infinite ponder" command was not sent to rybka, but it crashed anyway.

This is the interesting part of the log:

Scid  : uci
Scid  : isready
Engine: id name Rybka 3
Engine: id author Vasik Rajlich, Larry Kaufman
Engine: option name UCI_Query type button
..............
All the rybka options come here
..............
Engine: option name Time Buffer type spin default 0 min 0 max 900
Engine: uciok
Scid  : isready
Engine: readyok
Scid  : setoption name Hash value 512
Scid  : isready
Engine: readyok
Scid  : setoption name MultiPV value 1
Scid  : isready
NOTE  : Engine terminated without warning.

So, in my opinion, someone should try to give this commands, after launching rybka3 from a command line (in Linux with microwine)
   - uci
   - setoption name Hash value 512
   - setoption name MultiPV value 1
and see the output of rybka replying to these commands.

Bye.
Parent - - By pgeorges (*) Date 2008-08-10 10:20
Hello Giorgio,

With the rc1 of Scid I put online this morning, Rybka 3 works well under Linux and Windows.

Note that I use the 32 bit version and wine (not microwine).

Pascal Georges
Parent - - By brunjes (**) Date 2008-08-10 15:15
Pascal,

I picked up the changes you made and can confirm that Rybka 3 runs, but it still has a problem with using any reasonable amount of hash tables. For example, when I try to set hash size to 512 MB on a 3 GB system I get a huge spike in disk I/O that at first sight looks like swapping going on. However, the 'free' command shows 0 bytes of swap used. So what is causing that I/O? On my systems, pdflush gets very busy as does mount.ntfs while the engine attempts to run.  After 109 seconds, my Rybka-3 engine reached a depth of 12 ply and the system becomes almost unresponsive with that much disk I/O happening. SCID reported 1 kn/sec of "speed".  :-)  Also of note is that the load average jumps to 5 or 6 soon after Rybka is started. This seems unusually high to me with just two Rybka 3 32-bit engines running (and nothing else happening on my system -- that should make the load average 2, not 5 or 6).  vmstat shows the system is waiting for I/O 94% of the time on my 1 GB machine (a native Linux install) and on a borrowed Windows system with a wubi installed Ubuntu, it shows the mount.ntfs process using as much CPU time as the engines themselves. In Wubi installs, the native Windows NTFS partition is used to store the Ubuntu system as a directory off of C:\ hence the needs for ntfs accesses. The question is: why is the engine doing any disk I/O at all and if it is not, what is making the system think it needs to flush out changes to the file system in the first place?

I even ran a test with hash size set to 256 MB and at first, Rybka gets full cpu on both cores. However after about a minute, the now familiar (and dreaded) heavy disk I/O shows up, stealing a big chunk of CPU resources away from the engine. This is preventing me from using Rybka 3 at all as I have no interest in running it with 32 MB of hash (which works without this issue) on such a large hardware configuration.

On my 1 GB system, I tried using 128 MB of hash tables under SCID and the same scenario plays out. The free command shows I am using absolutely no swap space, but vmstat shows 94% of the time is waiting for I/O (leaving 6% CPU for the engine and all other processes).

I believe this is a Rybka engine thing and maybe Vas can comment on what may be happening - there was talk of encryption in the engine to protect his code from cloners -- would that have an effect here that does not show when running under Windows? There could be a system call he's making that Windows does very well and wine just sucks at? I should note that this problem does not happen on Windows (I've verified with a borrowed Windows PC just running Aquarium on it). However if there is some workaround you can think of (perhaps more of a wine configuration thing?) I'm eager to hear it.

Thanks,

Roy
Parent - - By pgeorges (*) Date 2008-08-10 16:45
My configuration is :
- Scid 3.6.25rc1
- Linux Suse 10.x
- Wine 1.1.2
- 2 GB RAM
-Rybka 3 1CPU 32 bits (without Aquarium, my error of the week-end :-( )

I tried 256 and 512 MB of hash tables, and after 4 minutes each time (from start position), I get strictly no slow down, no heavy I/O.

So could you please confirm to me that you use Rybka with Scid ? Just for me to be sure that the hash values I sent are taken into account by Rybka (wine hides this information).
I checked with Toga, and hash is correctly set, but I'd like to be sure Rybka's behavior is what I expect.

Something to try : unconfigure drive mappings in Wine setup to any NTFS or FAT file systems : maybe there is a bug in wine triggered by Rybka's behavior ?
Also could try to set Rybka to use 1 CPU only ?

Pascal
Parent - - By brunjes (**) Date 2008-08-10 17:52
Pascal,

On my 32-bit Ubuntu system with 1 GB RAM running Wine 1.1.2 on a Single 2.2 GHz Athlon (64-bit CPU) I see the behavior I described. I am NOT running Aquarium for these tests though Rybka 3 is still doing the same thing under Aquarium.

I cannot get more than 32 MB of hash without seeing the problem. I have gathered 'vmstat 10' output collected before, during and after Rybka ran under SCID. The number of context switches that occur when Rybka runs are several times higher than when she is not running.

There are no drive mappings on this particular computer (it is a straight 100% Ubuntu with no NTFS to be found). The problem is still there. I have MaxCPU set to 1 (it was by default) since this is a single processor. On my two core system, the default Max CPU was 2048 which I painfully lowered to 2 (painful because I cannot type in the window on SCID to lower this value, I must hold the mouse button down for almost 2 solid minutes to get the value lowered to 2! -- a fix from you on that would be nice to see by the way!).

I watch memory consumption with top and it shows more RAM being used by the Rybka process than I set -- this has been documented here by others already. Rybka seems to grab about 75 MB MORE RAM than you set for hash tables (this may well be per core though I have one core on this system under discussion). This is not Nalimov cache (I set that to one MB -- the minimum). I have not turned on persistent hash but even so I took the minimums for that (8 MB for several values related to Persistent Hash).

Here is the vmstat data for a run with hash set to 32 MB and 64 MB. Look at the differences in waiting for I/O time (last column on the right).

32 MB hash vmstat 10 output:

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
2  0      0 155832  71484 395008    0    0   199  7420  324 1159 18  3 45 34
0  0      0 158804  71492 395008    0    0     0     4  176  586  5  1 94  0
0  0      0 158572  71528 395008    0    0     0    72  184 2044 19  5 77  0
0  0      0 158572  71536 395008    0    0     0     2  127  268  1  1 98  0
0  0      0 158580  71544 395008    0    0     0     6  126  260  1  1 98  0
0  0      0 158580  71552 395008    0    0     0     8  128  279  1  1 98  0
1  0      0  50588  71604 427776    0    0     0    76  170 2081 82  7 11  0
1  0      0  50496  71620 427808    0    0     3     5  156  889 97  3  0  0
1  0      0  49980  71628 427808    0    0     0    10  133  534 98  2  0  0
1  0      0  49848  71684 427808    0    0     0  3464  137  396 98  2  0  0
1  0      0  50380  71708 427808    0    0     0 11021  150  241 98  2  0  0
1  0      0  50256  71732 427808    0    0     0  9258  147  240 98  2  0  0
1  0      0  50256  71756 427808    0    0     0  8850  145  241 98  2  0  0
1  0      0  50256  71780 427808    0    0     0  8868  147  236 97  3  0  0
1  0      0  50256  71804 427812    0    0     0  8832  154  644 97  3  0  0
1  0      0  50228  71828 427812    0    0     0  8890  146  239 98  2  0  0
1  0      0  49628  71852 427812    0    0     0  8862  146  238 98  2  0  0
1  0      0  49600  71876 427812    0    0     0 10192  154  571 97  3  0  0
1  0      0  50104  71920 427812    0    0     0 12227  167  324 97  3  0  0
1  0      0  50104  71936 427812    0    0     0  6394  139  241 98  2  0  0
1  0      0  50104  71944 427812    0    0     0     2  126  239 99  1  0  0
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
2  0      0  50104  71952 427812    0    0     0     3  126  238 99  1  0  0
1  0      0  49980  71964 427812    0    0     0  3440  133  237 98  2  0  0
1  0      0  49980  71988 427812    0    0     0 11395  151  242 98  2  0  0
1  0      0  49980  72028 427812    0    0     0  9308  147  246 98  2  0  0
1  0      0  49476  72052 427812    0    0     0  8858  147  345 98  2  0  0
1  0      0  49560  72084 427812    0    0     0  8852  149  420 97  3  0  0
1  0      0  49468  72116 427812    0    0     0  8816  147  369 98  2  0  0
1  0      0  49856  72140 427812    0    0     0  8861  145  239 98  2  0  0
1  0      0  49856  72156 427812    0    0     0  8819  145  241 95  5  0  0
1  0      0  49856  72196 427812    0    0     0  8837  146  245 98  2  0  0
1  0      0  49856  72212 427812    0    0     0  3616  133  240 99  2  0  0
1  0      0  49436  72220 427812    0    0     0     2  125  238 99  1  0  0
1  0      0  49396  72228 427812    0    0     0     3  125  239 99  1  0  0
1  0      0  49592  72244 427812    0    0     0  1129  127  238 97  1  0  2
1  0      0  49592  72268 427812    0    0     0  7757  143  244 98  2  0  0
1  0      0  49592  72300 427788    0    0     0   421  128  242 99  1  0  0
1  0      0  49088  72312 427812    0    0     0     4  125  238 99  1  0  0
1  0      0  49592  72320 427812    0    0     0     3  126  238 99  1  0  0
1  0      0  49460  72340 427812    0    0     0  5720  138  242 95  2  0  3
1  0      0  49468  72348 427812    0    0     0     2  126  238 99  1  0  0
1  0      0  49468  72356 427812    0    0     0     2  124  237 99  1  0  0
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
2  2      0  49016  72368 427808    0    0     0  1250  126  240 99  1  0  0
1  0      0  49076  72400 427812    0    0     0  4904  139  240 98  2  0  0
1  0      0  49020  72408 427812    0    0     0     2  125  238 99  1  0  0
1  0      0  49020  72416 427812    0    0     0     2  126  236 98  2  0  0

64 MB hash vmstat 10 output:

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
4  0      0 171252  74852 384272    0    0   118  1810  319  851 31  4 33 33
0  0      0 171120  74860 384276    0    0     0     4  155  575  2  1 97  0
0  0      0 171120  74868 384276    0    0     0     4  125  266  1  1 98  0
0  2      0  28404  74980 449792    0    0     0 23433  389 1486 33 10 11 47
1  9      0  29060  75000 449800    0    0     0 27171  418  499 27  6  0 67
0  3      0  28292  75032 449800    0    0     0 21054  606  709 29  6  0 65
0  1      0  29416  75064 449804    0    0     0 31894  389  379 48  6  0 47
0  3      0  28512  75084 449804    0    0     0 32362  472  540 36  6  0 58
0  3      0  27780  75112 449796    0    0     0 30913  508  494 38  6  0 56
0  3      0  28300  75132 449808    0    0     1 29509  456  328 39  6  0 55
0  2      0  29252  75176 449816    0    0     0 19353  576  400 11  4  0 85
0  3      0  29004  75196 449816    0    0     0 22217  588  626 35  6  0 58
0  7      0  28740  75216 449816    0    0     0 34736  414  344 35  7  0 58
0  2      0  27512  75240 449816    0    0     0 38615  365  349 40  6  0 55
0  5      0  29024  75264 449808    0    0     0 21234  523  325 15  4  0 80
1  0      0  29272  75288 449816    0    0     0 30397  353  410 40  6  0 54
0  2      0  27648  75312 449808    0    0     0 32420  497  389 45  6  0 49
0  3      0  28724  75328 449816    0    0     0 16828  569  405 34  6  0 60
0  3      0  28000  75348 449812    0    0     0 24608  581  339 48  5  0 47
1  8      0  29144  75368 449816    0    0     0 18587  637  353 47  5  0 48
0  8      0  28420  75392 449816    0    0     0 34673  456  323 26  6  0 68
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
1 10      0  28872  75416 449816    0    0     0 23735  481  332 27  5  0 68
0  2      0  28112  75440 449816    0    0     0 38509  405  334 31  7  0 62
0  2      0  27568  75488 449872    0    0     4 32620  357  283 37  6  0 57
1  9      0  28748  75516 449888    0    0     0 16449  607  305 43  5  0 52
0  4      0  28724  75568 449896    0    0     0 21988  542  476 48  6  0 46
1  8      0  28660  75600 449896    0    0     0 36689  339  372 34  7  0 59
0  9      0  27896  75624 449896    0    0     0 36660  413  331 28  7  0 65
1  0      0  28796  75648 449896    0    0     0 18571  264  278 72  3  0 26
1  5      0  27396  75664 449896    0    0     0 39568  378  309 34  6  0 59
1  9      0  28428  75688 449896    0    0     0 28518  473  312 36  5  0 59
1  7      0  27880  75712 449896    0    0     0 32382  427  327 34  6  0 60
1  7      0  27924  75736 449896    0    0     0 31028  425  312 30  5  0 65
0  3      0  28632  75760 449896    0    0     0 22807  522  326 31  4  0 65
1  9      0  28456  75792 449892    0    0     0 27399  542  331 42  5  0 53
0  9      0  27908  75824 449900    0    0     0 35998  377  294 29  7  0 64
0 12      0  28364  75848 449900    0    0     0 29646  420  339 26  6  0 68
0  1      0  27628  75880 449892    0    0     0 31111  512  330 40  5  0 55
1  7      0  28264  75896 449900    0    0     0 26368  456  331 22  6  0 72
1  8      0  28368  75920 449900    0    0     0 32268  449  309 46  6  0 48
0  6      0  27532  75944 449900    0    0     0 39919  341  316 36  6  0 58
0  3      0  27392  75976 449904    0    0     0 32266  416  295 31  6  0 63
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
1  1      0  26668  76000 449904    0    0     0 30824  461  311 45  5  0 50
0  3      0  27520  76020 449904    0    0     0 30226  479  320 31  6  0 64
0  2      0  26328  76044 449900    0    0     0 25885  626  355 42  5  0 53
0  2      0  26792  76064 449904    0    0     0 31114  454  319 33  6  0 61

Roy
Parent - - By pgeorges (*) Date 2008-08-10 18:15
Did you try microwine ? If I understood it correctly this is what to use to run Rybka on Linux 64 bits. There have been some posts in this forum.

Pascal
Parent - - By brunjes (**) Date 2008-08-10 18:19
Pascal,

Yes, Microwine is for running 64-bit engines. It has the same problem. Here, I'm running (on my 64-bit Ubuntu system) either 32 or 64-bit Rybka 3 engines -- exact same problems.

I can give you output of dpkg -l on my system if you like and that may point to something I have installed that you don't which may be causing my problem?

Please advise,

Roy
Parent - - By pgeorges (*) Date 2008-08-10 21:24
I cannot help : this is something between Wine and Rybka, and debugging / tracing is needed to understand what is going on.
Running yourself something like "strace wine rybka3.exe" will not help much to find the cause of the problem.

Pascal
Parent - By brunjes (**) Date 2008-08-10 22:28
Thanks all the same Pascal. I even tried to install OpenSuSE 11.0 on that system, but it cannot handle something about my disks and refuses to install. It even corrupted my /home partition so I have to reformat that as I re-install Ubuntu.

Well, Rybka-3 is not for me.

Roy
Parent - By Sesse (****) Date 2008-08-10 23:19
There's something very odd with your system, that's for sure.

You can use blktrace to figure out where all the I/O is coming from, but it's not something for the faint of heart.

/* Steinar */
Parent - By pgeorges (*) Date 2008-08-10 09:13
I uploaded a rc1 of next version of Scid that works well with Rybka 3 at sourceforge.

Pascal Georges
Parent - By Vasik Rajlich (Silver) Date 2008-08-10 08:02
Ok, thanks.

Vas
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka 3 under Linux crashes (also on Windows !)

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill