Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / GUI performance: Arena top, Fritz flop, Aquarium competitive
1 2 Previous Next  
- - By Axel Caro (**) Date 2008-08-23 22:11
Curious, which GUI will perform best according to node count, I did some research.

Testing Rybka 3 (x64, w32, single and multi core) and ohter engines on

- Aquarium (3.0.4)
- Fritz 11 (21.08.2008)
- Shredder Classic 3
- Arena 1.99 beta 5

I found some intensive discussed experiences and presumptions approved.

Fritz 11 suffers from relevant node-count-breakdowns and capricious fluctuations.

Aquarium - still a newcomer - seems to be a good choice, as well as the stable Shredder Classic 3.

Performance-winner is Arena - freeware but efficiently programmed.

In addition better multicore scaling of Rybka 3 becomes evident. She multiplies doubling of cores by 1.7, where Rybka 2.3.2a scaled by 1.6 on average.

Enclosed you will find a corresponding excel sheet.

Regards,
Axel
Parent - - By SummerKnight (**) Date 2008-08-23 22:38
Nice work on the analysis.  Have you tested the original Fritz 11 GUI?  That is, the pre-Rybka 3 version that is used with Fritz 11?
Parent - - By Axel Caro (**) Date 2008-08-24 07:59
Thanks.
Yes, I used the Pre-Rybka Chessbase Fritz 11 GUI (update 21.08.2008). I am not shure, if Rybka features are supported with the latest update.
Parent - - By SummerKnight (**) Date 2008-08-24 15:45 Edited 2008-08-24 15:49
I could be wrong but I thought the 21.08.2008 Fritz 11 GUI had the Rybka 3 special features.  Have you tested any of the older Fritz 11 GUIs (e.g., without support for Rybka 3 special features).

Update: I see my question was answered here: http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=6676
Parent - By Axel Caro (**) Date 2008-08-24 16:00
You are perfectly right - update from 2008-21-08 includes the new features like monte carlo!  *surprised*
Parent - - By cma6 (****) Date 2008-08-24 00:37
Victor, I hope you have noticed the implication of Axel Carol's table. On Win X64, quad-core, R3 in Arena to R3 in Aquarium is 130/116, i.e., R3 runs 12% faster in Arena than in Aquarium. The latter was supposed to be the GUI that would enhance R3's speed.

Instead, although Aquarium has many fine features it is a slower GUI for R3. However, when you create Aquarium Lite, may I suggest that this table proves that the chief design goal should be R3 speed, e.g., R3 should be faster in Aquarium Lite than in any other GUI.
Parent - By BigBen (****) Date 2008-08-24 06:52
Hi,
    The latest F11 GUI update that seems to support the Rybka output does some very odd stuff

Regards
Parent - - By Permanent Brain (*****) Date 2008-08-24 07:09 Edited 2008-08-24 08:07
Thanks for this first class methodical test and presentation. - If you are still in the mood for testing, I would suggest to compare time for depth x (maybe 14 or 15?), instead of kN/s. Also, I would like to ask if you run Rybka with normal, or with lower priority, in Fritz, and in the other GUIs if they offer that setting too.

I assume that the speed differences have to do with the output the interfaces display, how much/how fast, and how much CPU load they cause. For example, I guess that if Arena does not display the output of sampled search (the additional mainline with details) at all, it could be the explanation why Rybka 3 is fastest there.

If you have even more time, it would be also interesting what the results of this speed comparision would be, for Rybka 2.x or another engine. Then we could see if no sampled search output means, no speed difference in these GUIs each, or only a much smaller one.

(I did speed comparisons Fritz<->Arena in the past with other engines, and the results were almost identical.)

Edit: Meanwhile I noticed the kN/s results from other engines in the table. These figures are so similar that time for depth tests are not required with those other engines, in this case.
Parent - - By Axel Caro (**) Date 2008-08-24 13:59
Hi Mike,

thanks for your detailed reply.

> I assume that the speed differences have to do with the output the interfaces display, how much/how fast, and how much CPU load they cause. For example, I guess that if Arena does not display the output of sampled search (the additional mainline with details) at all, it could be the explanation why Rybka 3 is fastest there.


I agree, that speed differences might not be real and misleading display could be caused by GUI (mis-) functionality.

> If you have even more time, it would be also interesting what the results of this speed comparison would be, for Rybka 2.x or another engine. Then we could see if no sampled search output means, no speed difference in these GUIs each, or only a much smaller one.


You can see these results in the excel-sheet-screenshot. I tested Rybka 2.3.2a, HIARCS, Fruit and Toga too.

> Edit: Meanwhile I noticed the kN/s results from other engines in the table.


;-)

> These figures are so similar that time for depth tests are not required with those other engines, in this case.


Yes, but...

> If you are still in the mood for testing, I would suggest to compare time for depth x (maybe 14 or 15?), instead of kN/s.


... although I am still in the mood for testing, comparing time for depth seems not to be trivial. Even concentrating on single core cpu usage, results differ from GUI to GUI.

Comparing Rybka 3 1-cpu on Shredder Classic and Aquarium I got the following output for evaluating the start position for approx. 5 minutes (two runnings):

First running Rybka 3 1-cpu – Aquarium 3.0.4
[+0.22]  d=18  1.e4 Nf6 2.e5 Nd5 3.Nc3 Nxc3 4.dxc3 Nc6 5.Nf3 g6 6.Bc4 Bg7 7.O–O O–O 8.Qe2 d5 9.exd6 cxd6 10.Be3 Qa5 (0:05.36)
[+0.21]  d=17  1.e4 Nf6 2.e5 Nd5 3.Nc3 Nxc3 4.dxc3 Nc6 5.Nf3 g6 6.Bc4 Bg7 7.O–O O–O 8.Qe2 d5 (0:01.53)
[+0.21]  d=16  1.e4 Nf6 2.e5 Nd5 3.Nc3 Nxc3 4.dxc3 Nc6 5.Nf3 g6 6.Bc4 Bg7 7.O–O O–O 8.Qe2 d5 (0:01.30)
[+0.20]  d=15  1.e4 e5 2.Nf3 Nf6 3.d4 exd4 4.e5 Qe7 5.Be2 Ng4 6.Qxd4 h5 7.O–O Nc6 8.Qd2 (0:00.35)
[+0.20]  d=14  1.e4 e5 2.Nf3 Nf6 3.d4 exd4 4.e5 Qe7 5.Be2 Ng4 6.Qxd4 h5 7.O–O Nc6 8.Qd2 (0:00.20)
[+0.20]  d=13  1.e4 e5 2.Nf3 Nf6 3.d4 exd4 4.e5 Qe7 5.Be2 Ng4 6.Qxd4 h5 7.O–O Nc6 8.Qd2 (0:00.16)
[+0.07]  d=13  1.Nf3 Nf6 2.Nc3 d5 3.d4 Nc6 4.Bf4 Bf5 5.Nh4 Be6 (0:00.05)
[+0.14]  d=12  1.Nf3 Nf6 2.Nc3 Nc6 3.d4 d5 4.Bf4 Bf5 5.e3 (0:00.01)

Second runing  Rybka 3 1-cpu – Aquarium 3.0.4
[+0.22]  d=18  1.e4 Nf6 2.e5 Nd5 3.Nc3 Nxc3 4.dxc3 Nc6 5.Nf3 g6 6.Bc4 Bg7 7.O–O O–O 8.Qe2 d5 9.exd6 cxd6 10.Be3 Qa5 (0:05.35)
[+0.21]  d=17  1.e4 Nf6 2.e5 Nd5 3.Nc3 Nxc3 4.dxc3 Nc6 5.Nf3 g6 6.Bc4 Bg7 7.O–O O–O 8.Qe2 d5 (0:01.53)
[+0.21]  d=16  1.e4 Nf6 2.e5 Nd5 3.Nc3 Nxc3 4.dxc3 Nc6 5.Nf3 g6 6.Bc4 Bg7 7.O–O O–O 8.Qe2 d5 (0:01.30)
[+0.20]  d=15  1.e4 e5 2.Nf3 Nf6 3.d4 exd4 4.e5 Qe7 5.Be2 Ng4 6.Qxd4 h5 7.O–O Nc6 8.Qd2 (0:00.35)
[+0.20]  d=14  1.e4 e5 2.Nf3 Nf6 3.d4 exd4 4.e5 Qe7 5.Be2 Ng4 6.Qxd4 h5 7.O–O Nc6 8.Qd2 (0:00.20)
[+0.20]  d=13  1.e4 e5 2.Nf3 Nf6 3.d4 exd4 4.e5 Qe7 5.Be2 Ng4 6.Qxd4 h5 7.O–O Nc6 8.Qd2 (0:00.16)
[+0.07]  d=13  1.Nf3 Nf6 2.Nc3 d5 3.d4 Nc6 4.Bf4 Bf5 5.Nh4 Be6 (0:00.05)
[+0.14]  d=12  1.Nf3 Nf6 2.Nc3 Nc6 3.d4 d5 4.Bf4 Bf5 5.e3 (0:00.01)

First running Rybka 3 1-cpu – Shredder Classic 3
12.00   0:01   +0.14   1.Sf3 Sf6 2.Sc3 Sc6 3.d4 d5 4.Lf4 Lf5 5.e3 (71.813) 44
13.01   0:05   +0.07   1.Sf3 Sf6 2.Sc3 d5 3.d4 Sc6 4.Lf4 Lf5 5.Sh4 Le6 (235.179) 46
13.12   0:16   +0.20   1.e4 e5 2.Sf3 Sf6 3.d4 exd4 4.e5 De7 5.Le2 Sg4 6.Dxd4 h5 7.O-O Sc6 8.Dd2 (751.641) 47
14.01   0:20   +0.20   1.e4 e5 2.Sf3 Sf6 3.d4 exd4 4.e5 De7 5.Le2 Sg4 6.Dxd4 h5 7.O-O Sc6 8.Dd2 (923.183) 46
15.01   0:35   +0.20   1.e4 e5 2.Sf3 Sf6 3.d4 exd4 4.e5 De7 5.Le2 Sg4 6.Dxd4 h5 7.O-O Sc6 8.Dd2 (1.627.054) 47
16.01   1:03   +0.21   1.e4 e5 2.Sf3 Sc6 3.Lb5 Sf6 4.O-O Ld6 5.d3 O-O 6.Sbd2 De7 7.h3 a6 (2.827.990) 45
17.01   1:33   +0.21   1.e4 e5 2.Sf3 Sc6 3.Lb5 Sf6 4.O-O Ld6 5.d3 O-O 6.Sbd2 De7 7.h3 a6 (4.199.935) 45
18.01   2:37   +0.20   1.e4 e5 2.Sf3 Sc6 3.Lb5 Sf6 4.O-O Ld6 5.d3 a6 6.Lxc6 dxc6 7.Sbd2 Le6 8.Sg5 Ld7 9.h3 (7.282.291) 47

Second running Rybka 3 1-cpu – Shredder Classic 3
12.00   0:01   +0.14   1.Sf3 Sf6 2.Sc3 Sc6 3.d4 d5 4.Lf4 Lf5 5.e3 (71.813) 44
13.01   0:05   +0.07   1.Sf3 Sf6 2.Sc3 d5 3.d4 Sc6 4.Lf4 Lf5 5.Sh4 Le6 (235.179) 46
13.12   0:16   +0.20   1.e4 e5 2.Sf3 Sf6 3.d4 exd4 4.e5 De7 5.Le2 Sg4 6.Dxd4 h5 7.O-O Sc6 8.Dd2 (751.641) 47
14.01   0:20   +0.20   1.e4 e5 2.Sf3 Sf6 3.d4 exd4 4.e5 De7 5.Le2 Sg4 6.Dxd4 h5 7.O-O Sc6 8.Dd2 (923.183) 46
15.01   0:35   +0.20   1.e4 e5 2.Sf3 Sf6 3.d4 exd4 4.e5 De7 5.Le2 Sg4 6.Dxd4 h5 7.O-O Sc6 8.Dd2 (1.627.054) 47
16.01   1:02   +0.21   1.e4 e5 2.Sf3 Sc6 3.Lb5 Sf6 4.O-O Ld6 5.d3 O-O 6.Sbd2 De7 7.h3 a6 (2.827.990) 46
17.01   1:33   +0.21   1.e4 e5 2.Sf3 Sc6 3.Lb5 Sf6 4.O-O Ld6 5.d3 O-O 6.Sbd2 De7 7.h3 a6 (4.199.935) 46
18.01   2:37   +0.20   1.e4 e5 2.Sf3 Sc6 3.Lb5 Sf6 4.O-O Ld6 5.d3 a6 6.Lxc6 dxc6 7.Sbd2 Le6 8.Sg5 Ld7 9.h3 (7.282.291) 47


As you will notice, time for depth starts identical but from depth 16, 17, 18 aberration is significant. Have a look at the pv in ply 18:

Aquarium: +0.22    1.e4 Nf6 2.e5 Nd5 3.Nc3 Nxc3 4.dxc3 Nc6 5.Nf3 g6 6.Bc4 Bg7 7.O–O O–O 8.Qe2 d5
Shredder:  +0.20    1.e4 e5 2.Sf3 Sc6 3.Lb5 Sf6 4.O-O Ld6 5.d3 a6 6.Lxc6 dxc6 7.Sbd2 Le6 8.Sg5 Ld7

It seems, that the GUI and its engine-integration has a lot of impact to engine behaviour and performance.

Nonetheless I did some research on nps/time/GUI. Enclosed you will find a screenshot of my "work in progress".

Regards
Axel
Parent - - By Permanent Brain (*****) Date 2008-08-24 14:32 Edited 2008-08-24 14:35
Thanks. - I don't have these GUIs, but there is a possible problem:

The question is when the output for depth 16 for example, is presented. Shredder's 16.01 looks to me as if that was the pv after the first branch at depth 16 was finished. If d=16 means that depth 16 was completely finished (all branches), than this is not the same amount of analysis work. That could explain some very big differences.

Fritz typically starts displaying depth x PVs as soon as a branch was finished at depth x, but will usually update the PV before the next ply is started...
Parent - - By Axel Caro (**) Date 2008-08-24 14:49

> The question is when the output for depth 16 for example, is presented. Shredder's 16.01 looks to me as if that was the pv after the first branch at depth 16 was finished. If d=16 means that depth 16 was completely finished (all branches), than this is not the same amount of analysis work. That could explain some very big differences.


That's it! ;-)

Therefor I did not compare the pv output, but the depth output "above".Unfortunately Aquarium does not show depth besides the pv...
Parent - - By Dadi Jonsson (Silver) Date 2008-08-24 16:06

> Aquarium does not show depth besides the pv


Look at the "d=15" in front of the first move of the PV.
Parent - - By Axel Caro (**) Date 2008-08-24 16:10
Yes, that's what I meant: no depth display besides the pv. Display time of depth varies from implementation to implementation (have a look at the recent screenshot of Shredder and Aquarium GUI) - hence nps is best for evaluating speed-differences.
Parent - - By Dadi Jonsson (Silver) Date 2008-08-24 16:17 Edited 2008-08-25 11:22

> besides the pv


Oh, I see.

> hence nps is best for evaluating speed-differences


It depends on how exactly the "nps" is calculated. I guess your assumption is that it's the total number of nodes since the search started divided by the total length of the analysis time. If that assumption is not correct, then it may not be such a good measure.

Edit: I tested this myself with Rybka 3 from the command line. I only tested one position, but in that case the reported NPS was consistently 2.4% higher than the number calculated from time and nodes.

Edit 2: I see now that BB had already found the same problem with Rybka 2.3.2a: http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=57384#pid57384
Parent - - By Axel Caro (**) Date 2008-08-24 21:56
Hm, interesting. Question is, which is the accurate calculated one - nps, time or nodes in total? ;-)

For interest I compared solution times for a specific test position comparing different GUI. It's only one position, differences are obvious though. Fortunately results are constantly for every GUI. Sampled search seems to lead to higher solution time (in this specific case, in others it might be the opposite).

FEN
r1bq1rk1/pp4bp/2np4/2p1p1p1/P1N1P3/1P1P1NP1/1BP1QPKP/1R3R2 b - - 0 1


Fritz 11 (with latest update from August 21st) behaves similar to Aquarium, according to sampled search, I suppose.

Shredder Classic 3 and Fritz 9 react identical: same eval, same solution-time, same nodes in total (exactly).

You may have a look at the results enclosed.
Parent - - By SummerKnight (**) Date 2008-08-24 22:20
What is the best move?
Parent - - By Axel Caro (**) Date 2008-08-25 06:57
Sorry I forgot (it was mentioned in the excel sheet, but I removed it before posting). Her it is:

[Event "Blitz:4'+2""]
[Site "Hamburg"]
[Date "????.??.??"]
[Round "?"]
[White "Neue Partie"]
[Black "?"]
[Result "*"]
[SetUp "1"]
[FEN "r1bq1rk1/pp4bp/2np4/2p1p1p1/P1N1P3/1P1P1NP1/1BP1QPKP/1R3R2 b - - 0 1"]

1... Bh3+ 2. Kxh3 g4+ 3. Kxg4 Qd7+ 4. Kh4 Bf6+ 5. Ng5 Bxg5+ 6. Kxg5 Rf6 7. Qh5
Qg7+ 8. Kh4 Rh6 9. f4 Rxh5+ 10. Kxh5 Qg6+ 11. Kh4 Qh6+ 12. Kg4 Qxh2 *
Parent - - By Felix Kling (Gold) Date 2008-08-25 13:37
Actually Vas showed me that position in Mainz as an example where you can see quite nice what the engine is doing with sampled search :)
Parent - - By Axel Caro (**) Date 2008-08-25 14:05
Maybe we will find this example within documentation, which Vas recently advised.
So far as I understood, sampled search is not such a big technical impact in computerchess (engine-strength / paying-quality is not affected), but it offers some nice comfort to the user.  Therefore it is welcome!
Parent - By Vasik Rajlich (Silver) Date 2008-08-27 14:31
Yes, this is a perfect position for sampled search. When the engine switches from .. Bg4 to .. Bh3, that first real search of .. Bh3 is huge.

Vas
Parent - - By Uly (Gold) Date 2008-08-24 16:35
Sampled Search changes the tree in some way, so Rybka could find the best move at ply 14 instead of ply 16, this makes depth not a good measurement of speed. I think that "time to solution" (when the best move of a position is known) would be a better measure.
Parent - - By Axel Caro (**) Date 2008-08-24 17:38
Yes, I agree. Maybe I will rerun the GUI-speed-test with a defined test position (clear solution).
Parent - - By Dadi Jonsson (Silver) Date 2008-08-24 17:42
I'm not sure that anything conclusive will come out of that. In some positions sampled search needs more time, in others it's faster.
Parent - - By Uly (Gold) Date 2008-08-24 17:59
Then we'd need to test several positions. If after, say, 100 positions, sampled search is slower on 80 of them, I'd consider switching it off.
Parent - - By ernest (****) Date 2008-08-24 18:12

> I'd consider switching it off.


Not sure you can do that (yet)... :-)
Parent - - By Uly (Gold) Date 2008-08-24 18:14
I can "switch it off" by using a GUI that doesn't support sampled search :)

Seriously, this is a glitch, Vas was surprised because he meant both analysis to be identical.
Parent - - By Kapaun (****) Date 2008-08-24 18:24 Edited 2008-08-24 18:32
He obviously didn't check it with any GUI till the beta test began a week before release - very bad idea... :(
Morbus Non-checkia seems to be a widespread illness amongst programmers, though. I came across it very frequently in my time. Seems to be some sort of hubris: I programmed it so it must run...
Parent - - By Uly (Gold) Date 2008-08-25 05:26
Yeah, when he said "There won't be updates for Rybka 3", I thought it meant "There won't be bugs in Rybka 3" not "We won't fix the bugs" :(
Parent - - By Kapaun (****) Date 2008-08-25 09:19
Actually I thought he meant: "There will be bugs and we will fix them, but I will show some agitated melancholia before."
Parent - By Uly (Gold) Date 2008-08-25 13:52
Hopefully, it's just a case of "I didn't want to promise anything but here's your update!"
Parent - - By Axel Caro (**) Date 2008-08-25 11:18

> I assume that the speed differences have to do with the output the interfaces display


Hi Mike,

chessbase (Mathias Feist) just confirmed display-issues for strange nps values in Fritz 11 GUI:

http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=99650#pid99650
Parent - - By Permanent Brain (*****) Date 2008-08-25 11:39 Edited 2008-08-25 11:45
Again, that acknowledges that speed comparisons based on node rate are invalid, and depth comparisons should be done by time for depth x.

I would NEVER have had the idea to compare speed by node rate. If node rate causes so many misunderstandings, my proposal would be that node rate is simply not displayed anymore, in the future. It is useless anyway, and it is only good for causing misunderstandings, like: Why is Rybka stronger but displaying much smaller kN/s than my Fritz, Junior, Blah...

So please lets either get rid of node rate at all, or lets define a very clear standard! I am tired about all this stupid node rate talk, faked node rates, etc.etc. This just cannot be the future of computer chess. Let's get more serious please!!
Parent - By Roland Rösler (****) Date 2008-08-25 23:12
If node rate causes so many misunderstandings, my proposal would be that node rate is simply not displayed anymore, in the future.

If depth causes so many misunderstandings, my proposal would be that depth is simply not displayed anymore, in the future. :-)
Parent - - By Vempele (Silver) Date 2008-08-24 07:23
You forgot to test Rybka without a GUI (with and without sampled search). The relevant commands are:
uci // always start with this
cbr3 // turn on sampled search
position fen whatever // the starting position is used if Rybka doesn't get this command
go infinite // or go depth x or go movetime milliseconds

>In addition better multicore scaling of Rybka 3 becomes evident. She multiplies doubling of cores by 1.7, where Rybka 2.3.2a scaled by 1.6 on average.


That's meaningless because she actually does multiply the 1-CPU node count - changing the factor is trivial.

Naturally, you measured the CPU usage of the interfaces, right?
Parent - - By ernest (****) Date 2008-08-24 14:14

> You forgot to test Rybka without a GUI (with and without sampled search). The relevant commands are:
> uci // always start with this
> cbr3 // turn on sampled search
>


Teemu, how do you set the hash?
I tried: setoption name hash value 512  but it stays at default 32
Also, when I type cbr3 (to turn on sampled search), it invokes the randomizer (no word of sampled search):
cbr3
option name cbr3_randomizer_threshold type spin default 8 min 1 max 512
option name cbr3_randomizer_mode type check default false
option name cbr3_randomizer_reset type button
cbr3ok


Where is the sampled search???
Parent - - By Vempele (Silver) Date 2008-08-24 14:21 Edited 2008-08-24 14:25
cbr3 asks if the engine supports ChessBaseRybka3 features (I think). Sampled Search is implied in cbr3ok, the randomizer settings aren't.

You'll see info sample messages during search.

> I tried: setoption name hash value 512  but it stays at default 32


That's the correct syntax. It doesn't work before you send uci, I don't know what else could make it not work.
Parent - By ernest (****) Date 2008-08-24 16:25
Thanks, Teemu!

> Sampled Search is implied in cbr3ok

Yes indeed, with cbr3 I obtain the same result as with Aquarium (that is, with sampled search!) :-)

> setoption name hash value 512, but it stays at default 32
> That's the correct syntax.

Sorry, was a glitch... Now it works as it should!
Parent - - By Axel Caro (**) Date 2008-08-24 14:19

> You forgot to test Rybka without a GUI (with and without sampled search). The relevant commands are:


uci // always start with this
cbr3 // turn on sampled search
position fen whatever // the starting position is used if Rybka doesn't get this command
go infinite // or go depth x or go movetime milliseconds

I have no clue where to put in the commands you mentioned and where I will get the neccessary output...

> Naturally, you measured the CPU usage of the interfaces, right?


I did:

GUI                         cpu-usage
Aquarium:               0-1 %
Shredder Classic:    0 %
Fritz 11:                  0%
Fritz 9:                    0%
Arena 1.99b5          0%
Parent - - By Vempele (Silver) Date 2008-08-24 14:23

>I have no clue where to put in the commands you mentioned and where I will get the neccessary output...


Double-click on Rybka 1-cpu.exe and you'll get a console window.
Parent - - By Axel Caro (**) Date 2008-08-24 18:27
I see, thanks.

nps are 46.468 with and without sampled search. That's similar to Aquarium output.
Parent - - By ernest (****) Date 2008-08-24 18:53
> with and without sampled search

Not true!

The result is different with and without sampled search (and nps are similar but not the same)
Parent - - By Axel Caro (**) Date 2008-08-24 19:32

> The result is different with and without sampled search


A) command UCI -> go infinite ->              

a1: depth 16 time 57175 nps 46329
a2: depth 16 time 56988 nps 46481

B) command UCI -> cbr3 -> go infinite ->  

b1: depth 16 time 62838 nps 48463
b2: depth 16 time 55366 nps 47388

Too much fluctiation and too less control to validate or invalidate any implications.

> (and nps are similar but not the same)


That's what I said: "That's similar to Aquarium output." I can't see any detailed nps output in Aquarium GUI (just 47 nps or so, no digits).
Parent - By ernest (****) Date 2008-08-25 12:09
> A) command UCI -> go infinite ->

Rybka 3 1-cpu 32-bit
uci
setoption name hash value 512
go infinite

I obtain (completely reproducible):
info depth 16 score cp 19 time 59735 nodes 1867475 nps 32012 pv e2e4 b8c6 g1f3 g
8f6 e4e5 f6d5 c2c4 d5b6 c4c5 b6d5 b1c3 d5c3 d2c3 e7e6 b2b4 b7b6


> B) command UCI -> cbr3 -> go infinite ->

Rybka 3 1-cpu 32-bit
uci
setoption name hash value 512
cbr3
go infinite

I obtain (completely reproducible):
info depth 16 score cp 19 time 70985 nodes 2220482 nps 32031 pv e2e4 b8c6 g1f3 g
8f6 e4e5 f6d5 c2c4 d5b6 c4c5 b6d5 b1c3 d5c3 d2c3 e7e6 b2b4 b7b6


With cbr3 I obtain exactly the same as with Aquarium (I don't have the ChessBase Rybka)
Note: here at depth 16 the pv is the same without or with cbr3; this is no longer the case for depth 17, 18...
Parent - - By Razor (****) Date 2008-08-25 07:11
Hi Axle,

Do your results include time and outcome?  If they do could you publish please - e.g.,

GUI Name      Engine Name  PGN (Position)    Key Move Found     Time (time to find key move)

If you haven't used this approach I recommend it to you; it removes some of the data noise your current results have and deals with data that should be common to all, i.e., start position, outcome, time.   If you haven't done this test then save time by using the same engine - Rybka 3.
Parent - By Axel Caro (**) Date 2008-08-25 10:20
Testing refers to starting position, measuring the nps exactly after 1 minute, as mentioned in the screenshot.

I did another test, referring to a specific test positon, as described here:

http://www.rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=99464#pid99464
Parent - - By Vasik Rajlich (Silver) Date 2008-08-25 12:33
Please note that in the big scheme of things these sorts of differences are minor.

Vas
Parent - - By Axel Caro (**) Date 2008-08-25 13:17
Yes, I recognized it, nps are not as interesting as discussions imply...

Comparing nps on different GUI with different engines leads to no usable results. It's only worth comparing same engine on different hardware - like PC to PocketPC. ;-)

By the way, I now own the Pocket Fritz 3 with the Hiarcs 12.1 engine (beside CE Board and Pocketgrandmaster 2.1 and some other PPC chess stuff). If you need your mobile software beta-tested on different hardware, I won't refuse testing it on my MDA Pro PPC.

If you don't mind, one off-topic question: I noticed, that your "flag" changed from hungary to poland. Born in Cleveland, living in Budapest, Czech-American citizen by birth, now moving to Warsaw... A cosmopolitan at it's best! Did you change your citizenchip to Polish now?

Best regards,
Axel
Parent - - By Vasik Rajlich (Silver) Date 2008-08-27 14:33

> If you need your mobile software beta-tested on different hardware, I won't refuse testing it on my MDA Pro PPC.


Ok, thanks for the offer :)

> If you don't mind, one off-topic question: I noticed, that your "flag" changed from hungary to poland. Born in Cleveland, living in Budapest, Czech-American citizen by birth, now moving to Warsaw... A cosmopolitan at it's best! Did you change your citizenchip to Polish now?


I'm staying a US citizen. This certainly makes watching the Olympics more fun :)

Vas
Parent - By Axel Caro (**) Date 2008-08-28 04:44

> I'm staying a US citizen. This certainly makes watching the Olympics more fun :-)


...then I wonder, if you have ever considered to achieve the Chinese citizenship! ;-)

Axel
Parent - - By Permanent Brain (*****) Date 2008-08-25 15:18
In terms of engine performance, these differences are certainly NOT minor. So, this issue has to be researched and cleared up, by the responsible persons (=interface and engine programmers). The customers are expecting clear results of serious work about this problem, not sloppy desinterested talk.

Are you interested in providing a QUALITY product? If not, say so and the (possible) customers will draw their conclusions. But you should not try to give the impression that you provide a serious product while in fact, you don't care about nothing and you simply don't give a shit, not matter what. If that's your approach - and all I can see indicates it - SAY SO, and that will be fair. But don't try to act as if you want to create a serious commercial quality product if you simply don't care. Ok?
Up Topic Rybka Support & Discussion / Rybka Discussion / GUI performance: Arena top, Fritz flop, Aquarium competitive
1 2 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill