Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / CEGT 40/20 list uploaded by Michael Koppel
- - By Heinz van Kempen (***) Date 2008-08-10 10:52 Edited 2008-08-10 11:01
Hi all :-),

http://www.husvankempen.de/nunn/40_40%20Rating%20List/40_40%20All%20Versions/rangliste.html

Downloads will follow.

Because we could only test since Wednesday testers concentrated on the Rybka 3 32-bit single version first. More games will come for the x64 4CPU Rybka 3 surely with update next week.

Rybka 3 x64 on only 2 CPU´s is 121 ELO ahead of Rybka 2.3.2a x64 on 4 CPU´s but here still few games.

Werner will report with more detail later in the Computer Chess Subforum here later.

Best Regards
Heinz
Parent - By Axel Caro (**) Date 2008-08-10 13:01
Hello Heinz,

awesome! That means, Rybka-3 at 4-cpu may reach appr. 3250 ELO points on 40/20!?

That would be nearly +300 ELO in comparison to HIARCS 12 MP-4-CPU!

Incredible...

Best regards and thanks a lot for accurate testing,

Axel
Parent - - By Roland Rösler (****) Date 2008-08-10 16:41
Rybka 3 w32 sp "only" +84 Elo better than R232a (3048 vs 2964).
It´s like Larry suspect; R3 sp plays almost against opponents on 2 or even 4 cores. This let drop the Elo increase considerably.
Parent - - By Heinz van Kempen (***) Date 2008-08-10 16:47
Hi Roland :-),

CEGT 40/20 testers only had 3,5 days for the matches so far. I am sure that they will also run against Single CPU opponents over the next week or maybe they are fed up with seeing 45,5:4,5 results then? :-)

Best Regards
Heinz
Parent - By Roland Rösler (****) Date 2008-08-10 16:55
Hi Heinz,

no accusation from my side! :-) It was only my try to explain it.
Parent - - By lkaufman (*****) Date 2008-08-10 17:37
My final data on this question showed a sixty point spread between the blitz rating of Rybka 3 1 cpu 64 bit against six engines on 4 cpus (3090 CEGT scale) and the rating against the same six engines on 1 cpu (3150 CEGT scale). More than a hundred games for each of the twelve pairings. It seems you may be seeing this too. This needs an explanation; any ideas?
Parent - By buffos (Silver) Date 2008-08-10 18:08
Probably in the first plies the big differences are hidden and higher pc power gets the first crucial plies faster.
I think the slower time controls will show the real difference in chess power and will be about the same
Parent - - By lkaufman (*****) Date 2008-08-10 18:06
Looking at your blitz results, you show gains over 2.3.2a of 108 on 1 cpu (both 32 bit), 153 on 2 cores, and 166 on four. Allowing for some statistical "noise", we see a steady climb with more cores, which is completely in line with expectations due to the improved scaling. It's nice to see that confirmed. Your results seem to be even better than my own, although perhaps within the margin of error of my results. I did get better results when I switched from HS220 to Noomen200 as a testset; maybe Rybka plays a bit better in normal openings than in strange ones. I also note that your figures for the special versions average about 25 Elo worse than the default (for the well tested 4 cpu case), which is almost exactly my forecast before any testing even began. The only remaining question in my mind is whether the special versions might do better at longer time controls.
Parent - - By Heinz van Kempen (***) Date 2008-08-10 19:01
Hi Larry :-),

yes, your predictions regarding scaling, other Rybka 3 versions and general results were really good.

Regarding the long time controls 40/120 repeated we can surely take good conclusions around Christmas :-). I started here almost 12 hours ago and collected six games so far (on four quads combined). So the first games 40/120 real time control for replay and download will be only available tomorrow.

Best Wishes
Heinz
Parent - - By NATIONAL12 (Gold) Date 2008-08-10 19:14
i would like to see you testing on 8 cores at 40/120,a number of us have now got 8 core comps.i dont think R3 would be ahead of Zappa by anyware near these ELO ratings,although i agree it woud still be ahead.
Parent - - By Heinz van Kempen (***) Date 2008-08-10 19:22
[deleted]
Parent - By NATIONAL12 (Gold) Date 2008-08-10 19:24
and Zappa,well when better stuff comes out i will build it.
Parent - - By Heinz van Kempen (***) Date 2008-08-10 19:24 Edited 2008-08-10 19:32
Hi :-),

you even thought that our tests with 40/400 repeated and real time control were "Micky Mouse", although those are by far higher quality compared to 40/120 on octal.

My guess is that there is no need for octal hype at this point and they will be as cheap next year as quads are now. We should also wait for real 8 cores instead of what we have now. Frank Quisinsky who is a known expert for computer chess and hardware guessed from own experience that ELO gains for octal over quad are minimal +15 ELO. With good scaling for Rybka 3 this could be more.

Instead of testing on one expensive octal at this point I prefer to have more quads for better output and more reliable amount of games and most testers not only from our team think the same.

From my 40/400 tests I am sure that you are completely wrong. Rybka 3 would crush Zappa with long time controls by a large margin on octal.

Best Regards
Heinz
Parent - By NATIONAL12 (Gold) Date 2008-08-10 19:29
no way was your 40/400 micky mouse.it is just that Zappa scales much better with 8 core than R2.so results were not comparable.of course now R3 is out this will change.
Parent - - By lkaufman (*****) Date 2008-08-10 19:59
I think that the +15 Elo estimate for octal over quad is ridiculous. I ran 232 games of bullet chess between Rybka 3 on eight cores vs. four cores (on same machine), and got +59=157-16 for +65 Elo. I know that self-testing inflates rating gains, but not that much! Maybe he was specifically referring to Rybka 2.32a, which does get relatively little out of octal.
Parent - - By Heinz van Kempen (***) Date 2008-08-10 20:07
Hi Larry :-),

I should have mentioned that this guess was made by Frank in CSS Forum some months ago. Like you see I also wrote that with better scaling by Rybka 3 this might be more or even much more. We will see when hardware will be again cheaper and testers feel inclined to upgrade or maybe you could give some test results not relying on self-testing.

Best Regards
Heinz
Parent - - By lkaufman (*****) Date 2008-08-10 20:27
     Another result I got which is not self-testing but family-testing is that on octal, R3 beat 2.32a by 205 Elo, but on quad, it was only 147 Elo (after 135 games), other conditions the same. Of course scaling cannot be 58 Elo better from quad to octal comparing those versions, some of this is surely sample error, but maybe I seriously underestimated how much scaling had improved when I wrote about how much better R3 did against 2.32 when directly paired than when tested against other engines. Now it seems that on quads there was no real difference, which is reasonable as R3 and R2.32a are hardly much more alike than R3 and some other programs might be.
     But one question: was that estimate in CSS Forum specific to Rybka or about engines in general? Some engines, including 2.32a, did not scale well on eight cores, but some did.
Parent - - By Heinz van Kempen (***) Date 2008-08-10 20:50
   But one question: was that estimate in CSS Forum specific to Rybka or about engines in general?

Hi Larry :-),

it was meant as average for top engines, what of course is also interesting for people who want that all will profit a lot what seems to be not the case for the majority. But clearly this all has to be proven. Proof is always better than guess and in some fora we have still people taking conclusions from very few games.

Best Regards
Heinz
Parent - By lkaufman (*****) Date 2008-08-11 02:27
What would you say is the average elo gain from one core to two, and from two cores to four? Of course I could do the math myself, but there are questions as to whether to treat each version as a separate entity or to count all versions of Fritz as one, all Rybkas as one, etc. Also the time controls are a factor. Probably the gain from four to eight cores is less than, but not dramatically less than, the gain from two to four.
Parent - - By Hamlet (**) Date 2008-08-10 22:07
Hi Larry,
lets think about this. OK.
carefully. octal is less than 2 times faster than quad. Do you say that doubling times (around one ply) is worth of 65 elo?
And using what time limits. Like 14ply vs. 15ply or 25ply vs. 24ply? big difference i think. i would like to say that in bliz octal is relatively stronger measured in elo points than in 40/400.

regards,
hamlet 
Parent - - By lkaufman (*****) Date 2008-08-11 02:59
I agree that 65 Elo is too high for 8 cores vs. 4; self-play inflates this number, and these were blitz games which as you say also gives higher values for a doubling than you would get at 40/120 (I don't even talk about 40/400, that's not a time limit used in real chess events). But the difference between blitz and 40/120 is not so large as you suggest, I think. My own results for Rybka 3 on one core show that the six programs I tested her against gained an average of 145 Elo from one to four cores (in blitz), or 72.5 per doubling of cores. Going from four to eight should produce less, maybe 50-60. At 40/120' this might drop to 35-40, I would say.
Parent - By Hamlet (**) Date 2008-08-11 22:15
Ok. This actually very interesting question for us cc players because most of players don't use all time and empirical results seems to implie that doubling the time wil improve results significantly. 30 points may be difference between first and second place in the tournament. I have to do good notes from my next high level tournament.
hamlet
Parent - - By lkaufman (*****) Date 2008-08-10 19:54
I think that there is little difference in scaling now between Rybka 3 and Zappa, maybe Rybka 3 could even be better. So there is no reason to think that results on eight cores would be any different than on four cores at a somewhat slower time control.
Parent - By NATIONAL12 (Gold) Date 2008-08-10 20:00
i expect you are right,i dont get R3 till tomorrow,i hope.i am unable to do any testing myself as i am involved in match against hamsters.

liked your write up on chessbase.
Parent - By lkaufman (*****) Date 2008-08-10 19:52
Long before that you should have enough 40/20 games to see if there is a strong trend for one of the special versions to get better with depth (comparing to blitz).
Parent - By X3 (**) Date 2008-08-11 08:54
Hi!

The latest download at 40/20 (10/8 2008) are from 10/2 2008.

Bertil
Up Topic Rybka Support & Discussion / Rybka Discussion / CEGT 40/20 list uploaded by Michael Koppel

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill