I decided to get some data on Rybka 3 beta SP (1 core) performance. Overnight I ran her against Deep Fritz 10.1 and Deep Shredder 11 (both running on quads), at 1'+1". Currently Rybka 3 sp is beating Deep Shredder 11 by 65-21 with 49 draws, and leads Deep Fritz 10.1 by an even bigger margin, 86-18 with 33 draws. Averaging the two, this works out to projected ratings of 3177 CCRL and 3100 CEGT for Rybka 3 SP. These are gains of 108 and 76 respectively over 2.3.2a, averaging 92. So based on all the data so far, I can estimate the gains as 90 on 1 core, 100 on 2 cores (by interpolation), 110 on 4 cores, and 125 on 8 cores. As Rybka 3 is still not finalized, these numbers could still be improved.
![[il]](/mwf/v12/flags/il.png "Israel")
Note that CEGT and CCRL use 40/4 or 40/40 time control that is different type of time control than the time control that you use.
It is possible that rybka3 has relative better time management for one of these time controls.
Did you test rybka3 at 40 moves/x minutes time control?
It is possible that rybka3 has relative better time management for one of these time controls.
Did you test rybka3 at 40 moves/x minutes time control?
I did run one overnight test of this. On my octal machine, Rybka 3 currently leads Rybka 2.3.2a in a direct match at 40/4 by 120 Elo after 48 games. It's a small sample but the result is pretty consistent with the results of the same test at 1'+1". Anyway I don't think that Vas has worked on time control much if at all since Rybka 2.32a, though I could be mistaken.
![[no]](/mwf/v12/flags/no.png "Norway")
There was talks about improved time control before the rapid world championships, at least -- I think someone mentioned that it was rewritten, but that Rybka ran with the old time management in the tournament.
/* Steinar */
/* Steinar */
Perhaps he did a bit of work on it back then, but I don't think it was changed radically. My impression is that Rybka 3 still moves too fast in general, certainly faster than the opponents in these matches I've been running. In fact any time I glance at a game in progress, Rybka usually has way more time remaining. I think it's too late to change this for Rybka 3; at least we have one likely rating gain "in the bag" for Rybka 4!
Note that it is not clear that moving faster than the opponent means bad time management.
If you play games with ponder on then it may cause the opponent to play worse and blunder because of having less time to ponder and even without pondering nothing is clear because you may earn by playing a better endgame when you have more time.
Uri
If you play games with ponder on then it may cause the opponent to play worse and blunder because of having less time to ponder and even without pondering nothing is clear because you may earn by playing a better endgame when you have more time.
Uri
![[us]](/mwf/v12/flags/us.png "United States")
Would you say that it is better for weaker programs to perhaps take slightly more time than stronger programs, the other way around, or neither?
![[de]](/mwf/v12/flags/de.png "Germany")
I'd expect it would be good for a weaker program to use more time especially in the middle game. And if it manages to survive this part then it must keep the eyes shut and run through the hell of the endgame, hoping for the best. :->
Well, my point in asking is that one could also consider the fact that the stronger program is thinking on the weaker program's time, and if there is a ponder hit, there is a substantial advantage.
Actually, no. The stronger engine may be able to look deeper, and it will use this while the weaker one is thinking. But first of all it won't always be pondering on the right move. And then, the more plies the stronger engine achieves, the longer it will take to get another ply. So I'd expect the gain from pondering being not much higher or rather even lower than the gain the weak engine will get from taking it's time. This may lead us to the opinion that in fact the stronger engine is right to use less time...
Are we talking " Hash Memory " here? I would say, put Rybka 32 or no more than 64 at best. If this is what you are discussing.
![[tr]](/mwf/v12/flags/tr.png "Turkey")
Moving faster than all strong chess engines (most of which are tuned for
different time management protocols) means bad time management,
unless you made enough tests with different time management protocols.
If you play games with ponder on, then playing very fast can cause Rybka
to play worse and blunder.
Generally, an earlier move of a game is more important than a later move.
Earlier moves effect the rest of the game to the end. Later moves also effect the rest of the game,
but in this case the rest of the game is smaller.
You play a game against a strong engine, and you are allowed to use R3.
You have to choose 4th best move in one of your moves.
When would you decide to make that intentional mistake ? At the opening,
or later ?
Most of Rybka users - as far as I saw in this forum - think that Rybka is
playing faster than optimal.
different time management protocols) means bad time management,
unless you made enough tests with different time management protocols.
If you play games with ponder on, then playing very fast can cause Rybka
to play worse and blunder.
Generally, an earlier move of a game is more important than a later move.
Earlier moves effect the rest of the game to the end. Later moves also effect the rest of the game,
but in this case the rest of the game is smaller.
You play a game against a strong engine, and you are allowed to use R3.
You have to choose 4th best move in one of your moves.
When would you decide to make that intentional mistake ? At the opening,
or later ?
Most of Rybka users - as far as I saw in this forum - think that Rybka is
playing faster than optimal.
I did not test and I do not know.
Nothing is obvious.
You assume that you are going to play inferior move but there is a positive probablity that using less time in the opening may
cause you to play the same moves faster.
Things are dependent on the program.
If you take an extreme example of a program that plays perfectly in the middle game even without much time but needs time to play perfectly in the endgame then it is clear that it is better to save time to the endgame.
Uri
Nothing is obvious.
You assume that you are going to play inferior move but there is a positive probablity that using less time in the opening may
cause you to play the same moves faster.
Things are dependent on the program.
If you take an extreme example of a program that plays perfectly in the middle game even without much time but needs time to play perfectly in the endgame then it is clear that it is better to save time to the endgame.
Uri
I believe the testing organizations all use ponder off. I agree that playing faster may or may not be wise; my point is that many people feel that Rybka 2.3.2a moves too fast in general, and apparently this has not been changed in Rybka 3. My feeling is that Rybka does move too fast, but perhaps some other programs move too slowly.
>I believe the testing organizations all use ponder off.
SSDF uses separate computers for each match with ponder=on. They have had out-dated hardware for awhile, but as of this year updated to Q6600's and have already played a large number of test matches. They are awaiting Rybka 3 before publishing their revised rating list. Last I checked, Hiarcs 12 leads on the Q6600.
Clearly correct time use differs based on ponder on or ponder off. This might be something for us to work on for Rybka 4.
![[be]](/mwf/v12/flags/be.png "Belgium")
> many people feel that Rybka 2.3.2a moves too fast in general, and apparently this has not been changed in Rybka 3.
That's why is asked for such parameter here : http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=48656;hl=parameter
>I don't think that Vas has worked on time control
Then he should first or then, as rybka time management was certainly inferior to other engines
I have tried the settings Rate of play: slow, and ultraslow, and didn't notice time problems. I used that even together with small emergency time buffer.
In the forum match I have been using Rybka, Hiarcs and zappa with deep position analysis in CB GUi. Rybka always plays more quickly than Zappa and Hiarcs which take about the same amount of time to finish!?
hmm, why don't you use IDeA? Fritzie's dpa is so limited...
Why don't you do a comparison, Felix, Old boy! (hehehe) :-)
I'm against comparing my posts :)
Well, in that case, your statement lacks creditability and has no substance to it. Where is your documentation, dude?
The only man that can dig you out of this one is, Dadi, and judging from his earlier response, it doesn't look good for you.
![[cr]](/mwf/v12/flags/cr.png "Costa Rica")
''as rybka time management was certainly inferior to other engines''
Yes it was clearly inferior and Vas knows that well. He promised that he would make changes and that Rybka 3 would have a much better time control management. This is important, since with an appropriate handling of time Rybka would see a bit deeper and will play stronger.
I remember many games - blitz games - where Rybka always had a lot of available time at the end - and of course this was not very pleasant to see; I also remember that Vas said that it was because Rybka had this time reserved to use in critical parts of the game, but indeed, that this would change with the following version.
We need to wait a bit more time to hear Vas last confirmation on this point.
Regards,
Gaмßito.
Yes it was clearly inferior and Vas knows that well. He promised that he would make changes and that Rybka 3 would have a much better time control management. This is important, since with an appropriate handling of time Rybka would see a bit deeper and will play stronger.
I remember many games - blitz games - where Rybka always had a lot of available time at the end - and of course this was not very pleasant to see; I also remember that Vas said that it was because Rybka had this time reserved to use in critical parts of the game, but indeed, that this would change with the following version.
We need to wait a bit more time to hear Vas last confirmation on this point.
Regards,
Gaмßito.
Well, it's only important if you actually play engine-engine games. For someone using Rybka for analysis, it's largely irrelevant.
/* Steinar */
/* Steinar */
It's also important if we play humans. It might make the difference between a 3210 performance and a 3200 performance!
All of these prelimanary results are unbelievable as the two engines you ran against are by no means pushovers.The remarkable thing is Rybka ran on one core and the others on 4.You guys are teasing us with all of this testing---I want one NOW!Keep feeding us info.So then after the release of 3.0 she will lead the pack by about 200 ELO?
My best result so far (rating-wise) was for Rybka 3 beta on octal (contempt set to 40) vs. Hiarcs 10 (which is only single core). It's a mismatch, but the score was 87-3 (one win and four draws for Hiarcs). Since Hiarcs 10 has ratings on the lists about the same as Kasparov, Anand, and Kramnik, does this mean Rybka 3 could beat them by 87-3? Well, I don't really believe this, but it's an interesting statistic.
You mean same rating as Carlsen, Ivanchuk and Morozevich? :-)
I think you should show us all the one Hiarcs 10 win in a separate thread with the title "Hiarcs 10 Rybka 3 danger?" :-)
Fantastic! This seems almost a new dimension.
Although, I found CCRL blitz results for single Rybka 2.3.2a 32-bit, winning against Fritz 10 (2 CPU) 13.5-6.5 and against Zap!Zanzibar (2 CPU) 14-6. But that were "overperformances" of 51 and 70 Elo respectively, and only 20 games each.
The CCRL 40/40 results contain some match wins of single Rybka 2.3.2a 64-bit against quads, too:
http://tinyurl.com/6qzywv
Although, I found CCRL blitz results for single Rybka 2.3.2a 32-bit, winning against Fritz 10 (2 CPU) 13.5-6.5 and against Zap!Zanzibar (2 CPU) 14-6. But that were "overperformances" of 51 and 70 Elo respectively, and only 20 games each.
The CCRL 40/40 results contain some match wins of single Rybka 2.3.2a 64-bit against quads, too:
http://tinyurl.com/6qzywv
Interesting--so Rybka 3 1 CPU would probably lose against Rybka 2.3.2a 4 CPU in a direct match, but these results are consistent with it being above it on the rating lists. I remember you saying before that something like this might happen because of the slowdown (hurting against Rybka 2.3.2a) and the increased knowledge (helping more greatly against other programs).
Yes, that was and is my explanation, but it is complicated by the fact (as pointed out to me by Vas) that apparently Rybka 3 is not really "slower" except in nodes per second because the search improvements have apparently roughly cancelled out the slowdown on average. I need to rerun the Rybka 3 1 core vs 2.3.2 4 core match with the latest version to make sure that Rybka 3 still loses; if so this looks like a fruitful area to explore.
Maybe you can test it with contempt=0 (like in the first match) and with contempt=15 (the new default?).
The contempt thing could be interesting on Playchess, where the default contempt would clearly be best against non-Rybka opponents, but wouldn't it be logical that contempt = 0 would be best against opponents who are also using Rybka?
For sure. But the problem is, you have only one default contempt. And all what I hear from Larry is that contempt=15 will be the new default. So you have to change the settings to play (or analyse) with contempt=0. I don´t like this, but I think, this will be reality with Rybka 3.
For analysis mode, I believe the default will be zero.
Can Aqua make contempt change automatically to predefined values depending on the opponent, mode, etc ?
![[fi]](/mwf/v12/flags/fi.png "Finland")
>Can Aqua make contempt change automatically to predefined values depending on the opponent
No.
>, mode
Rybka will make the change automatically when UCI_AnalyseMode is set, I think.
![[au]](/mwf/v12/flags/au.png "Australia")
>but wouldn't it be logical that contempt = 0 would be best against opponents who are also using Rybka?
To use the curt boxing analogy:
contempt=0 is for when you are in a 15-round championship fight, and contempt=15 is for when you want to beat up the drunks. :)
I did rerun it today, with contempt at zero (since Rybka 3 on 1 core should treat Rybka 2.32.a on 4 cores as an equal), and still Rybka 3 loses by about 50 Elo as of right now. With contempt set it should lose even worse. It's very interesting; on 4 cores compared to 1 core Rybka 3 beta only gained 68.6 Elo against Deep Fritz 10.1, she gained 130.6 against Deep Shredder 11 (which is said to have incorporated much of Rybka), and the difference against Rybka 2.32a is about 190 Elo! Eventually I'll solve this mystery.
>Deep Shredder 11 (which is said to have incorporated much of Rybka)
This is interesting and I have not heard this before--what do you mean in this sense?
I've heard that some of Rybka's eval and perhaps also search is used in Deep Shredder 11, but I have no direct knowledge about this.
![[nl]](/mwf/v12/flags/nl.png "Netherlands")
C'mon Larry, lets not forget about the code origins of Rybka. If SMK would have done what you are implying here Shredder11 would have surpassed Rybka. Some people have their dignity you know.
I'm not saying Shredder 11 is a Rybka clone, just that it (reportedly) used many ideas from Rybka to get the nearly one hundred Elo jump over Shredder 10. All programmers use ideas from other programmers, but only the recent ones learned enough about Rybka (at least early Rybka versions) to borrow ideas from Rybka.
Oh, my God!!! Larry! Shredder-Rybka Clone! Oh-damn! What's happened to you over the years? You used to be so a political. No look! For God Sad! Shredder???? What is this the Rybka Enquire Forum? What next, Larry, SMK is an alien from deep space?
Well, I also think that a lot of programmers had a closer look at Rybka and tried to do the same things. Nothing wrong with that, Vas also didn't invent everything he uses for Rybka himself.
Can you please elaborate on "reportedly"? Do you have evidence or do you believe that only after examining the strelka source other programmers are able to gain 100 elo? Could it also be that, not being number one anymore was a trigger improve? Are you talking about reading source, decompiling or a change of idea, like making a program less speculative?
Being a part of the Shredder team, I did a lot for Rybka, like operating paderborn dec 2005 and 2006, planning a viit to the OPCC until Vas and Erdo decided to cancel, offering my help for paderborn 2007, but Joeroen went.
I did this for the love of computerchess and I like Vas, and know Jeroen from the time I was Rebel beta tester. Rumors like the one you are talking about spread easily and I would like to have it cleared.
Being a part of the Shredder team, I did a lot for Rybka, like operating paderborn dec 2005 and 2006, planning a viit to the OPCC until Vas and Erdo decided to cancel, offering my help for paderborn 2007, but Joeroen went.
I did this for the love of computerchess and I like Vas, and know Jeroen from the time I was Rebel beta tester. Rumors like the one you are talking about spread easily and I would like to have it cleared.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill