Topic Rybka Support & Discussion / Rybka Discussion / Rybka-Zappa 70 games on a quad - Turbo style (1660 hits)
![[is]](/mwf/v2/flags/is.png "Iceland")
One of the most memorable recipes presented on this forum last year was Turbo's recipe for seeing "Zappa be as good as Rybka (or maybe even better)." It goes like this:
Zappa Mexico vs. Rybka 2.3.2a.
TC: 60'+20"
Zappa book: TourBookII.ctg
Rybka book: RybkaII.ctg
Tournament settings with book learning on.
Additionally (not mentioned by Turbo) when I decided to test his recipe I added to following ingredients:
Pondering: Off (avoid memory bottlenecks)
EGTB: None. I based this decision on Turbo's statement that Zappa was at least as strong as Rybka in the endgame so I didn't want Rybka to be helped by tablebases.
Besides Zappa being at least as strong as Rybka when under these conditions, Turbo added that "24 games in these sorts of conditions is more like about 384 games (i.e. about a factor of 16) on CEGT or CCRL lists."
I played a match following Turbo's recipe to the letter a few months ago on a quad machine. Shortly after this I sold the machine and forgot about this match. However, recently when I was scanning my old disks I ran into the results again. I played 70 games and the results were as follows:
So, here Rybka scores 69%, compared to 62% in Rybka 2.3.2a mp 4CPU against Zappa Mexico 4 CPU in CEGT 40/120. But CEGT only played 50 games, which of course is quite a small number compared to mine, if we assume that the factor of 16 is correct :)
I also played 70 games under the same conditions on an 8 core machine. Again Zappa was burnt and not fit for consumption after this treatment.
Zappa Mexico vs. Rybka 2.3.2a.
TC: 60'+20"
Zappa book: TourBookII.ctg
Rybka book: RybkaII.ctg
Tournament settings with book learning on.
Additionally (not mentioned by Turbo) when I decided to test his recipe I added to following ingredients:
Pondering: Off (avoid memory bottlenecks)
EGTB: None. I based this decision on Turbo's statement that Zappa was at least as strong as Rybka in the endgame so I didn't want Rybka to be helped by tablebases.
Besides Zappa being at least as strong as Rybka when under these conditions, Turbo added that "24 games in these sorts of conditions is more like about 384 games (i.e. about a factor of 16) on CEGT or CCRL lists."
I played a match following Turbo's recipe to the letter a few months ago on a quad machine. Shortly after this I sold the machine and forgot about this match. However, recently when I was scanning my old disks I ran into the results again. I played 70 games and the results were as follows:
1 Rybka 2.3.2a mp +30/-4/=36 68.57 48.0/70
2 Zappa Mexico X64t4 +4/-30/=36 31.43 22.0/70
So, here Rybka scores 69%, compared to 62% in Rybka 2.3.2a mp 4CPU against Zappa Mexico 4 CPU in CEGT 40/120. But CEGT only played 50 games, which of course is quite a small number compared to mine, if we assume that the factor of 16 is correct :)
I also played 70 games under the same conditions on an 8 core machine. Again Zappa was burnt and not fit for consumption after this treatment.
No doubt Rybka would be favored in this match regardless, but it should be noted that both the opening book choice and the lack of tablebases work against Zappa. The book, because RybkaII.ctg came out long after TourBookII.ctg, and tablebases, because Zappa has more efficient TB integration into the search than Rybka.
Alan
Alan
> No doubt Rybka would be favored in this match regardless
There is no doubt about that, but clearly Turbo didn't agree with us :)
> but it should be noted that both the opening book choice and the lack of tablebases work against Zappa
The opening book choice was a critical ingredient of Turbo's theory. He had not stated anything about tablebases, but he did state that Zappa was at least as strong as Rybka in the endgame. Therefore I decided to skip tablebases. However, that decision did not make much difference for the outcome of the match.
I also think that Rybka's more selective search hurts in the endgame, but this is only one aspect of tablebase usage. If you believe the relative node counts from the two engines, you will see a much larger slowdown from Rybka on Normal TB access setting than from Zappa on its most extreme setting (1). I have no idea why this is so.
Alan
Alan
![[hu]](/mwf/v2/flags/hu.png "Hungary")
The probing code is going to be identical, unless Anthony was crazy enough to tinker with Nalimov's mess :)
It sounds like Rybka just probes more aggressively. This type of thing won't make a difference of more than 2 Elo or so.
Vas
It sounds like Rybka just probes more aggressively. This type of thing won't make a difference of more than 2 Elo or so.
Vas
I doubt that Anthony has modified Nalimov's code, but it is not obvious to me that the method of making use of multiple processors is independent from the time spent waiting for disk accesses. This is another profiler question and one of these days I'll break down and get a profiler for my chess computer. Based on anecdotal evidence, I don't believe Rybka probes the endgame tables more aggressively. After more than 500 16-0 Rybka-Zappa games on the CB server, its clear that Zappa almost always sees TB mates a few moves before Rybka (this is using Zappa's most aggressive TB setting) with a reported drop in kn/s from ~4000 to ~2800. I guess this is mostly due to Zappa focusing more search resources on possible mates though.
As far as difference in Elo goes, your previous posts indicated this was ~0. :-) Of course there are two factors here. How much improvement is possible from TBs with no slow down (this is what we were discussing earlier in the fixed depth scenario), and how much of a loss is associated with the slowdown from disk accesses.
Regards,
Alan
As far as difference in Elo goes, your previous posts indicated this was ~0. :-) Of course there are two factors here. How much improvement is possible from TBs with no slow down (this is what we were discussing earlier in the fixed depth scenario), and how much of a loss is associated with the slowdown from disk accesses.
Regards,
Alan
> it is not obvious to me that the method of making use of multiple processors is independent from the time spent waiting for disk accesses
This was suggested here - that the engine do something useful while waiting for the disk probe to return. This is quite a bit of complexity for practically zero benefit.
> Zappa almost always sees TB mates a few moves before Rybka
This is probably true for non-TB mates as well.
> your previous posts indicated this was ~0.
This time, I decided to be super-safe :)
> Of course there are two factors here. How much improvement is possible from TBs with no slow down (this is what we were discussing earlier in the fixed depth scenario), and how much of a loss is associated with the slowdown from disk accesses.
Of course, it's possible that in chasing the <2 Elo points from tablebase accesses, a program weakens itself by >2 Elo. This would be quite incompetent :), I would be really humiliated if it turned out to be the case for Rybka.
Vas
![[us]](/mwf/v2/flags/us.png "United States")
I agree that the tablebases had pretty much no impact on the match outcome. Here is a question, though: in your tests, did you ever have a period of 10 consecutive games in which Zappa won or drew? How about a period of eight consecutive games in which Zappa scored 5 points? If not, then we have evidence for the assertion that random statistics were not the reason for the outcome of the match in Mexico.
I don't have access to the games right now. All I can remember is seeing a sequence of 10 games with an even result in one the matches. I didn't check the other one.
Haha, nice tests :-)
Have you tried this with the Sheebar book? I could never really vouch for the good effects of the TourBookII.ctg since didn't try it in anymore than a few games. It's definitely not as good as RybkaII.ctg or Sheebar.ctg, that's for sure, and is nowhere near as deep. Sheebar is interesting, though, as is Rybka--when you take out "book wins" (i.e. RybkaII is more recent than Sheebar), Zappa was winning my matches. However, when I turned Rybka's book to "normal", suddenly Rybka was making scores more in line with what you have said, leading to the belief I've stated that Zappa Mexico is only better than Rybka when their books are made specialized against each other, i.e. Rybka's specialized lines walk right into the places where Zappa is best, but with Rybka's book at "normal" and Zappa's at either "tournament" or "normal", Rybka was definitely better.
Have you tried this with the Sheebar book? I could never really vouch for the good effects of the TourBookII.ctg since didn't try it in anymore than a few games. It's definitely not as good as RybkaII.ctg or Sheebar.ctg, that's for sure, and is nowhere near as deep. Sheebar is interesting, though, as is Rybka--when you take out "book wins" (i.e. RybkaII is more recent than Sheebar), Zappa was winning my matches. However, when I turned Rybka's book to "normal", suddenly Rybka was making scores more in line with what you have said, leading to the belief I've stated that Zappa Mexico is only better than Rybka when their books are made specialized against each other, i.e. Rybka's specialized lines walk right into the places where Zappa is best, but with Rybka's book at "normal" and Zappa's at either "tournament" or "normal", Rybka was definitely better.
> Haha, nice tests :-)
Thanks :)
> Have you tried this with the Sheebar book?
A new recipe? I think that one is enough for me :) Maybe someone else...
> I could never really vouch for the good effects of the TourBookII.ctg
Hmmm. A quick forum search made me think otherwise :)
Anyway, I enjoy your posts on the forum and it wouldn't be the same without you.
![[mx]](/mwf/v2/flags/mx.png "Mexico")
>> I could never really vouch for the good effects of the TourBookII.ctg
> Hmmm. A quick forum search made me think otherwise :-)
And I was the one that spread the rumor that it was the best book for Zappa Mexico through the Computer Chess fora...
That's okay--it was TourBookII.ctg after all--I had a bit of a memory lapse this afternoon :-)
Crikey, you're right! I checked back on my old tourney to make sure that it was as you said, and yes, those results of which I speak were for TourBookII.ctg, not Sheebar. It was Sheebar that I tried for a few games and didn't have as good of results, in spite of the fact that it's the deeper and overall better book.
Sorry about the confusion!
Sorry about the confusion!
![[gb]](/mwf/v2/flags/gb.png "United Kingdom")
nobody doubts that rybka is stronger than mexico, but any experienced tester on any of the testing sites will tell you in a match of 10 games anything can happen,i would go as far as to say 20 games. cozzie took his chance and won against the odds and won, best of luck to cozzie & erdo, no other programmers had the bottle to take rybka on.
![[nl]](/mwf/v2/flags/nl.png "Netherlands")
Do you still have the games?.
Hi Dadi,
Interesting! Can you send me the games?
Thanks, Jeroen
Interesting! Can you send me the games?
Thanks, Jeroen
Hee, I was first to ask ;)
![[ng]](/mwf/v2/flags/ng.png "Nigeria")
I think it is better to keep the games in a link for public consumption.
In addition, may be some one with 8 cores vs 8 cores systems could run the same match (with the same book or nunn matches) with time control of say 180'/120" or 180'/180".
In addition, may be some one with 8 cores vs 8 cores systems could run the same match (with the same book or nunn matches) with time control of say 180'/120" or 180'/180".
> In addition, may be some one with 8 cores vs 8 cores systems could run the same match (with the same book or nunn matches) with time control of say 180'/120" or 180'/180".
Isn't 180/180 like 40/40? I.E. roughly one minute per move?
He means 180'+120" or 180'+180", so these would be very long games, i.e. only two games per day in the second case.
Oooh, one more thing I forgot to mention, though I think I probably mentioned it when I did this months ago--after every few games, I would take a look at the lines in both books and make improvements and get rid of bad lines, etc. so as to make it more like "match" conditions (and to also completely eliminate repeat games). My guess is that doing this will make things far more even, though the most noticeable result would probably be a lot more draws.
Topic Rybka Support & Discussion / Rybka Discussion / Rybka-Zappa 70 games on a quad - Turbo style (1660 hits)
Powered by mwForum 2.22.1 © 1999-2010 Markus Wichitill