Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka @ 16th WCCC in Beijing
1 2 Previous Next  
- - By Vasik Rajlich (Silver) [pl] Date 2008-10-04 12:03
The 16th ICGA World Computer Chess Championship has finished and Rybka has won with 8/9, ahead of Hiarcs and Junior.

The full Rybka team was:

Operator: Hans van der Zijden
Hardware: Lukas Cimiotti
Openings: Jeroen Noomen
Engine: Vasik Rajlich, Larry Kaufman

I'd like to give a huge thanks to everyone on the Rybka team for their really hard work, to the ICGA for again organizing the event, and to the other participants for their spirited efforts as well as for working cooperatively to solve the internet problems at the site. The sporting level among the participants in these events is very high.

Special mention this time has to start with Lukas Cimiotti, who provided for us one awesome piece of hardware: an overclocked 40-core (5x2x4) Harpertown cluster. Lukas also spent about two weeks working with me to make sure that everything ran properly, as (predictably) my attempts to simulate his cluster on my weaker home hardware were not 100% successful. We must have had more than 100 phone conversations related to this.

For what it's worth, in self-play, the performance of this Rybka configuration seems to be around 100 Elo higher than that of Rybka 3 running on a normal overclocked Skulltrail. The search tree is shaped differently, giving the entity a somewhat different playing style. I find it quite attractive - the play is very precise. There is some food for thought here in the area of normal multi-processor search. Lukas is a regular in the Playchess engine room and no doubt anyone who is curious can learn more there.

On the opening book side, this tournament was an interesting point for us. Two months ago, Jeroen went all-out and published an uber-book for general reference, releasing all sorts of private analysis which in the past he would have kept private. This was good for users and good for Rybka, but what about the WCCC? As it turned out, it was no problem at all. Jeroen investigated and became attracted to a number of offbeat ideas which he came to regard as underrated, and after due consideration decided to risk them in Beijing, with really good results. We had a pleasant book advantage or more in every white game and full equality or better in every game with black.

Interestingly, for the second WCCC in a row, the decisive game was a book win. The battle-hardened Hiarcs team has as good a book as you'll find outside the Rybka team, but, with the tournament situation calling for trying to win with black, they decided to gamble and surprised us with an offbeat and risky line of their own. I'm not sure where the critical moment was but something went terribly wrong and the game was over by move 15.

On the engine side, a big thanks to Larry for continuing the gradual, painstaking task of tuning Rybka's evaluation. It's a centipawn here and a centipawn there - there are no signature moments from this work, but it is vital for everything else.

Thanks also to Hans for representing us in Beijing. Operating in these tournaments is a lot more work than people realize. I hope he managed to squeeze some fun in as well :)

Finally, a tip of the hat to Junior for reminding us about just how much work still remains.
Attachment: WCCC08.pgn (40k)
Parent - - By Dragon Mist (****) [hr] Date 2008-10-04 12:09
Hello Vasik, glad to see you back! :-)
Congratulations on your program, your team, and the title!
Parent - By FWCC (***) [us] Date 2008-10-04 14:16
Congrats,can I borrow that 40 core monster!?We should use that for the next Super GM match!
Parent - By Vasik Rajlich (Silver) [pl] Date 2008-10-04 17:44
Thanks :)

Vas
Parent - By Laszlo (***) [fr] Date 2008-10-04 12:59
Hi Vas,

Bravo to the team!
Parent - By turbojuice1122 (Gold) [us] Date 2008-10-04 13:08
Congratulations to you and your team!  It was a successful experiment with the cluster, and that was also good to see.

I think that you all could gain a lot of critical information from an informal match against the current version of Junior if that team is up for something like that.
Parent - - By Carl Bicknell (*****) [gb] Date 2008-10-04 13:13
Congratulations.

Has there ever been a world champion 2 times running? Isn't this a first?
Parent - - By Felix Kling (Gold) [de] Date 2008-10-04 14:59
Cray blitz won 83 and 86 :)
Parent - By Carl Bicknell (*****) [gb] Date 2008-10-04 16:04
oh really? Goodness it has been a long time since this has happened then. If Rybka wins next year it will be one for the Guiness Books!
Parent - By Silvian (***) [ro] Date 2008-10-04 13:17
BRAVO AGAIN !
Silvian

:):):)
Parent - - By Banned for Life (Gold) Date 2008-10-04 14:46
It would be interesting to hear how you came up with the estimate of 100 Elo improvement. I guess the fact that it is a round number may indicate it is more of a guestimate. I suspect the "normal" method of playing thousands of quick games might not be possible with a cluster configuration due to inter-computer latencies.

Regards,
Alan
Parent - - By Vasik Rajlich (Silver) [pl] Date 2008-10-04 17:49
Sure, it's a ballpark figure. We didn't try to measure it precisely. At 10' + 5" I got a score of 17-7 (+12 =10 -2).

The cluster is perfectly capable of playing fast games, there is no point below which the extra hardware doesn't help. I would often play a hundred or so games at a time control of 2' + 1" with the various versions.

Lukas will probably come up with some performance figures over the next few weeks, and then we'll start tuning some of the parameters.

Vas
Parent - - By Banned for Life (Gold) Date 2008-10-04 17:55
OK, so I guess we can assume that the parallelism is very coarse. I hope you will empower Lukas to play the cluster on the Playchess server so that we can witness this in an open forum! :-)

Regards,
Alan
Parent - - By Lukas Cimiotti (Bronze) [de] Date 2008-10-04 18:45
Hello Alan,

I already played on Playchess using the cluster. Here's a screenshot from the first tournament I played using it:

http://www.abdul-h.de/pictures/29092008Tour.JPG

I like the result ;)

Regards,
Lukas
Parent - By Banned for Life (Gold) Date 2008-10-04 18:52
Yes, that is a very convincing  result. I am looking forward to my engine meeting up with yours at some point in the future. :-)

Regards,
Alan
Parent - By Vasik Rajlich (Silver) [pl] Date 2008-10-05 16:31

> OK, so I guess we can assume that the parallelism is very coarse.


Yes. In general, fine-grained parallelism has problems with inter-process communication overhead while coarse-grained parallelism has problems with dependencies. Running on a cluster is an interesting problem.

As for testing, this will be up to Lukas, he knows what he's doing. We have a lot of time now to figure out how everything behaves.

Vas
Parent - - By Venator (Silver) [nl] Date 2008-10-04 15:37 Edited 2008-10-04 15:47
As promised I will write something about the openings used in Beijing. This time we used a completely different strategy compared with the tournaments before. When we were posting test results about Rybka 3 beta some months ago, one conclusion was important: Larry stated that less book moves favours the stronger program. First I was not sure about this, but later on I changed my opinion. So the opening tactics for Beijing were going to be:

1. Strive for short(er) lines with a lot of pieces on the board and a position rich in potential for playing for a win;
2. Avoid long theoretical battles;
3. Skip the idea of getting book wins;
4. Use Rybka's elo advantage to make good use of the short opening lines.

This meant something for my book, as this is about the opposite of what I am doing normally. So the work concentrated on finding slightly offbeat, but playable opening lines, not well covered in other books or in Playchess theory. I came up with some very interesting stuff and asked Larry to help me testing. The results of these lines were sometimes amazing: scores of 80% with black were no exception. So the rule given above - less book moves favours Rybka - really seemed to pay off.

Now how was this going to be in Beijing? I must say I was full of optimism. Not only did we have this huge 40 cores cluster machine, but the opening lines were very well tested as well. So let's analyse the games from round 1 through 9:

R1: Rybka - The Baron  1:0
Some of the participants were going to play the Petroff, that was clear when analysing the opponents in the WCCC. So I came up with this 4.Nc4!? idea, recently used by Carlsen and Morozevich. Both Rybka and The Baron were quickly out of book, but Rybka took the initiative fast after some strange Baron moves and if she would have played Rxf6 instead of Qxf6? it would have been a superb game. Luckily the bug didn't affect the end result, although the internet connection was rather shaky that day, with several disconnects.

R2: Rybka - Shredder  1:0
Of course several Sicilians were quite likely to be played, with the Najdorf being the 1st on my list. Once again I was influenced by Magnus Carlsen here, who played some very interesting games with 6.h3!? lately. When looking at the Playchess theory I was surprised how badly this line was covered. So 6.h3!? was going to be one of the tries against the Najdorf. I was quite surprised to see it already being played in round 2 vs Shredder. Again we had not so many book moves, but it was clear that the Shredder book hadn't covered it well: Rybka got an advantage pretty soon after book ended and although Shredder put up stiff resistance, Rybka nicely transformed the ending into a win.

R3: Cluster Toga - Rybka  draw
As black Larry proposed to try the O'Kelly Sicilian 1.e4 c5 2.Nf3 a6. I studied the O'Kelly and once again I found out that most books do not play the critical lines. Test results were amazing, so it was clear this line was going to be one of the main weapons for Beijing. Toga's book allowed the freeing Bc8-d7-b5! and I think after that manoeuver black has already easy equality. In my test games Toga kept on losing these kind of positions, quickly stuck with a bad bishop, with Rybka outplaying Toga at the queen's side. However, Toga was able to swap the lousy Ba1 for the knight at h6 and although black had nice pressure against d4, there was no way to improve Rybka's position. Toga even got a slight initiative, but it all petered out in a draw.

R4: Rybka - Jonny  1:0
The Ruy Lopez always has been a popular choice for black when facing Rybka with the white pieces, so Larry came up with the idea of a quick d2-d3 which I liked immediately. The line could already be used against Jonny (although I rather expected another Petroff here) and it became a swift and clean victory. When Jonny weakened its K-side and moved the queen away from the battle, Rybka attacked and crashed through with some nice sacs.

R5: Falcon - Rybka  0:1
For this game I abandoned the O'Kelly and used our 2nd Sicilian weapon. I hoped book author Erdo would try one of his Nc3, g3 and h3 Sicilians, as he tried against Hiarcs, but this time it was 2.Nf3. We had a normal 2... e6 Sicilian this time, with the white book playing Nc3 and Bd3, which is not really critical. Black equalised quickly and grasped the initiative after a few sub optimal Falcon moves. Still, the Israeli program put up a tough fight and it took Rybka some time to convert the advantage.

R6: Rybka - Mobile Chess  1:0
Mobile Chess was the only Chinese participant, running on a Nokia mobile phone. Once again we had a 6.h3 Najdorf, although I'd rather tried something else when looking back on this game. The result was never in doubt, but probably our opponents now sensed that 6.h3 was not a 'one way try', which also could be to our advantage, of course.

R7: Sjeng - Rybka  0:1
IMO the most 'obscure' game of the WCCC in Beijing from Rybka's point of view. Once again the opening tactics worked excellently, as we left book (a Grunfeld) in an unbalanced position with many pieces on the board. Shortly after book ended Rybka traded three pieces for the enemy queen and a pawn. When the black pawn phalanx was shattered around move 20 and Rybka allowed a protected passed pawn on d5, most programs are very happy with white and so was Sjeng! But Rybka disagreed, activated her pieces and the a-pawn turned out to be very annoying. At one point there was a score difference between Sjeng and Rybka of 1.32 pawns, amazing! Rybka proved right, pressurised white's pieces and Sjeng never got any counterplay at all. The nice blow 44... Rxc4! settled it, although it took the chess players a long time to see why white was lost.

R8: Rybka - Hiarcs  1:0
Somehow I was sure team Hiarcs was not going to find out what I had in mind against the Najdorf. So that remains a secret :-). They tried the Kalashnikov 4... e5 and threw us out of book at move 8, by using the very obscure 7... Rb8?! This completely suited us: 2 moves later black was out of book too, in a position where white is better. So in our philosophy explained above, this couldn't be better! Rybka quickly got the better of it and when the black king had to settle for the dangerous square d8 instead of the safe heaven at g8, it was clear that Rybka was going to score a crushing win. The game was over before move 20, team Hiarcs resigned at move 35.

R9: Junior - Rybka  1/2-1/2
As round 8 was already the decisive one giving Rybka the title, I got a bit reckless. Of course I could and should know that team Junior was going to prepare something for the O'Kelly, still I played it. Wrong choice. Although the opening was no problem for black, the resulting position suited Junior, that sacrificed a pawn for strong black square pressure. As a 2200 player I would never take that pawn so easily and the position after 19... g6 looks extremely dangerous for black. Every human player will agree this is a 'two results only' position: black holds, or he loses. Rybka thought there was no problem, but Junior outplayed her and things started to get rather gloomy. With Rybka's eval going down to -1 the game seemed to be over, but suddenly Junior overlooked a rook sacrifice, leading to a perpetual. A narrow escape for Rybka, although it was a bit strange seeing Junior struggling to accept the perpetual. Only 2 moves before the draw was agreed, it still thought it was winning.

All in all I am satisfied with my book preparation: we never had a bad or worse position out of book, equalised rather easily with black and the slightly off beat lines worked well for white: 5 out of 5.

Congrats to Vas and Larry for the 2nd WCCC title, thanks to Lukas for the great cluster and assisting me to broadcast some of the games. And Hans for doing a nice job in Beijing.

To end with: also big congrats for team Hiarcs scoring a well deserved 2nd place, IMO they played an excellent tournament. Well done!

Same goes for Amir Ban and Junior, let's hope there will be a new Junior in the near future, as the playing style of Junior is so much fun to watch!
Parent - By Carl Bicknell (*****) [gb] Date 2008-10-04 16:10
WELL DONE. Your hard work has paid off. Now you have a full year to break all the world records and get a program to win the World Championship 3 times running!
Parent - By Dragon Mist (****) [hr] Date 2008-10-04 16:27
... and the only thing we are waiting to hear is the explanation of Qxf6? in the first game.
Parent - - By FWCC (***) [us] Date 2008-10-04 17:03
I may be mistaken but it seems to me the O,Kelly variation is a bit of a thorn for Rybka as Toga drew also and Junior had a good game with it.Myabe more homework?
Parent - By Venator (Silver) [nl] Date 2008-10-04 19:46
From a statistical point of view you are correct: 1 out of 2 is only 50% and besides, these were the only 2 drawn games. But from an objective point of view black had nothing to worry about in these games after both books ended. And we had lots of test games with the O'Kelly scoring very well against ALL opponents.

If I were the book author of Toga, I would be quite disappointed with the opening in Toga-Rybka, as after a mere 10 moves white has absolutely nothing and a few moves later black already had a slight edge. Against Junior I should have followed my instinct which told me they would have been well prepared. Still the opening was OK for black and the pawn hunting lead to trouble, while it is impossible to foresee that Rybka would underestimate the danger of the black squares being so weak.
Parent - By Dark Horse (***) [in] Date 2008-10-04 17:03
congratulations on the win!!
Parent - - By Venator (Silver) [nl] Date 2008-10-05 18:40
As the official ICGA site gives an incomplete (and a bit messy) PGN file of the 16th WCCC in Beijing, I have done a little cleaning up and made a new PGN. It is available from Rybka's website:

www.rybkachess.com

See under 'Latest reports', 2008-10-04, klick on link 'games'.
Parent - - By Dragon Mist (****) [hr] Date 2008-10-05 18:42
I was just about to suggest someone do something like this! :-)
Thanks Jeroen!
Parent - - By NATIONAL12 (Gold) [gb] Date 2008-10-05 21:16
i cant find them.
Parent - By NATIONAL12 (Gold) [gb] Date 2008-10-05 21:51
thanks George.
Parent - - By George Tsavdaris (****) Date 2008-10-05 20:31

>and if she would have played Rxf6 instead of Qxf6? it would have been a superb game. Luckily the bug didn't affect the end result


Is this a bug that has been spotted and will be fixed or remains of unknown cause until now?
Parent - - By Vempele (Silver) [fi] Date 2008-10-05 20:39
Parent - - By George Tsavdaris (****) Date 2008-10-05 20:45
What words did i have to put on the search in order to find this post?
And i mean words that were logical to put and not after seeing the post of Lucas that mentions that the bug was fixed.
Parent - By Vempele (Silver) [fi] Date 2008-10-06 05:19
Fixed.
Parent - - By Bouddha (****) [ch] Date 2008-10-04 17:03
Hi Vas,

What is your comment about Rybka game vs Junior ?

regards
Parent - - By Vasik Rajlich (Silver) [pl] Date 2008-10-04 17:51
It was a gross misevaluation by Rybka. It's not an easy problem - you have to understand that the dark squares are absolutely key, worth much more than pawn + weak d4 pawn + control of the c-file.

Vas
Parent - - By Bouddha (****) [ch] Date 2008-10-04 21:00
Where should Rybka have played differently and what move should Rybka have played ?

regards
Parent - - By Vasik Rajlich (Silver) [pl] Date 2008-10-05 16:32
I'm not really sure. Giving up the dark-squared bishop was one key moment. By the time the kingside pawns are all on light squares, black is going to suffer.

Vas
Parent - By Bouddha (****) [ch] Date 2008-10-05 17:10
Thanks for your answer.

I do agree with you.

regards
Parent - - By Roland Rösler (****) [de] Date 2008-10-04 23:36
It was a gross misevaluation by Rybka.

Amir has seen it (ok., he is a clever one)!? Are Rybkas weak sides so easy to figure out?

PS: Congrats!
Parent - By Vasik Rajlich (Silver) [pl] Date 2008-10-05 16:34

> Amir has seen it (ok., he is a clever one)!? Are Rybkas weak sides so easy to figure out?


Figure out - not that hard. Exploit in games - not so easy. (We've tried in freestyle many times.)

Vas
Parent - - By Quapsel (****) [de] Date 2008-10-04 19:25
Thanx for these informations.
And congratulations for that very fine WM-title.

While these WM I firstly heard of clusterd chess-engines.
Will the coming buyable Rybka4-engine have such cluster features? I assume in many homes there are working several machines, which could get clusterd. It is an interesting feature which touches the play instinkt. :-)

When using 2 instead of 1 core we can estimate an effective speed increase of maybe 1.7.
Do you have an estimation of the effective increase when using two instead of one 8-core-machines?

Thanx
(also for that interesting event and the coming ones)
Quap 
Parent - - By Vasik Rajlich (Silver) [pl] Date 2008-10-05 16:36

> Will the coming buyable Rybka4-engine have such cluster features?


Almost certainly not. We have a really full schedule for Rybka 4 and this feature would need a lot of cleaning, support, etc. Maybe for Rybka 5 - we'll see.

> Do you have an estimation of the effective increase when using two instead of one 8-core-machines?


It seems that with 5 nodes, the speedup is comparable to normal MP speedup. In other words, the machine might as well have been a single 40-core machine. This equivalence will likely break down at some point.

Vas
Parent - - By nine castles (**) [us] Date 2008-10-07 14:48
Is it really sensible to make such a direct comparison? As I understood it, cluster computing results in a quite different kind of search.
Parent - By Vasik Rajlich (Silver) [pl] Date 2008-10-12 06:10
This is also true for MP search (although less so).

Speedup is nothing more than Elo gain.

Vas
Parent - By Axel Caro (**) Date 2008-10-04 20:40
Hi Vas and all the hard working and successful Rybka team members,

congratulations to the well-deserved second world championship!

It was great fun watching Rybka and her opponents fighting ;-)

Nevertheless there is some space for optimization to be left (Rybka vs. Junior, dark square evaluation) - and to go for the triple-WCCC!

Best regards,
Axel
Parent - By hal9000 (**) [no] Date 2008-10-04 22:01
Congratulations to Vas and the entire Rybka team!
Parent - - By Roland Rösler (****) [de] Date 2008-10-05 00:14
Okay I see, no information to the Rybka Cluster!!

M Ansaris thinking was very much better than this "100 Elo better than Skulltrail". Okay, 5 Skulltrails are 100 Elo better than one Skulltrail; but where is the clou?
October 2009 we all have Skulltrail power on desktop and Rybka 4 for the 100 Elo. But I hope, we will have 100 intelligent Elo points in endgames. : -)
Parent - By turbojuice1122 (Gold) [us] Date 2008-10-05 01:49
Don't forget, that is 100 elo versus other Rybka 3's on weaker hardware.  It's going to be a bit less than that in competition against other engines, especially highly tuned competition.
Parent - - By Nick (*****) [us] Date 2008-10-05 03:03
Congratulations guys.

"Rybka, World Computer Chess Champion 2008, 2007." :)

I know how much work you all put into this, Lukas has a day job too right and was continually up at 3am, a simply heroic commitment.  Some great alien openings from Jeroen (assisted in places by Larry) and simply awesome play from Rybka - with a great save in the Junior game too. 

Halycon days !!
Parent - - By Lukas Cimiotti (Bronze) [de] Date 2008-10-05 09:17
Thanks.
well, in fact I had holidays :) Else it would have been really difficult with all these connection problems caused by crappy Chinese internet. And it was pure coincidence that I did not have to work. In my job we always plan holidays for a whole year - we did that already last year before the date of WCCC was known. I chose the date because my daughters had holidays and my wife had to work. So it was pure luck.
Also the number of computers available had something to do with luck. I offered some parts (mainboard, CPUs, memory modules) for sale about a month ago. The guy who wanted to buy that stuff after some time told me he didn't have enough money. As it might be easier to sell such hardware as a ready built computer I decided to do so.  I  ordered a new case as that was the only part that I didn't find in my shelves. This was some time before I knew that Vas would really get a cluster version ready. That case arrived Thursday night - so I built the computer on Friday afternoon as soon as I came back from my last working day. As it is not overclocked I could use it without much testing. Btw. this computer's mainboard was already used in WCCC 2007. Maybe this also helped  ;)

Best regards,
Lukas
Parent - - By Nick (*****) [us] Date 2008-10-05 12:10

> I chose the date because my daughters had holidays and my wife had to work.


This statement could be interpreted in a number of different ways :)

As we discussed, I think Linux on the cluster nodes is well worth testing, in most middlegame positions on my quad I get between 280-300 kN/s using Windows and 340-360 kN/s using microwine/Linux+huge pages, this seems to make the machine competitive with octals on the Playchess server, where I only tend to play mediocre openings or copy and paste stuff.
Parent - By Venator (Silver) [nl] Date 2008-10-05 19:52
This statement could be interpreted in a number of different ways

Oh, I just love this typical British humour ;-).
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka @ 16th WCCC in Beijing
1 2 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill