Hardware and Clustering
Lukas Cimiotti ran Rybka on his massive cluster. He has made tremendous improvements in the last few months, in three areas:
I) The cluster is now even bigger. There are eight Nehalem twelve-core machines, plus thirteen Nehalem eight-core machines, all running at base speeds of 2.93 GHz and up, for a total of twenty-one Nehalem computers and an even two hundred cores. Due to issues with space, power and cooling, Lukas has split his cluster into two sections. Thirteen of his computers remain in his main cluster room, while the remaining eight are now in his basement, controlled remotely via KVM over ip.
II) The administration of the cluster has been automated via thousands of lines of new custom PowerShell scripts which Lukas developed for this purpose. These scripts cover nearly every aspect of the cluster's operation, from managing executables and log files, to monitoring computer health and performance, to even detecting and correcting errors. For example, hardware or software crashes can now be automatically fixed without interrupting the client and without requiring any human intervention.
III) The work-splitting algorithm and the contempt mechanism are improved and better tuned, boosting the cluster's parallel speedup and reducing the frequency of draws yielded to weaker opponents.
Jiří Dufek prepared our opening book. His preparation consisted of manually adding new analysis to the Rybka 4 book, selecting lines against specific opponents, and testing in conditions which were as similar as possible to the tournament conditions. The ever-challenging task of winning games with black was met by handpicking variations which were solid yet active, and not too forcing. Jiří's test results leading up to the tournament were good and our opening play during the event was very successful. Rybka stood better and could quickly further improve her position in every white game, while our black positions were all at least equal and left opponents with plenty of practical problems to solve.
Jiří mentions Jeroen Noomen and Dagh Nielsen (yes, really!) as helping him with his preparations. Nick Carlin also chipped in with some technical help.
Hans van der Zijden was our operator and representative. Everything went smoothly for him and he kept us entertained with his photo reports from the site. Hans also continued his streak of doing better every year in the blitz tournament, showing off his fast reflexes and good nerves by winning for the second year in a row.
There were a lot of interesting moments, here is one:
Pandix-Rybka, round 3
Rybka 4 suggests the obvious 29. .. Rd7 with an advantage for black. During the game, however, Lukas' cluster managed to work 29. .. Rd7 out to a draw, which is quite amazing. The critical line runs 29. .. Rd7 30. Qe4 Nd6 31. Qf3!! bxa4 32. fxe5 Nb5 33. Bxh6!, with complications which peter out in a draw.
Hans' photo reports from the site: http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=18935
Jiří's opening report: http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=19017
Tournament crosstable: http://ticc.uvt.nl/icga/cg2010results/WCCC.html
Browsable games: http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=18950
Lukas' live game threads with cluster scores:
Pandix-Rybka (3): http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=18957
Rybka-Jonny (4): http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=18960
Rondo-Rybka (5): http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=18976
Hector-Rybka (6): http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=18984
Rybka-Fridolin (7): http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=18988
Junior-Rybka (8): http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=19001
Rybka-Shredder (9): http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=19012
Report about the blitz tournament: http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=18996
Thread about blitz tournament: http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=18991
> II) The administration of the cluster has been automated via thousands of lines of new custom PowerShell scripts which Lukas developed for this purpose. These scripts cover nearly every aspect of the cluster's operation, from managing executables and log files, to monitoring computer health and performance, to even detecting and correcting errors. For example, hardware or software crashes can now be automatically fixed without interrupting the client and without requiring any human intervention.
First, congratulations !
Does that mean that the cloud Rybka is ready ?
Rybka on the cluster gives hope that the draw death of chess is still far away, if Dagh? had the goal to create an opening book that could draw with any opponent, he might be a little put off after seeing the games we saw in Kanazawa.
Rybka, Computer Chess World Champion, 2010, 2009, 2008, 2007.
>Rybka on the cluster gives hope that the draw death of chess is still far away
Those other engine I guess is like 3000-3100 elos strong. There are also entrants that i think it is not even 3000 elos strong. And there are also draws, rybka did not win all the games ;)
And also engines like stockfish or the "thing" were not even included.
We still need to do a little bit more to reduce the draws, some of the draws are still too easy.
>reducing the frequency of draws yielded to weaker opponents.
but what is the draw rate of cluster selfplay?
So Rybka played 29.. Qd3x which petered out into a draw as well. It is doubtfull whether Pandix would have found the drawing line after 29.. Rd7 so it seems Rybka was playing against herself.
In the distant future Rybka cluster will probably offer a draw at the first move as it has found a drawing line for its opponent
The way that the next version of the cluster wins this game is by understanding earlier that 29. .. Rd7 only draws. In this game, the cluster worked it out on move 29, and by then the only alternative was a drawish endgame.
> What happened on the "software" section? e.g. who participated, who won, etc.
Here are the results below from the "software" section:
Shredder narrowly outpaced Rondo. Thinker also made a good showing in this section as well
Rank Program Origin Score Place
1 Shredder Germany 7 (Gold)
2 Rondo USA 6 ½ (Silver)
3 Thinker USA 5 ½ (Bronze)
4 Pandix Hungary 5 4
5 Junior Israel 5 5
6 Jonny Germany 4 6
7 Darmenios Poland 2 7
8 Fridolin Germany 1 8
9 Hector Hungary 0 9
> well, for the "software" thing rating lists are more important. There's no point in playing such a tournament, what you want is state of the art chess (software, hardware, opening books). I think opening books are e.g. also allowed in the "software" tournament, somehow the logic is flawed. Tournaments are not to find out which program is the best :)
I disagree wholeheartedly.
Of course there is a reason for playing such a tournament. Things are level. (except for maybe opening book-but even most of those are fairly level).
And as far as your comment that "tournaments" are not to find out which program is the best"? Are tournaments instead meant to see who has the biggest cluster?
I say tournaments are held for the beauty and the spirit of a good chess competition. And in some respects I think the software tourney (which Rybka probably would have won anyway) is more in keeping with the spirit and beauty of true competition. (and I mean no disrespect to all the hard work that Lukas has done on the cluster).
And as for you saying. "...what you want is state of the art chess..". Yes for analysis BUT what I really want when following a tournament is good old fashioned competition.
> when you had deep blue noone said "but hey, it's a super computer, that's unfair, it uses multiple processors" and so on. This is still the beauty of a good competition, optimizing a program to run on a cluster is not that easy. Those programmers who don't compete in that area shouldn't complain. Getting the hardware if you have written a cluster software is not "that" hard. So it's still about software and not that much about hardware. What I mean with "state of the art" is also "state of the art software". Software not supporting clusters is not state of the art.
Okay...first of all I never said anything about fair or unfair. And I am not a programmer, so whether any of them complain or don't complain is not pertinent to what we are discussing. (and they have no right to complain because they are freely agreeing to the rules) So those points are irrelevant.
I still say that I find the competition aspect of a tournament much more interesting when the playing field is level in terms of hardware. And if it isn't so much about hardware as you say, then why didn't Rybka enter the software section?
For me the hardware clouds the issue of the competition.
Note that at the time of deep blue there was a WMCCC that did not allow using parallel search.
>Are tournaments instead meant to see who has the biggest cluster?
If I'm not totally mistaken Jonny had the biggest cluster - 800 cores. http://ticc.uvt.nl/icga/cg2010results/WCCC.html
>> If I'm not totally mistaken Jonny had the biggest cluster - 800 cores.
They may have but that still doesn't address the topic I brought up. And I do understand how much work you put into the cluser. All I stated was that I find the software section much more interesting from a competitive point of view. And I still do.
> ok, but would a tournament where every program plays on the same let's say 128 core cluster ok for you? :)
Yes it would! That would be awesome!! Again I have nothing against the cluster per se (although as I said before, I will most likely never own one so my interest in it is negligible), I just enjoy the idea of real competition.
The analogy might not be completely valid but in sports (except for golf and I believe tennis) they use the same equipment so that skill, training, and talent are the deciding factors (except in American Baseball where steroids have unfortunately killed that idea )
In fact Jack Nicklaus has for years stated that Professional Golfers should be required to use the same equipment for just the same reasons.
So yes a tournament where all programs play on the same cluster would be great indeed because that would also give an indication who can really program for a cluster as well!
>although as I said before, I will most likely never own one so my interest in it is negligible
You are only interested in things you can / will own? That's hard to understand.
There are so many things that I will never own - for example a war bird, a Corvette or a rocket. Still I think they are really fascinating.
> Yes, and the practical is well represented by SSDF, CEGT, and CCRL. People like to dream, so that is the interest in these tournaments as they are.
Partly true but the ratings lists are not competitions. So that is a bit different.
> But there is no meaning in having a 9-round tournament that you call a "competition" if the playing field is already known to be "equal". This will give random results much more often. The fact that we're dealing with state-of-the-art equipment is what makes this interesting.
I think part of the difference here is that I am that much concerned with randomness of results. I don't really follow any of this as a scientific experiment. So once again I say that for me that equal hardware is a more interesting competition to follow. If I am the only one who feels that way that is fine.
I am not dismissing the value of the cluster with regards to analysis. I am sure it does amazing things and will help lead to more powerful computers and engines in the future. However I just prefer to follow a chess engine competition when the hardware is of equal strength.
> Well, upon being asked earlier if you'd be okay with equal hardware with a large number of cores provided, and you strongly replied that you would. However, consider it this way--in real life, nothing is really "provided"--somewhere, you're really paying for it even if it is. Thus, they cut out the middleman and leave it to the people to deal with their own hardware issues, among the vast reserves potentially available. Thus, the real limitation nowadays, especially if you can get teams like Rondo and Joker to have potential access to gigantic numbers of cores, is whether the programmer is successfully able to program for that number. Thus, it really is a software competition in the end.
And I also said in an earlier post that in most other competitions they use the same equipment. (ie: Baseball, soccer etc, etc,) Hopefully they use the best equipment.
So once again I say that.... and even if it is only me...I find the notion of a tournament with equal hardware to be a much more interesting competition.
> They don't use the same equipment in baseball (and others). Making the analogy in baseball, some teams have stronger hitters and pitchers (hardware!) than other teams because they can afford to pay more money to get these. In addition, some hitters and pitchers are more into strategy and using good knowledge of the opponents (software!). Baseball is a very good analogy to how the WCCC is currently, and has always, been run (with the exception of that one year recently that made most of the programmers angry).
Ok, so for whatever reason you just don't want me to have my own opinion of what I think is more interesting to follow. (which all I have ever said) Not once in these postings have I said it should be changed to the way I think. I have not even tried to dissuade you from your opinion. All I ever said was what I found more interesting, and that my interest in the cluster is not very high for practical reasons.
And yes they do use the same equipment in baseball. Bats, balls and gloves all have to meet certain requirements.
So one again I say that I find it a more interesting and compelling competition if the hardware is equal. (even if it is equal clusters!)
Hopefully it is okay with you that I do have my own opinion of what I find interesting to follow.
The equipment that you mention in baseball is not as essential for being good as are the strengths of the players (hardware) and strategies employed by the players and managers (software). Your mention of balls and bats is only analogous to the size of the chess board and pieces used in OTB play. This is also equal, or within limitations, in both human and computer chess where players or programmers meet OTB.
> The equipment that you mention in baseball is not as essential for being good as are the strengths of the players (hardware) and strategies employed by the players and managers (software).
And at the risk of being irrational again....you just made my point for me. I tend to think of the strengths and talents of the players in baseball as the software...not the hardware. They have to learn how to use their talents..which requires practice and skill and knowledge of the game. That to me is all software.
To go to a different analogy (although I will admit this is a loose analogy because there is no actual "competition"):
I am a professional musician. (drums to be exact)
The drums I use are my hardware. As are my cymbals, choice of sticks etc. etc. And I should opt for the hardware that allows me to best express myself BUT a true measure of my software is to be able to express myself on the crappiest of hardware...
In fact I often tell my students that they are never to blame the equipment for their bad play.
I think that is why I enjoy the equal hardware setting (regardless whether is an old commodore 64 or a cluster), I feel like I am seeing the true talents of the programmer and his or her ability to program an exciting chess entity regardless of the hardware.
So maybe (and I am thinking out loud now) that is the difference in opinion? I tend to think of chess as art most of the time. (the parallels between chess and jazz are AMAZING) And maybe for these competitions most others are thinking in terms of concrete science?
And before Lukas chimes in and calls me a charlatan, I do see the need for the cluster and I assume that for analysis it is without peer.
It is true that most of us are thinking of chess as concrete science here...because I think that's exactly what it is when computers play the game. Computers don't really have much of an understanding of when they should create unclear complications, and in terms of the art that we come across in human games, any decent engine on a single-core computer can and recognize create most of that...which is another reason why you don't see it as often in computer competitions.
This is an absurd argument, given the fact that no ratings group has access to Rybka or Rondo cluster software today or even the prospect of access in the near future.
The ratings lists give very little information about the WCCC entries unless they are actually testing them. This seems like it will be increasingly uncommon in the future. I suspect Cluster Rybka will be dominating tournaments long after other available engines surpass the latest available desktop Rybka.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill