Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Deep Sjeng @ 29th Dutch Open
- - By GCP (***) [be] Date 2009-10-19 12:52 Edited 2009-10-19 19:20
Hi all,

a little retrospective on the Dutch Open from our side!

The Deep Sjeng team was:

Gian-Carlo Pascutto (programmer)
Erdogan Gunes (book)
Jens Heerklotz (engine test)
Sujay Jagannathan (hw test)
Bert Leznir (book test)

After the ACCA tournament I worked for a while to fix some longstanding issues in the engine (for example to prevent the disaster we had in the last round there!) and try to tweak the single-core performance a little. I got some nice improvement but was also a bit too stubborn trying to get some things to work. Given that the Dutch Open was an unlimited tournament, and Suj has been extending his hardware store, I wanted to rework the clustering algorithm extensively to scale better over a large number of clients. Eventually it was a scramble to get the revised clustering ready in time, with some unexpected issues propping up in what is easily the most complex code I have ever written.

I wasn't able to deliver a more or less completed new version until less than a week before the tournament, which meant that again it was a total scramble to get everything tested in time. Jens, Suj, Erdo and Bert have been essentially running nonstop tests until the wee hours of morning to get both the book adapted to the new version on such short notice, and to figure out the best cluster setups. Some of you who follow playchess closely might have seen some of that. Eventually I got confirmation that both the engine was a clear single-core improvement and that (with some settings) the clustering code was giving a nice extra speedup, so we could go the tournament in the knowledge that we should be able to put up quite a fight against anyone. However I also know the other programmers have worked hard so that unpleasant surprises can always happen.

The start of the tournament was quite a let-down due to very serious issues with the internet connection, which persisted throughout the entire weekend. We eventually started much later than intended and had to play with reduced time controls to make up for the lost time. Given that I am very much not a morning person and had to get up very early to get to Leiden in time for the announced starting time this wasn't very pleasant. Not to mention - we could not go into the nice city to relax afterward.

We started the first day with a configuration of 3 x 24 core 2.4Ghz AMD machines.

- Joker played the first game without book, made some positional mistakes, and was summarily mopped up. This was not so spectacular but we were happy to confirm that the cluster seemed to be stable and no bugs with mating and hash tables showed up (unlike some other participants).

- I was happy when I saw The King joined, as it has an active-attacking playing style that mixes well with that of Sjeng, meaning games from both programs tend to be huge fights. It did not look like much was going to happen after a passive opening from the King, until it suddenly sacrificed a pawn for pressure against the white King. However, black also got his king stuck in the center and truly massive complications ensued, which even with all 72 cores we could not punch through. At move 22, Sjeng was planning f5 and a double piece sacrifice, thinking this would give white a small advantage, when suddenly the roof fell in and the score dropped by 2 pawns.
4rk1r/pb1n1p2/1p1bp1p1/3n2N1/5P1P/3q1B2/PP1BNRP1/R2Q2K1 w - -
Even when going into emergency mode (fail low time extension), the complications were too much to see through, Sjeng timed out and played the best move it had so far, which was probably still not very good. The King won a quality and started trading down to a won endgame. Unfortunately for it, it traded down just a bit too eagerly and left a misplaced rook. The cluster locked on to this very quickly and our score jumped above zero again. Deep Sjeng put the rook definitely out of play, activated its king, and won the resulting endgame. Truly one of the most amazing computer chess games I have ever played.

- It was a bit sad that we met Rybka already in round 3, because this was likely to be a decisive game. On the positive side, we had the white colors. We also made some slight changes to the hardware setup and settings from the latest incoming test results, slightly reducing the number of cores to 64, but getting a better overall performance for the reduced time control (75 mins instead of 90). In the game Rybka was in book quite a while longer and seemed to exit in an equal position, where it's a question if black's space grab on the queenside provides sufficient compensation for the uncastled king. After a deep search Rybka found c3! and Sjengs score plummeted the next move. Deep Sjeng tried hard to make something of the black's king position, and scarified a piece to get to it, but in the end white didn't really seem to have something and black won being a piece up. For me at least one positive side of this game was that I had been watching Rybka's scores and they tracked Deep Sjeng's quite closely (as well as the best moves, causing us to lose lots of time when Rybka kept getting ponderhits). This seems to indicate Rybka is no longer greatly outsearching us, which opens some prospects for future face-offs.

For the second day, we again changed the hw setup and the settings based on overnight testing, although I can't remember to what :P

- The game against Hermann was a quite simple victory. White scarified a pawn for better development, but could not understand it's own compensation. This was a bit of a funny situation as Hermann evaluated itself at -0.6 even though Sjeng only though black had a small advantage (-0.15). As a result, white's advantage did evaporate pretty quickly and black easily won being a pawn up.

- In the game against Hansdamf, we got a typical Erdo-Sicilian. Unfortunately at some point white wanted to repeat moves, and Deep Sjeng avoided this with a positionally suspect move (Bxc3), giving up it's Dragon bishop to give white two doubled isolated pawns on an open file.
r2q1rk1/1p1npp1p/p1npb1p1/8/4PP2/1Nb1B3/PPP1B1PP/R2Q1RK1 w - -
Although strongly condemned by Erdo and most chess players in the hall, Jeroen seemed to be more cautious as blacks compensation is indeed non-negligible. In fact Hansdamf almost instantly proceeded in the wrong way and could not defend all of it's weak pawns. Deep Sjeng snatched a few off and traded down into and endgame 2 pawns up.

We had a bye in the next round so there was some time for a walk in the city. I had not eaten anything during the second day as the games were ongoing, but this was easily fixed in the Greek restaurant Delphi in the center of Leiden, where for the first time since 3 attempts in the last years I was able to finish one of their huge and very tasty meat platters. Jummy!

In day 3, we again had a change of hw and settings, this time moving to 96 cores with fairly conservative parameters.

- In the first game against Pandix, I was somewhat concerned as Pandix was a bit of an unknown factor, but I heard several times the engine has improved tremendously, so this might well have been a very strong remaining opponent. We played a deep Spanish line, where my opponent remarked several times he prepared this opening quite deeply. Deep Sjeng was out of book first, that is to say, we still had bookmoves but Erdo had set it so the engine had to find them itself. This resulted in a lot of gloating from our opponent over his superior preparation. After his last bookmove our initial score was about -0.5 and we were already in a bit of a panic. However, during the search the cluster took a very long time over a move that left our rook hanging. After a while the score for it returned and we went from -0.5 to +1 for N3h4. After another minute the score for Nxh6 returned: +4.5.
2r1rbk1/1b1n1pp1/nq1p3N/3P4/1pp1P3/R3RN1P/1P3PP1/1BBQ2K1 b - -
The game was over at that point although our opponent played on for a while (his engine was still a piece up, after all :)). Afterward the opponent remarked how lucky we had been, to which I replied I was happy to play him again, and again, and again :) :) ...

- The next round against Ktulu was also a quite crazy game. Our opponent was out of book quickly but found all the bookmoves for a long while. After a bit of wiggling in a balanced position Sjeng saw a way to find the advantage, but several moves into the maneuvering we suddenly had a big fail low on Qc1. Unfortunately the opponent also saw this trick and a few moves later our position was busted. Deep Sjeng thought the position was defensible (+0.2 for white) but we didn't really agree. It defended as accurately as possible, and after the opponent missed a chance to finish us off, he pushed too hard for a win, allowing us to get a single passer that outran his two connected ones (the Ktulu operator saw this problem several moves before the engine did). For the second time this tournament we ended up in a losing position, but accurate defense against opponents too eager to win allow us to turn the tables on them. I analyzed this game a bit afterward as I was not happy with the cluster missing white's trick, but Rybka 3 also did not seem to see it sufficiently ahead of time on my laptop, so I've decided not to worry about it too much. We also reached our peak NPS in this game: 110mnps, just 10mnps shy of Deep Blue's speed.

- The last game against The Baron should have been quite interesting, as The Baron has snatched away many half points from us in the past, and did in fact do so again against Rybka. The very solid search combined with a solid evaluation and a solid book ... make for a solid opponent! However the game was somewhat of a letdown. After the opening white had more space, better development and a simple and straightforward winning plan. Deep Sjeng makes no mistakes in such positions and The Baron lost without being able to put up a fight. Nevertheless a good tournament for Richard who won his first trophy.

Although we again were half a point short of stopping Rybka's winning streak, we're nevertheless very happy with our performance and look forward to the next tournament, where we will again go on a fishing expedition. There will probably be no opportunity to make sushi in Japan, but I've heard Hollandaise sauce goes pretty well with fish platters.
Parent - By Uly (Gold) [mx] Date 2009-10-19 13:13

>Moderators, can someone put a diagram here perhaps?


Just paste the FEN or EPD into the post and a diagram will be generated.

And congrats on the second place!
Parent - - By George Tsavdaris (****) Date 2009-10-19 13:18
Thanks for the report but do you have the games Sjeng played?
And all the games of the tournament if possible?
Parent - By George Tsavdaris (****) Date 2009-10-19 13:52

>After his last bookmove our initial score was about -0.5 and we were already in a bit of a panic. This resulted in a lot of gloating from our opponent over his superior preparation. After his last bookmove our initial score was about -0.5 and we were already in a bit of a panic. However, during the search the cluster took a very long time over a move that left our rook hanging. After a while the score for it returned and we went from -0.5 to +1 for N3h4. After another minute the score for Nxh6 returned: +4.5.


2r1rbk1/1b1n1pp1/nq1p3p/3P1N2/1pp1P3/R3RN1P/1P3PP1/1BBQ2K1 w - - 0 23


I have 20...Qb6 as bad in my book and only playable the 20...Ne5. But perhaps it's bad this also, you can  ever be sure with all these clusters. :-P

>Afterward the opponent remarked how lucky we had been, to which I replied I was happy to play him again, and again, and again


You should have asked him why he believed you were lucky and explain the reasons. I can't see how luck is involved in Chess and in this position.
You've said that this position occurred after his last bookmove, so obviously the only one he should blame is the book author and not luck.
Parent - By Carl Bicknell (*****) [gb] Date 2009-10-19 15:39
Thanks for an outstanding report GCP and well done on the result.

I wish you all the best for the future and I'm looking forward to the next release of your program.
Parent - - By Venator (Silver) [nl] Date 2009-10-19 15:42
Congrats on a fine performance!

After his last bookmove our initial score was about -0.5 and we were already in a bit of a panic. However, during the search the cluster took a very long time over a move that left our rook hanging. After a while the score for it returned and we went from -0.5 to +1 for N3h4. After another minute the score for Nxh6 returned: +4.5.

Interesting comment. When did you leave book exactly? BTW, 23.N3h4 is in the Rybka 3 book (CB and Aquarium) and seems to be winning, but 23.Nxh6+! is indeed more straightforward. A truly brilliant game and the easiest way to punish a 'bla bla I am better prepared opponent' ;-)
Parent - By Nelson Hernandez (Gold) [us] Date 2009-10-19 16:26
I don't know what the Pandix guy was smoking; very clearly you guys had the advantage throughout.  After 22.Nf5 I have white with an 87% successful position.  I'm a bit surprised you chose 18.Ree3 though, as that is a quintessential computer-generated opening line which I think is drawn with perfect play.  Human have seldom if ever played it while computers have many thousands of times.
Parent - By Felix Kling (Gold) [de] Date 2009-10-19 21:08
website updated :)
Parent - - By Vasik Rajlich (Silver) [pl] Date 2009-10-20 06:34
Interesting report!

Our clustering experience is actually pretty similar - my clustering code is also really complicated and terribly messy. There are several reasons for this:

1) Asynchronous stuff is always complex and hard to think about.
2) This is the one place where the task is complex and yet you don't care at all about speed (in terms of clock cycles). With normal multi-processing, you have to keep the overhead reasonable, which leads to simpler design.
3) There is no public domain code to keep your thinking process in check.

Anyway, I hope at some point to simplify everything. Complexity/simplicity often go in cycles.

We also had some crazy scrambling periods to get ready (or to get almost ready :)).

Vas
Parent - - By GCP (***) [be] Date 2009-10-20 09:21

>1) Asynchronous stuff is always complex and hard to think about.


And it's hard to miss hidden synchronicity. I went through several "why is it idling???" "oh!" "back to the drawing board" cycles.

>2) This is the one place where the task is complex and yet you don't care at all about speed (in terms of clock cycles). With normal multi-processing,
>you have to keep the overhead reasonable, which leads to simpler design.


True, but there is also a flip side: you can afford to write higher level code, knowing that the small penalties will now turn into negligible penalties. My clustering uses things like reference counted pointers (automatic garbage collection), dynamic memory allocation and the STL to manipulate some of the more complicated datastructures. This would probably not be acceptable performancewise within normal SMP. But it helps in developer faster, clearer code where the real algorithm stands out.

>3) There is no public domain code to keep your thinking process in check.


This is probably an advantage, you might get dragged into a "how do we do this better" line of thinking, whereas now the thinking is "how do I do this right".
Parent - - By Vasik Rajlich (Silver) [pl] Date 2009-10-21 08:13

> True, but there is also a flip side: you can afford to write higher level code, knowing that the small penalties will now turn into negligible penalties. My clustering uses things like reference counted pointers (automatic garbage collection), dynamic memory allocation and the STL to manipulate some of the more complicated datastructures. This would probably not be acceptable performancewise within normal SMP. But it helps in developer faster, clearer code where the real algorithm stands out.


I had a long argument with Stefan about this in Mainz. He's also written an engine in a high-level language (C#, IIRC), and claims that the code is better in every way except for speed.

I actually like getting the low-level stuff right. It's a good way to make sure you're thinking straight and keeping things simple enough. If you're using the STL in a chess program, chances are what you really need is another round of thinking.

C++ and high-level programming don't mix very well either, but that's another story.

Vas
Parent - - By GCP (***) [be] Date 2009-10-21 08:45

>I had a long argument with Stefan about this in Mainz....If you're using the STL in a chess program, chances are
>what you really need is another round of thinking.


I have to say I firmly disagree with your stance also.

Let me put it differently. If you're going to write a chess GUI (also mostly not performance critical), are you going to do it in bare C?

If you say no, what's the difference?

>C++ and high-level programming don't mix very well either, but that's another story.


I agree on that one, but there isn't really anything that can totally replace it so we're going to be stuck with it for the foreseeable future.
Parent - - By Vasik Rajlich (Silver) [pl] Date 2009-10-22 07:34
Of course, it depends on the task.

Object-oriented programming is in general overrated and overused, a lot of people try to squeeze all sorts of problems into an object-oriented framework which don't belong there. I have seen some incredibly convoluted examples in my previous jobs. For graphical components, though, I think an object-oriented approach is really an almost-perfect fit. A graphical component truly is an object, and inheritance truly describes what you want to do when you create a customized graphical component.

Algorithmic stuff tends to be sequential (although I've also seen some elegant LISP-type approaches for a very small set of problems) - it's usually best handled as a set of steps. I think it's good if the steps are small, simple, terse and concrete. It's a sign of good thinking. My current cluster code is a sign of unfinished thinking. :)

Vas
Parent - - By GCP (***) [fr] Date 2009-10-22 09:46
I'm not sure how we suddenly got into OOP here.

Standard containers, automatic memory management, and non-fixed-size datastructures can be very convenient, but they don't require OOP.

If we're talking about (for example) manipulating a splitlist and one does:

struct splitlist {
};

void splitlist_add_position(state_t *s, ...);

then this is the same as doing:

class splitlist {
    void add_position(state_t *s, ...);
};

Likewise you could say that:

move_t moves[MAX_MOVES];
moves[movecnt++]  = construct_move(from, to, ...);

is the same as:

std::vector<move_t> moves;
moves.push_back(construct_move(from, to, ....));

Arguing about OOP for that comparison is in my opinion besides the point. The point is that the second form doesn't require considering in advance how many moves are possibly needed in any situation, is easier to debug(*), but has a performance penalty due to dynamic memory allocation. This doesn't have much if anything to do with OOP design.

This example is for the most basic thing possible: an array. If you need to manipulate something more complicated, the advantage of the STL code rises rapidly. I would say that in code where you can suddenly get an error on a socket and have to terminate a client, while holding various locks on various datastructures, the advantage is big enough to take the burdens.

I've written an entire chess program in the first style (lets call it classical C) and an entire Go program in the second style (heavy use of C++ features but no more OOP than necessary) and there is no doubt in my mind the second approach produces code faster and with less bugs.

(*) As in: you'll catch buffer overruns. C++ being what it is, it will also make your stack traces harder to read, sigh.
Parent - By Highendman (****) Date 2009-10-22 10:26
Fun to read.
I hope you find reasons to continue this discussion.
Parent - - By Vasik Rajlich (Silver) [pl] Date 2009-10-23 07:16
We got into OOP when you asked me how I'd write a GUI.

The problem with for example using a vector for a move list is that you just added an abstraction layer and it makes everything a little bit fuzzier to think about. This example is pretty non-intrusive (tiny layer, tiny benefits) but

int moves [512];
moves [ct ++] = ..;

is easier to visualize. It also forces you to think a bit more, in a good way.

For an analogy: research shows that people have fewer motorbike accidents when they have to pedal themselves than when the power comes only from the vehicle (corrected for speed, of course). The extra "concrete engagement" helps.

Another analogy: lifting free weights tends to be more effective than using machines.

Of course, this all has its limits. I wouldn't want to program in assembly (or better yet CPU op codes). For some tasks, a little more abstraction makes sense. For a chess program, I feel like "classical C" is the right level - it forces you to think about just the right things.

> I've written an entire chess program in the first style (lets call it classical C) and an entire Go program in the second style (heavy use of C++ features but no more OOP than necessary) and there is no doubt in my mind the second approach produces code faster and with less bugs.


Interesting. I thought Go had an even higher "Elo per search time" figure than chess (maybe it just has a better "Elo per improvement" figure).

Vas
Parent - - By GCP (***) [be] Date 2009-10-25 18:09

>For an analogy: research shows that people have fewer motorbike accidents when they have to pedal themselves than when the power comes only
>from the vehicle (corrected for speed, of course). The extra "concrete engagement" helps.


Interesting analogy. But I would like to point out that those who don't have to pedal for themselves reach higher top speeds.

>For a chess program, I feel like "classical C" is the right level - it forces you to think about just the right things.


I would agree on this for the bare engine (obviously), but as stated, clustering is a level above it which IMHO has a different optimal trade off.

>Interesting. I thought Go had an even higher "Elo per search time" figure than chess (maybe it just has a better "Elo per improvement" figure).


I'm not sure what the relation between that is and what I said. Please don't think Leela is slower because she's in in C++ rather than in C.
Parent - - By Banned for Life (Gold) Date 2009-10-25 18:39
A better analogy might be that people using cruise control in a car, or autopilot in an airplane, are much more likely to get in trouble than they would maintaining manual control. Of course its much more convenient to use the automation...
Parent - By GCP (***) [be] Date 2009-10-25 18:42

>A better analogy might be


No.
Parent - By Uly (Gold) [mx] Date 2009-10-26 07:07
What about people that let their engines running overnight and play the output move that get crushed against people that interact heavily with the position, In Correspondence chess?
Parent - - By Sesse (****) [no] Date 2009-10-24 08:38
Is really your definition of OOP dependent on whether you write "foo(obj);" or "obj.foo();"? I'd say your "struct splitlist" implementation is a typical use of OOP in C :-)

/* Steinar */
Parent - By GCP (***) [be] Date 2009-10-25 18:03
I said:

...

>then this is the same as doing:


...

>Arguing about OOP for that comparison is in my opinion besides the point.

Parent - - By tomgdrums (****) Date 2009-10-21 01:00
Hey Vas!

So to put chess aside for a second:

It doesn't seem like my Eagles or The 'Skins are doing much this year.  Yeah the Eagles have a winning record but Reid is starting to show his game management "skills" again.  I think he called 50 some pass plays to 14 run plays against the Raiders.  Ick.  And the vaunted Wildcat with Vick has not even seen the light of day.  Watching Reid coach is like watching me try to play chess against Rybka 3!  :)

How long do you think Zorn has as the Washington coach?
Parent - - By Vasik Rajlich (Silver) [pl] Date 2009-10-21 08:14
The Redskins are breaking my heart. We'll still beat the Eagles though. :)

I don't get this trend of gambling on unproven coaches. For every Tomlin, there are three Zorns.

Vas
Parent - - By tomgdrums (****) Date 2009-10-21 17:48
Who knows maybe Reid's patzer like play calling and clock management might even save Zorn's job.    But at least I still have my Phillies who are looking really good right now!
Parent - By Vasik Rajlich (Silver) [pl] Date 2009-10-22 07:34

> Who knows maybe Reid's patzer like play calling and clock management might even save Zorn's job.


For another week! :)

Vas
Parent - By z0qxz (**) Date 2009-10-21 18:51
Redskins...need to change to Rybka..like Washington Rybka.. maybe they will win some games   :-))))
- By Lukas Cimiotti (Bronze) [de] Date 2009-10-19 13:34
Thanks for that very good report. And congrats - you played a very good tournament :)
Up Topic Rybka Support & Discussion / Rybka Discussion / Deep Sjeng @ 29th Dutch Open

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill