Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Is the Rybka Cluster hot air?
- - By Uly (Gold) Date 2012-11-14 01:45 Edited 2012-11-14 18:48
Finally the Rubka Cluster is being tested properly against latest Houdini 3. People just seemed to assume that Cluster Rybka was the strongest entity on the planet.

Black is the Cluster playing with 64 cores in this game as white, Houdini is playing on 16 cores. Do you think Rybka has the best software?

[Event "105m+15s, unrated"]
[Site "Engine Room"]
[Date "2012.11.13"]
[Round "?"]
[White "TryMe, Rybka Cluster 64 Cor"]
[Black "Felix 2, Houdini 3 Pro x64"]
[Result "0-1"]
[PlyCount "146"]
[EventDate "2012.11.13"]

1. d4 {0.26/21 215} Nf6 {0.02/28 224} 2. c4 {0.46/22 305} e6 {0.09/29 447} 3.
Nc3 {0.36/24 0} Bb4 {0.08/27 179} 4. e3 {0.14/23 481} b6 {0.06/29 234} 5. Bd3 {
0.16/22 19} Bb7 {0.05/28 226} 6. f3 {0.12/23 0} c5 {0.02/28 161} 7. Nge2 {0.17/
22 9} Nc6 {-0.04/28 122} 8. O-O {0.11/22 305} cxd4 {-0.03/28 0} 9. exd4 {0.08/
23 290} O-O {0.01/28 0} 10. Qc2 {0.02/22 415} h6 {-0.10/26 133} 11. a3 {0.07/
23 260} Be7 {-0.01/28 534} 12. Na4 {0.11/21 279} Rc8 {-0.04/27 1} 13. b4 {0.21/
19 127} Re8 {-0.11/26 553} 14. Bb2 {0.21/21 0} b5 {-0.06/25 423} 15. Nc5 {0.15/
23 0} bxc4 {-0.08/27 159} 16. Bxc4 {0.13/23 21} Ba8 {-0.14/26 126} 17. Qd2 {0.
05/22 226} Nd5 {-0.11/27 0} 18. Ba6 {0.00/22 180} Rb8 {-0.45/28 288} 19. Bd3 {
0.00/23 190} Bg5 {-0.43/28 0} 20. f4 {0.02/19 60} Bh4 {-0.43/28 53} 21. Rac1 {
-0.10/20 382} d6 {-0.44/28 142} 22. Ne4 {-0.08/21 0} Nce7 {-0.50/26 75} 23. g3
{-0.10/20 24} Bf6 {-0.47/25 96} 24. h3 {-0.13/19 84} h5 {-0.51/27 0} 25. Rf2 {
-0.19/21 306} Nf5 {-0.68/24 93} 26. Re1 {-0.28/18 24} Nb6 {-0.80/26 56} 27.
Nxf6+ {-0.13/19 49} Qxf6 {-0.45/28 87} 28. d5 {-0.16/20 40} Qg6 {-0.52/29 66}
29. Nd4 {-0.49/20 67} Nxd5 {-0.88/25 65} 30. Nxf5 {-0.56/21 66} exf5 {-0.97/24
20} 31. Kh2 {-0.57/21 1} Nf6 {-0.83/28 57} 32. Rfe2 {-0.59/21 67} Ne4 {-1.25/
26 86 Game resumed} 33. Qe3 {-0.65/21 200} Re7 {-1.24/27 192} 34. Bxe4 {-0.81/
22 0} Bxe4 {-1.29/27 66} 35. Bd4 {-0.83/22 54} a6 {-1.33/29 142} 36. Qc3 {-0.
83/24 12} Ree8 {-1.44/29 82} 37. Rc1 {-0.91/23 36} Qe6 {-1.59/30 36} 38. Qa1 {
-1.00/23 81 (Dd2)} Rec8 {-1.79/29 97} 39. Rce1 {-1.08/23 0} Qg6 {-1.78/30 46
(f7-f6)} 40. Rc1 {-0.92/24 118} Rxc1 {-1.81/31 0} 41. Qxc1 {-1.01/25 7} Qe6 {
-2.09/27 64} 42. Bb2 {-1.01/24 0} a5 {-2.12/27 92 (Tb8-c8)} 43. Qc3 {-1.82/22
170 (Kg1)} f6 {-2.58/24 23} 44. Kg1 {-1.94/21 117 (h4)} axb4 {-3.08/27 90
(Tb8-c8)} 45. axb4 {-1.83/19 22} Rc8 {-3.35/28 137} 46. Qd2 {-2.04/22 0} Qd7 {
-3.36/27 73} 47. Qe3 {-1.97/21 64 (De1)} Qb5 {-3.53/26 41} 48. g4 {-2.14/21
104 (Td2)} Rc4 {-4.01/26 73} 49. gxf5 {-2.81/20 24} d5 {-4.38/24 21} 50. Qf2 {
-3.25/19 93} Bxf5 {-4.95/26 0} 51. h4 {-3.30/20 106 (Te7)} d4 {-6.18/23 55
(Db5xb4)} 52. Qg3 {-4.32/17 65 (Te7)} Rxb4 {-7.89/24 13 (d4-d3)} 53. Re7 {-6.
64/19 51} Bg4 {-11.06/28 7} 54. Bc1 {-7.51/20 0} Rb1 {-13.47/25 16} 55. Re1 {
-65.19/23 507 (Tc7)} Qc4 {-21.47/24 17} 56. Bd2 {-14.85/22 0} Rb2 {-17.76/23 5}
57. Rc1 {-7.23/21 1 (Te8+)} Qa2 {-23.20/21 1 (Dc4-e2)} 58. Be1 {-#15/22 452} d3
{-#17/33 0} 59. f5 {-#14/22 233} d2 {-#16/33 0 (Da2-d5)} 60. Bxd2 {-#15/22 124}
Rxd2 {-#15/33 0} 61. Re1 {-#13/12 69 (Tf1)} Kh7 {-#13/33 37 (Da2-d5) '} 62. Rf1
{-#13/23 19} Qa7+ {-#12/33 6} 63. Kh1 {-#11/23 10 (Tf2)} Qd4 {-#11/31 8
(Lg4xf5)} 64. Qg2 {-#10/26 21} Rxg2 {-#10/34 3} 65. Kxg2 {-#10/26 12} Be2 {-#9/
36 1 (Dd4-e4+)} 66. Rf2 {-#8/31 18} Bd3 {-#8/41 1 (Le2-a6)} 67. Kg3 {-#7/29 16
(Kf3)} Qg4+ {-#7/23 0} 68. Kh2 {-#6/25 0} Be4 {-#6/45 0} 69. Ra2 {-#5/37 29}
Qxh4+ {-#5/99 0} 70. Kg1 {-#4/35 1} Qg3+ {-#4/98 204} 71. Kf1 {-#3/46 0} Bd3+ {
-#3/97 173} 72. Re2 {-#2/51 0} Qh2 {-#2/96 0} 73. Ke1 {-#1/42 1} Qxe2# {-#1/99
0 (Lag: Av=0.15s, max=0.6s)} 0-1

Source:

http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=26016

Another game tomorrow, will this be a carnage?
Parent - By Cole Date 2012-11-14 02:59
Could someone who has contact with Kingscrusher on youtube ask him to maybe do a video on this and the following games as he did for the TCEC matches between Houdini 2.0 and Rybka 4.  Those were awesome!

https://www.youtube.com/watch?v=xWdMqvGMxF4
Parent - - By RFK (Gold) Date 2012-11-14 03:56 Edited 2012-11-14 03:59
What was going between move 32 and 33 that the game and come to an apparent halt. Is there any indication as to how long the game was down? I don't know how to interpret the data time wise in parenthesis.
Parent - By Lukas Cimiotti (Bronze) Date 2012-11-14 07:15
There must have been a problem with data transmission and / or GUIs. The cluster was waiting for the oppo's reply (should have been "ponderhit"), but a stop command came instead.
Parent - By Kapaun (****) Date 2012-11-14 09:46
The cluster did not reply to H's latest move. The issue was resolved and the game was continued after 10 minutes or so.
Parent - - By Denkko (**) Date 2012-11-14 04:29
Hi, do you know around what time does this second game will start?
Thx
Parent - By Lukas Cimiotti (Bronze) Date 2012-11-14 07:12
The 2. game will start tonight 20:00 playchess server time (UTC +1).
Parent - By Lukas Cimiotti (Bronze) Date 2012-11-14 07:10

>Black is the Cluster playing with 64 cores in this game as black, Houdini is playing on 16 cores.


You are wrong. The cluster had the white pieces.
Parent - - By Quapsel (****) Date 2012-11-14 08:22 Edited 2012-11-14 08:27

> Houdini is playing on 16 cores


AFAIK Rybkas 64 cores are placed at several machines which are connected.
16 cores in one machine might bring a much better benefit than if they are placed in maybe 4 connected machines.
Do we have an astimation of the number of cluster-cores which are neccessary to give Rybka the power which she would have on a single 16-core-machine? Maybe 32 cores? If this is true, could anybody really  assume that she can produce enough strength on 64 cores to hold against Houdini 3?

Maybe we would see the more interesting and balanced matches if Houdini would have only 8 or even 4(!) fast cores.
But let's see what the next match(es) will show!
The match today 20:00 can easily be followed via Fritz-Software on schach.de in the machine-room.

Quap
Parent - - By M ANSARI (*****) Date 2012-11-14 22:16
Well a 64 core cluster should not really be confused with a 64 core computer where the memory is shared.  Obviously there is a large difference in performance when the memory is shared on the same motherboard.  I think the way a 64core cluster works is that it has 4 x 16 core computers or 8 x 8 computers running together and one is designated a Master and the other computers Slaves, and you would have the Master choose the best 3 lines and delegate the 3 Slave computers to look at the other 3 lines (on a 4 motherboard system).  So basically you would have the engine running the 3 most effective variations at full strength.  Now if you play against an equivalent 8 core engine (if this is an 8 x 8 setup), the Cluster should be quite a bit stronger than the 8 core system because it will anticipate basically the same moves, but it will go deeper in those moves.  But if it goes against an engine with a slightly different evaluation that is running 8 cores, then many of the moves would not be predicted and looking in depth at the first 3 variations will not get you much of an advantage.  But now if you go to a 16 core system using shared memory, not sure if the 64 cores could deal with it as on many moves it will be 8 cores against 16 cores for the non Cluster.  I think the cluster will only be competitive with a shared memory 16 core system when it is around 128 cores or maybe more.
Parent - - By Quapsel (****) Date 2012-11-14 23:50 Edited 2012-11-14 23:55

> Well a 64 core cluster should not really be confused with a 64 core computer


Yes, and this experiment's sense also is to show, how good the benefit to clustering several machines is.
I'm afraid, that this benefit is very small.

But some time ago Vas declared in this forum, the benefit to have cores in clustered machines is not significant smaller than that to have these cores in one machine!
We hat been very surprised then(!), and maybe this astimation simply wasn't true. :-(

Quap

PS:
Ceterum censeo that perhaps simply Houdini3 on an actual 4core-i7 should pe the opponent of 64-core-ClusterRybka!
Parent - By Leto (***) Date 2012-11-15 02:22
Or it could simply be that Houdini 3 being a much stronger engine (perhaps 120 elo or more than Rybka 4.1) provides a larger advantage than clustering.
Parent - By suj (***) Date 2012-11-15 10:44
I don't think cores has that much of an effect here. The point is rybka is missing a lot of houdini moves in the search for some strange reason and may be only Lukas will know if the other moves were searched at all and at what point they were rejected.

I could be down to wrong splits or client queues for the cluster.Running more cores means also more overheads but I could marginally help find some moves which get missed by virtue of going deeper in plies.
Parent - - By M ANSARI (*****) Date 2012-11-14 11:54 Edited 2012-11-14 11:59
Well it looks like Rybka Cluster evaluation doesn't have too much change from Rybka 4.1.  I pretty much got the same move choices with Rybka 4.1 on 8 cores, and this goes to show that H3 simply has a better evaluation and completely outplayed Rybka here.  I think it is hard for a human to try to identify where an entity (that is probably half a dozen hundred ELO's stronger than the strongest human) went wrong, but IMHO things started going really bad for Rybka here were Rybka played BxN

br4k1/p3rpp1/3p2q1/5p1p/1P2nP2/P2BQ1PP/1B2R2K/4R3 w - - 0 34


I can see why Rybka would think the Knight is annoying and needs to be removed, but really the Knight is also in the way of the white bishop and white can only be better off keeping the bishop pair especially if white's bishop covers the c2 square, where now that the bishop goes the black rook with white squared bishop have total control of c2.  Later H3 surprised Rybka with a5!

1r4k1/5pp1/p2pq3/5p1p/1P2bP2/P5PP/1B2R2K/2Q5 b - - 0 42


I think Rybka was expecting Rc8 as that dominates the c file and especially c2 square, but this simply eliminates one of white's only trump cards that might be able to draw the position which is a connected pawn.  The position might have been lost already but after a5! things quickly went from bad to totally lost.

Let's see how the other games go, but so far it doesn't look like there was much work done to improve Rybka evaluation in the Rybka Cluster.  Here it simply was totally outplayed by a much better evaluation and as we have seen many times before, hardware alone cannot always cover the deficit of an inferior evaluation.  I am still not sure how much stronger a 64 core Rybka Cluster based on several machines that do not share memory is compared to say a 16 core or 32 core shared memory setup, but what would be great is if we could have an Open World Computer Championship where everyone brings his best hardware and best book to the table.
Parent - By Quapsel (****) Date 2012-11-14 13:12

> but so far it doesn't look like there was much work done to improve Rybka evaluation in the Rybka Cluster.


Do we have an astimation what could be the benefit of that 64core-cluster?
A 4-core-i7 with what theoretical Speed would give Rybka the plattform to reach just that depth which 64-core-cluster-Rybka can reach?

Quap
Parent - By Kappatoo (*****) Date 2012-11-14 13:41
That knight is a monster. White is probably lost here in any case.
Parent - - By Uly (Gold) Date 2012-11-14 18:52
Thanks for the insight M ANSARI.

>I pretty much got the same move choices with Rybka 4.1 on 8 cores


After this I think it's safe to say that Rybka 4 wasn't a lobotomized version of Rybka Cluster, but that Vas released the best that he had.
Parent - - By Moz (****) Date 2012-11-14 21:50

> After this I think it's safe to say that Rybka 4 wasn't a lobotomized version of Rybka Cluster, but that Vas released the best that he had.


Agreed. Sadly, I think that's what many Rybka lovers (myself included) were holding on to. The idea that Vas hadn't revealed his hand with a public release, that he was saving his best cards for the private cluster.

So, Lukas, any chance you'll be entering into a partnership with Houdart to build a Houdini cluster?
Parent - By Quapsel (****) Date 2012-11-15 00:03

> So, Lukas, any chance you'll be entering into a partnership with Houdart to build a Houdini cluster?


Please let's firstly look at the question: "Which benefit really exists when doubling the number of machines (and the cores) in a chess-engine-cluster?"
And let's compare this to the benefit of doubling the core-number in one machine.
Quap
Parent - - By Kapaun (****) Date 2012-11-14 23:08
Bye-bye R5 ...
Parent - - By RFK (Gold) Date 2012-11-15 00:25 Edited 2012-11-15 00:33
Unless...he's working on something totally new  by way of a version 5!:::eek::fat::surprised::roll::wink::twisted:
Parent - - By tomgdrums (****) Date 2012-11-15 01:01

> Unless...he's working on something totally new  by way of a version 5!:::eek::fat::surprised::roll::wink::twisted:


Which would be AWESOME!

But likely?  :fat:
Parent - - By RFK (Gold) Date 2012-11-15 03:50
I don't think the guy has any other choice at this point. Maybe that's why it's taking him so long to release Rybka 5?!  Just a thought.
Parent - - By Moz (****) Date 2012-11-15 04:18

> Maybe that's why it's taking him so long to release Rybka 5


It's far more likely that Rybka 5 is vaporware and Vas has moved on to bigger and better things. I just can't see Vas adopting the Hiarcs business model. Continuing to release sad and hopelessly outdated versions of Rybka in order to profit off of the good will earned by former glories isn't his style...
Parent - By Ray (****) Date 2012-11-15 05:41

> I just can't see Vas adopting the Hiarcs business model. Continuing to release sad and hopelessly outdated versions of Rybka in order to profit off of the good will earned by former glories isn't his style...


The Hiarcs business model is pretty smart - it includes Mac and a lot of mobile platforms as well, plus a GUI, that is how they have remained relevant today despite the engine being not one of the best anymore.
Parent - - By Quapsel (****) Date 2012-11-15 12:48
It would have been fair then to tell this to his customers, who are sitting since many months ahead of the drawbridge of his castle, waiting for the bright and strengthend and space-consuming galahad riding out with a sword in the one and the new Rybka 5 in the other hand!

Quap
Parent - - By RFK (Gold) Date 2012-11-15 17:19

> It would have been fair then to tell this to his customers


I can't argue that point. His reticence  is appalling.

IMHO -It really does come down to- Rybka out of sight out of mind.

If he shows up with something -fine. If not , wish all the best wherever his career may take him. It's been a real trip.:wink:
Parent - By Quapsel (****) Date 2012-11-16 07:05

> If not , wish all the best wherever his career may take him.


+1
And it seems that we have to deal with this option.
If he is at the end of his computerchess development it would be nice if he could tell and teach about those things, he brought into his Rybka while her strength grew heavily over any other engine and even over that one he might have taken souce-parts from. This would be a real contribution to computer chess! This would give him a very honoured place in computer chess history.
Vas, how do you see this?

Quap
Parent - - By Dragon Mist (****) Date 2012-11-15 19:24
This is far more "Bye Bye Rybka Cluster Commercial", these games showing that what was offered for rent and for relatively huge amount of money way BS, to be honest. It is a shame what people payed for, and disgrace.
Parent - By Kapaun (****) Date 2012-11-15 23:06
Yes, but the cluster was always supposed to be the best R version, version 4.1 only being a crippled variant. Now we see that Cluster R is simply inferior software compared to H - so who would buy R5 now? Even in order to get the new Fritz-GUI I'd rather buy H3 again than buying any R5 which might come out .
Parent - - By donkasand (***) Date 2012-11-14 17:51
To all the statistics boffins out there, a question....

What would the score have to be a 6 game match, to say, with 95% certainty, that the winner was a stronger (elo wise) entity than the loser?

Looking forward to the answer. Or is 6 games to few to say with 95% certainty?

Regards
Donkasand
Parent - - By Leto (***) Date 2012-11-14 19:29
In a ten game match you'd need to have 5 more wins than losses in order to be more than 95% certain of superiority.  You can refer to this table:
http://chessprogramming.wikispaces.com/LOS+Table
Parent - - By donkasand (***) Date 2012-11-14 19:51
Thanks Leto

So to make sure I understand the table, in a theoretical 10 game match between A and B, with the result 4-5-1, one can say with about 86.95% confidence that A is stronger than B?

If so

does the result 3-7-0 give the same confidence level? In other words does the draw percentage contribute or not? (both examples have a plus 3 score) ?
Parent - By Patrick G. (*) Date 2012-11-15 12:21
No, because:

"The table was calculated for a draw ratio of 32%."
- - By Kapaun (****) Date 2012-11-14 09:53 Edited 2012-11-14 12:06
It was a very impressing game. I did not really expect H to win - especially not with Black. But H simply outcalculated R - even if you take into account, that R does not show the real depth. It's like I always said: Being the best chess playing entity is a matter of software, not of hardware.
Parent - - By Ray (****) Date 2012-11-15 05:45

> It's like I always said: Being the best chess playing entity is a matter of software, not of hardware.


Well it is a bit of both. Komodo 5 will never be a world leader on 1 core. On the other hand a cluster version of Micromax for example would not be a world-beater either.
Parent - - By Kapaun (****) Date 2012-11-15 11:43
Yes, but what I meant is: If you are not already top anyway, 64 cores or more won't make you top.
Parent - - By Ray (****) Date 2012-11-15 13:44
Yes I think that is right, especially with the diminishing returns of extra cores. Although I'd reword it "If you are not already close to the top, 64 cores or more won't make you top"
Parent - By Kapaun (****) Date 2012-11-16 00:16
Guess I could sign that. ;-)
Parent - By InspectorGadget (*****) Date 2012-11-15 20:44

> Well it is a bit of both. Komodo 5 will never be a world leader on 1 core.


:lol:
Up Topic Rybka Support & Discussion / Rybka Discussion / Is the Rybka Cluster hot air?

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill