Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / empirical estimate of God's elo rating
1 2 3 Previous Next  
Parent - - By skulibj (*) Date 2007-12-31 17:19
Would be interresting to see the results with some contempt factor.
Parent - By lkaufman (*****) Date 2007-12-31 22:50
Perhaps I'll try this after the version for the Benjamin match is finalized, which will have a new contempt factor.
Parent - By lkaufman (*****) Date 2007-12-31 04:19
I ran off forty blitz games (4'+2" and 2'+1") between the same opponents (Rybka Paderborn quad vs. Fritz 5.32) at knight odds, using twenty pairs of reasonable first moves for variety, and Rybka lost by 12-28 (9 wins, 6 draws, 25 losses). This was a bit disappointing, as older versions of Rybka had managed a 10.5 out of 20 score in similar tests, but different openings were used then so perhaps that was a factor, or else Rybka might have just been lucky in the earlier test. So knight odds is a bit too much for equal chances against Fritz 5.32, but even a 30% score giving knight odds to an engine that was certainly of strong GM level in blitz on today's computer is not a bad result.
Parent - By 8lrr8 (***) Date 2007-12-31 02:33
"...everything depends on the rating difference between the opponents.  It is quite possible that the current rybka with a good enough book can draw pretty often with White against anyone, assuming a book written expressly for this purpose."
"...and on this basis it is much easier to imagine a 4000 rating than would be the case with 30 move deep opening books."

yes, this reiterates what i've previously stated regarding very strong books and the nature of the elo system.
Parent - By skulibj (*) Date 2007-12-30 17:37
If I would play infinite games with god I would never get a point. So God has infinite rating. Even though there is a small chance for a random generator I would be tricked by God everytime. The same is true about most grandmasters though.
Parent - - By Jessica (**) Date 2007-12-30 23:46
I told ya'll when there is nothing for us to talk about we find other stupid things to talk and fight about!
Parent - - By M ANSARI (*****) Date 2007-12-31 00:29
I am quite sure that within 20 years chess will have an entity that cannot be beaten.  This will happen due to many factors ... but mostly due to the vast capabilities of storing and being able to look up millions of played games against powerful computer opponents.  You will reach a point where every promising line that wins will be analyzed to a draw.  Winning ideas will become less and less as more opening ideas that win will be "fixed".  Add to this fact that hardware expansion ... and engine improvements ... will make it almost impossible to lose a game that is not already lost in the opening ... you have an entity that cannot lose.  I already see this in many computer games ... Rybka on a single or dual core can easily hold a draw in a drawn game even against tremendously faster quad and eight core hardware ... even at blitz time controls where the time limit dramatically favours the faster hardware.  I am not sure what the ELO will be on such a machine ... but surely it will not be much more than 3200 ELO.  There is no doubt that chess is a draw otherwise we would have already found the one opening that cannot lose even with best play by the opposing side.
Parent - By 8lrr8 (***) Date 2007-12-31 02:37
"...but surely it will not be much more than 3200 ELO."

glad to see at least one person doesnt think i'm crazy.  :-)
Parent - - By ebutaljib (****) Date 2007-12-31 12:11

> I am quite sure that within 20 years chess will have an entity that cannot be beaten.


I assume you mean "cannot be beaten by anybody or anything". See you in 20 years. I bet you will have to eat your words :)

> Rybka on a single or dual core can easily hold a draw in a drawn game even against tremendously faster quad and eight core hardware ... even at blitz time controls where the time limit dramatically favours the faster hardware. 


But can she do it ALL THE TIME in EVERY opening?

As I explained in other post, god knows every possible outcome in every possible setup in advance. No computer in forseable future will be able to do that. There are no 32 men tablebases in sight either. So there is no way Rybka or anything else is able to get even one draw out of infinite games against god (except Mrs. god ;)) The reason why Rybka on single core can draw (from time to time it can even win!) against Rybka on eight cores is because both are flawed. By god's standard they are not only flawed, but are complete beginners. It is much, much, much, much,..., much, much, much, much bigger difference between god and Rybka than it is between Anand and a 3 year old who just learned how the pieces move.
Parent - - By lavioala Date 2007-12-31 15:02

>So there is no way Rybka or anything else is able to get even one draw out of infinite >games against god


Wrong. Even a random mover can draw 32 men bases with probability roughly 10^(-90). Rybka would do much better than that. Shortly, there is an upper limit for chess playing strength.
Parent - By ebutaljib (****) Date 2007-12-31 15:38
Maybe against 32 men tablebases but not god. He would have foreseen it and played a variation where this random moving person blunders.
Parent - - By lavioala Date 2007-12-31 01:50
I don't know about the God, but a 32 tablebase will not have an infinte ELO score. Even a random mover can draw against it at a rough rate one per 30^60~10^90 games. That is a difference of about 60,000 elo points. In the case of Rybka, an empirical estimation would be 3^60~10^30 and the difference of 20,000. Given Rybka's 3000 rating, the perfect mover (32 tablebases) is 23,000 elo strong (playing both white and black).

Laviola
Parent - - By Vinvin (***) Date 2008-01-01 03:39
Even a random mover can draw against it at a rough rate one per 30^60~10^90 games.

Where this idea come from ??
Parent - - By vroger007 (**) Date 2008-01-02 11:14
Hi Vinvin,

I think the idea comes from the fact that on average there are 30 legal moves in any position,
and an average game lasts about 60 moves.  Thus there are 30**60 "possible games" of 60 moves,
and a random mover will "succeed" in playing the "correct move sequences to draw" in one of those
10**90 games .

Happy New Year,
Roger
Parent - - By Vinvin (***) Date 2008-01-02 19:18
I think this idea is completely wrong here.
When playing against "god" the game would last at least 1000 moves when he try to win.
Parent - By Uri Blass (*****) Date 2008-01-02 20:21 Edited 2008-01-02 20:24
You do not know it.
It is possible that white has a way to force draw in 60 moves by repetition and every way of black to prevent it is losing.

I also see no reason to assume that god tries to win.
If you talk about the perfect player you may be right but it is also possible that the perfect player can risk losing against the random player in order to avoid a draw.

The question is if you allow the perfect player to use a strategy that is dependent on the opponent.

Uri
Parent - - By Uri Blass (*****) Date 2008-01-02 20:26
I see no way to get an estimate for Rybka.
If you play a match between rybka and the perfect player then
Rybka may be deterministic in some mistakes and perform worse than the random mover against the perfect player.

Uri
Parent - - By Banned for Life (Gold) Date 2008-01-02 23:14
Rybka may be deterministic in some mistakes and perform worse than the random mover against the perfect player.

I take this as a given and believe a perfect player could take advantage of this to win every game.

Alan
Parent - - By lavioala Date 2008-01-03 00:50
>>Rybka may be deterministic in some mistakes and perform worse than the random >>mover against the perfect player.

>I take this as a given and believe a perfect player could take advantage of this to win >every game.


>Alan



You cannot take this as given. Rybka is probably non-deterministic (no more than Crafty is, see the discussion about this on Crafty) and it will probably play better than the random mover against the perfect player.

From ponder hits statistics through a lengthy argument one can deduce that Rybka at most plays 2 perfect moves out of 3 and at least 1 out of 20. Therefore the perfect player is very roughly 5,000-40,000 ELO points stronger than Rybka.

Laviola
Parent - - By Banned for Life (Gold) Date 2008-01-03 01:00
The only time you know when you see a perfect move is when it leads inescapably to a draw or win. In the majority of the game, a ponder hit that 37 GMs say is the right move may not be optimal. It is very hard for GMs to understand some required moves in long sequences from 6-man TBs so there is no reason to believe that these same GMs can see a long sequence in a much more complex position.

As far as deterministic mistakes, there are many 6-man TB positions where Rybka will never select a move that leads to the best result, so iin these positions, a random selection could produce a better result.

Alan
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 01:44
I perfect move in a drawn position is any move that leaves the position drawn.  It might not be an "intelligent" perfect move, but it's still a perfect move.

It would be interesting to consider what percentage of drawn games in such tournaments as Wijk, Linares, and Dortmund nowadays are objectively perfect games--I bet it's more than 10%, possibly even close to 50%.  Of course, if these players were playing against God, He would make perfect moves that make things far more difficult for the GMs...
Parent - By Uly (Gold) Date 2008-01-03 01:58
Maybe we need two definitions:

Perfect move: A move that doesn't change the outcome of the game from the actual position (Once the opponent blunders and enters a lost game, you keep doing winning moves, etc.)

Optimal move: A move that maximizes your chances of winning in a drawn position and your chances of drawing in a lost position.
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 01:41
I can tell you with 100% certainty that Rybka plays far more than 2 out of 3 perfect moves in most of its games.  Keep in mind that the positions out of most openings are objectively draws, so all one has to do to be perfect is to play a drawing move.  I would guess that the ratio is probably more like 9/10 against good opposition, maybe more in Rybka-Rybka games.  It is probably lower against lessor opposition because Rybka might not always find the correct winning moves when the opponent makes a mistake.
Parent - - By lavioala Date 2008-01-03 01:58

>I would guess that the ratio is probably more like 9/10


I doubt this. Then, in order to keep the difference at only 50 elo points, Fritz11 would have to have a ratio like 0.88 perfect moves, and that would hurt ponder hits at 0.8 and more. It would be a long argument, but the ponder hits exhibited among the engines indirectly shows the highest value of the best engine (Rybka) percentage of perfect moves, roughly 2/3.

Laviola
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 02:54
No, the ponder hits among engines has almost nothing to do with the percentage of perfect moves.  You are assuming that each position has only one perfect move, which is clearly wrong.  The majority of positions will have more than one perfect move; which move is played depends on the style of the engine.  Of course, most coincidence statistics among engines are over 50%, and that's because many of these perfect moves are clearly "bad" moves in that they make things more difficult for the side making them.  In other positions, which of the perfect moves is chosen is simply a matter of style.
Parent - - By lavioala Date 2008-01-03 03:12 Edited 2008-01-03 03:28
Actually we don't know how many perfect moves are per position beyound 3-4-5-6 tbs. On the other side, some perfect moves are better than others, they give the same result, but not the same length of the game. I assume that the winning/drawing engine would go to the shortest path to the result in case it is advantegeous to it. That is why ponder hits are important in this case. Otherwise we would have ponder hits of order 0.3 and not typical 0.6

Laviola
Parent - By turbojuice1122 (Gold) Date 2008-01-03 05:37
But since such paths are far beyond the horizon of any engine on normal (i.e. not quantum) computers forever except in very obvious cases, it all comes down to the matter of style--no engine will go to the shortest path and it will not know which case is advantageous, so the "knowledge" of such things defines its style.  Objectively speaking, if chess is drawn, then most drawn positions arising from "typical" play will definitely have a number of different drawing moves.  Anyway, I think there should be absolutely no doubt that ponder hits has nothing to do with the percentage of perfect moves an engine plays when by "perfect" we mean not changing the objective evaluation of the position.
Parent - - By Banned for Life (Gold) Date 2008-01-03 02:16
No, that 100% guessing on your part and there's no certainty at all. You suspect, but don't really know that the opening position is a draw. Likewise from any positions following the opening. It is very possible that moves that look like drawing moves to you or any available engine are actually winning or losing moves if only you could see the supporting 300 ply tree that showed the win against all possible opponents moves.

Given that you really don't have any clue which moves go into which category, its rather presumptuous to say that 90% of the moves from Rybka don't change the outcome of the game.

Regards,
Alan
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 02:51
It's not mathematically proven that the opening position is drawn, and we probably need 32-man tablebases to do this.  However, the fact that there is a substantial increase in drawing percentages as the playing level goes from 2600 to 3000 in engine matches is a very, very good bit of support for the "obvious" intuitive conclusion that the opening position must be drawn.  As for the positions, we have good indications of their drawish nature by the fact that the best human opponents are so willing to go into them time and time again, and very good indications based on the fact that when engines are left analyzing such positions for hours and days, the absolute value of the evaluations usually goes down, not up.

Don't worry--this is all very far from saying that chess is "solved".  It could be that 90% of games at tournament time controls between different quads running Rybka 3 will objectively be perfect games (in fact, I wouldn't be surprised if it turned out that this was the case), but nonetheless, God would make minced meat out of such players if involved in such games.

But back to the beginning, just imagine how absurd things would be if one out of three Rybka moves was a ?? move!  There's absolutely no way that this could be the case--if it was, then the drawing percentage of engine matches would be near zero (random walk argument).
Parent - - By Banned for Life (Gold) Date 2008-01-03 04:12
If we define any move that goes from a win to a draw or a draw to a win as a ?? move, there may be many non-obvious members of this set. Any member with a distance to result of > 100 is probably hidden from both man and machine, so the fact that people or engines aren't aware of them wouldn't be significant.

This is all OK by me. It allows me to insist that only 1. b3 is a win for white, and nobody can prove me wrong! :-)

Regards,
Alan
Parent - By turbojuice1122 (Gold) Date 2008-01-03 05:40
I've always thought that the great thing about 1.b3 is that it immediately puts the game into familiar territory for one used to playing it.  This isn't the case with other moves, which can develop completely different themes based on whatever is black's reply on the future moves.  Even with 1...a5, it seems like there is a general "course of the game" resulting from 1.b3, and that's especially good for you, since you probably have more intuitive knowledge of what types of things work and what types don't than almost anyone in the world.
Parent - - By BB (****) Date 2008-01-03 11:27
It could be that 90% of games at tournament time controls between different quads running Rybka 3 will objectively be perfect games (in fact, I wouldn't be surprised if it turned out that this was the case)

If we assume that the (effective) "length" of a game is 25 ply [a bit low, I would say], then this 90% rate for perfect games (amongst draws?) seems to imply perfect moves at a rate of 99.6%. I find this somewhat on the high side.
Parent - By turbojuice1122 (Gold) Date 2008-01-03 13:35
I think that the 99.6% estimate is on the high side.  I think that among games that aren't objectively perfect games, there are lots of mistakes made, and this is why God would easily win quite a bit over 90% of games against Rybka 3--God would go for games that would tend to result in these sorts of lines (after all, even if God doesn't have perfect knowledge of the future, He still has a 20 billion game database of all chess games ever played and, of course, extremely effective ways of cataloging things).
Parent - By SR (****) Date 2008-01-03 18:57
As I have argued elsewhere a 32 tabel base would have a rating more or less identical to that of the opposition. As I already argued this is valid even if the opposition has less than an elo of 2000.    If each move is chosen randomly among all "optimal" moves (where optimal is defined with solely with regards the best outcome with regards to V. Neumann's min-max strategy) the table base would normally - even against a quite weak player - get a somewhat inferior position and as long as the player makes sensible move will never (normally) be in danger of loosing the game. All games (except a very small portion where the random moves for the table base happens to generate tricky positions that are hard to answer correctly, or the rare case where the human makes a blunder) will end in a draw.

One game against a white table base that follows some simple deterministic (but random like) strategy might lead to a game like
1.a4, e5 2. d4,Nf6 3.Bb5, Nc6 4.Bf1,Nb8! 5.Bb5,Nc6 6.Bf1,Nb8 7.Bb5 draw where both parties (I suppose) played perfect chess.
Up Topic Rybka Support & Discussion / Rybka Discussion / empirical estimate of God's elo rating
1 2 3 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill