Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Chess Engine Rating Accuracy
- - By bayselo (*) Date 2007-12-26 17:39
Which is the the most accurate game control to use when testing different chess engines ratings, A game with a fixed time control for both sides (eg. 3 seconds per move) or a game with a fixed search depth for both sides (eg. 7 plies per move)?
Parent - - By turbojuice1122 (Gold) Date 2007-12-26 17:48
Fixed time control is far more accurate.  Fixed depth has pretty much no accuracy at all, as different engines measure depth differently.
Parent - By Uly (Gold) Date 2007-12-26 21:56
But fixed time will nullify the part of the engine that manages time and is part of the engine. Also, if you are running a fixed time of 45 seconds per move, in obvious recaptures that the engine would have played instantly, it's still going to take 45 seconds, so that's losing time unnecessarily.

So, I think that both methods are bad (though fixed depth is worse.)
Parent - By Nelson Hernandez (Gold) Date 2007-12-26 19:47
Echoing Turbo.  Every engine has its own search characteristics.  Some get to deeper depths in the same length of time, and that characteristic (and how it is implemented) are what make the engine what it is.  If you neutralized the search efficiency the author built into his program by only going to a fixed depth you would get some very warped results.
Parent - - By Vempele (Silver) Date 2007-12-26 20:07
I give you the chess-solving extension: every move that doesn't end the game (i.e. mate, stalemate etc.) is extended by one ply. That'll beat every engine in fixed-depth matches, and as a bonus also solves chess every time it plays a move.

Flipping a coin is arguably better than fixed depth. At least nobody will have any illusions about the validity of the results. :)
Parent - - By BB (****) Date 2007-12-27 05:36

> Flipping a coin is arguably better than fixed depth.


Flipping a coin need not be that unbiased. See this press release or the 5MB pdf maths analysis (with pictures of experimental devices) about this. For instance, "Diaconis himself has trained his thumb to flip a coin and make it come up heads 10 out of 10 times."
Parent - - By Uly (Gold) Date 2007-12-27 19:21

> Flipping a coin need not be that unbiased.


I think he was referring to unbiased coin flipping. Yet again, fixed depth is still worse than biased coin flipping.
Parent - By BB (****) Date 2007-12-28 05:44
I think he was referring to unbiased coin flipping.

Some of us are always looking for an edge. :)
Up Topic Rybka Support & Discussion / Rybka Discussion / Chess Engine Rating Accuracy

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill