Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka elo per ply
1 2 3 Previous Next  
Parent - - By Uly (Gold) Date 2008-03-12 21:51

> but, now, how do i use scid to delete duplicate games....


File > Maintenance > Twin Checker Window

Then click on the "Delete Twin Games" button, and un-check everything but "Same Moves." then click "Delete Games."
Parent - - By tomski1981 (*****) Date 2008-03-12 21:52
it gives me an error that the database is read only... i checked the attributes on the file, and it seems fine. did you ever have this problem?
Parent - - By Vempele (Silver) Date 2008-03-12 21:57
Could be it's open in some other application. That should give a different error, though.
Parent - By tomski1981 (*****) Date 2008-03-12 22:04
i thought of that too, but so i closed other applications... and even did what vytron suggested... :(
Parent - - By Uly (Gold) Date 2008-03-12 21:58
No. Maybe you can try opening the PGN with Notepad, and copying all the games then pasting them to a new file, and save that as PGN (not txt!), I hope your computer doesn't explode (it may due to Notepad not being made to handle long files, like 1000 games as PGN...)

After all this hassle, I wonder if it would be better to accept the random noise and live with the duplicate games...
Parent - - By tomski1981 (*****) Date 2008-03-12 22:03
i tried that, didn't work. when i hit Ctrl+M, the description box reads:"This database is read-only; it cannot be altered."

once we get this going, it'll be worth it... just hang in there :)
Parent - - By Uly (Gold) Date 2008-03-12 22:06
Can you attach the PGN to one of your posts so I get a look at it?
Parent - - By tomski1981 (*****) Date 2008-03-12 22:08
sure, i'm assuming you're talking about the big-@ss pgn database file.. here it is:
Attachment: RPly2v1.pgn (420k)
Parent - - By Vempele (Silver) Date 2008-03-12 22:27
Checked with Chess Assistant - no doubles found. Weird and inconsistent results when I changed the settings, though.

Also, looks like CA uses a signed 7-bit integer type for "round". :)
Parent - By tomski1981 (*****) Date 2008-03-12 22:36
not sure what that means :) i think i goofed, i used HS-Rybka opening book.. i should have used Fritz's.. it's more balanced i think
Parent - - By Uly (Gold) Date 2008-03-12 22:41
Ok, I forgot that Scid treats PGNs as read only, and so you have to export it to a Scid Database, delete twin games and then export as PGN.

So:

Do File > New and name your new .si3 file

File > Open and select the file you created

File > Open and select your PGN

Windows > Database Switcher

Now, on the new window, drag the PGN into the white rectangle so the games are added to the Scid database.

Now, right click the PGN and close it.

And now you can do the delete twins thing. After that, just export back to PGN:

Tools > Export All Filter Games > Export Filter to PGN file

And you are done!

Also, guess what, the file you attached has no duplicate games :-P - But this may prove useful when you're managing 1000 games...
Parent - By tomski1981 (*****) Date 2008-03-12 22:50
thank you and vempele for your patience... i'll be back home later to try it... and guess what, i have to redo these 100 games anyway.. i used the wrong opening book..hahha.. anyhow, thank you again
Parent - - By tomski1981 (*****) Date 2008-03-12 22:17
okay, i got it to work. i created a new Scid Database. Opened the pgn file in notepad. copied it. then i opened the clipboard in scid, Edit>"paste clipboard text as PGN game" and then i pasted it there... imported and i think it will work
Parent - By tomski1981 (*****) Date 2008-03-12 22:18
never mind, that only imports one game!! argh!!
Parent - By tomski1981 (*****) Date 2008-03-12 20:34
alright, i'm restarting. variety of book all the way up, learning way down... and i'm going to delete duplicates.. i think that's reasonable.. let's see what happens
Parent - - By AsosLight (***) Date 2008-03-13 11:29
i wouldn't be so aware about openings as you. you should have just playable and complicate positions. the opening book should follow a real match situation else you test two different things at the same time. opening lines results and ply elo difference. keep a good opening book instead but cut it early around move 10 maybe. hash tables=0 should not be so simple but if there isn't other way to clear hash before each game ok.
Parent - By tomski1981 (*****) Date 2008-03-13 13:37
i found that the minimum for hash tables is 1MB
Parent - - By tomski1981 (*****) Date 2008-03-13 20:08
Redone.. no duplicate games:

  2plyVs1ply +60/  =35/-05   77.5%
  3plyVs2ply +59/  =30/-07   77.1%
  4plyVs3ply +51/  =47/-10   69.0%
  5plyVs4ply +53/  =41/-10   70.7%
  6plyVs5ply +53/  =45/-16   66.2%
  7plyVs6ply +42/  =55/-09   65.6%
  8plyVs7ply +46/  =50/-08   68.3%
  9plyVs8ply +44/  =53/-09   66.5%
10plyVs9ply +77/=155/-16   62.29%
Parent - - By turbojuice1122 (Gold) Date 2008-03-14 18:15
I have thought of a much better (I think) method than what we're currently discussing.  The percentages won't be comparable, but it will definitely yield a good estimate of elo gain per ply.  Unfortunately, the method is far longer and much more extensive.

(1)  Create a test suite of 100 positions
(2)  Find an opponent that is (obviously) weaker than Rybka, but give that opponent a ply bonus that you believe will make it the same strength (roughly) as Rybka (the result of your first test will confirm or refute your hypothesis).
(3)  Run an engine match from this test suite at such and such low depth (the total will, of course, be 200 games: once white and once black for each position).  If you choose, say, Fritz 11 against Rybka 2.3.2a on single processor, you might have Rybka at depth 4 and Fritz 11 at depth 6.
(4)  If the score is equal, then drop Rybka down by 2 ply, keeping Fritz 11 at the same ply and repeat the test.
(5)  Repeat the test with Rybka raised one ply from (4).  From this, you will see the elo gain from (4) and the elo loss from (3).  Then repeat the test with one ply higher than (3) and then 2 ply higher than (3).
(6)  Next, run a test with the ply of Fritz 11 raised by 1 or 2 and raise the ply of Rybka in (4) by 1 or 2 and repeat the five tests.
(7)  Repeat these steps until it takes far too long to reasonably do this
(8)  Repeat all steps above with different programs and average the results of Rybka's elo increase from one ply to the next.

This will probably take at least weeks, possibly months, but I think that the results would be worthy of journal publication.
Parent - - By tomski1981 (*****) Date 2008-03-14 20:17
hmmm, very interesting. but i will have to reread this a few times to make sure i grasp the whole concept. your idea gave me an idea, but first i need to know something: is it possible to play rybka at a fixed ply depth while letting another engine use all the time it needs? (is there a config file that let's you do that?)
Parent - - By turbojuice1122 (Gold) Date 2008-03-15 03:53
I don't know what you mean by "all the time it needs".  In truth, "all the time it needs" is infinitely long because what seems clear at ply 20 could turn out to be a blunder in a 40-ply search.  Basically, the general idea of what I stated above is that you would want Rybka to be playing non-Rybka engines (Rybka vs. Rybka results may tend to be exaggerated toward the side with the advantage) and compare how Rybka's result change as you increase the ply by one and leave the other engine constant.  This must be done near regions where the strengths are reasonably similar or else the statistical errors will be too large.
Parent - - By tomski1981 (*****) Date 2008-03-15 12:25
sorry, that was a very inaccurate statement and not what i mean at all. what i'm really asking is this: is there a way to set up a match where one engine is limited by ply depth, and another by time?
Parent - - By turbojuice1122 (Gold) Date 2008-03-15 14:41
Ahh!  Okay, that makes sense.  I don't think there is a way of doing this in the Chessbase GUIs, but there might be some other GUIs where you can do this.
Parent - By Uly (Gold) Date 2008-03-15 16:20

> Ahh!  Okay, that makes sense.  I don't think there is a way of doing this in the Chessbase GUIs, but there might be some other GUIs where you can do this.


Someone needs to suggest this for the Rybka GUI, as I think we should be able to make these kind of experiments with it.
Parent - By Hetman (*****) Date 2008-03-15 15:13
Good work :-).
I think that the border(limes) of the chain it is 50%.

77.5% it is about 200 Elo difference ?

Rgds
Hetman
Parent - By BB (****) Date 2008-03-12 09:48 Edited 2008-03-12 10:44

> I don't remember the exact details


Just after I tracked down much of these, I found a previous Forum post (that includes other topics) that has a number of links. [See especially the "serious test" of Heinz].
Parent - - By Vasik Rajlich (Silver) Date 2008-03-13 18:11

> Intuitively, though, there has to be an imperceptible difference between a search that goes 50 plies and one that goes 51.


This is really hard to speculate about. It's related to the question of what is God's elo. At some point, the weaker engine is just going to be able to hold the draw in most of the games.

Basically, if you have datapoints like 12 ply vs 13 ply and 13 ply vs 14 ply, it's quite a stretch to try to apply this to 50 ply vs 51 ply.

Vas
Parent - By turbojuice1122 (Gold) Date 2008-03-13 18:29
I think this is why Larry was talking about the proportional difference going down linearly with depth, which I assume he meant that it could potentially be modeled with an inverse function.  It's difficult to say if it can be modeled with y = a/x + b, but I think it's very reasonable to expect that it can be modeled fairly well with y = a/x^c + b.  The value of b should be 0.50 if y is winning percentage for the higher side and x is depth of higher side.  It could be more complicated than this--if we have the higher side being 2 ply or 3 ply or more ahead of the lower side, we should get the same results as x becomes large, but have noticeably different results if x is small.  One of these days, after I finish a certain correspondence game, I will be interested in checking all of this out...
Parent - - By 8lrr8 (***) Date 2008-03-12 04:47
are u in possession of actual data showing the difference in elo rating btwn rybka 28-ply vs rybka 30-ply?  i'm guessing no (who has the time to run such games at those ply-depths?).  in that case, no one knows where the diminishing returns starts to kick in.  however my guess is the curve starts to flatten by the 30-ply mark, tops.  this assumes an engine w/ a good search to begin with (which means fritz 5 at 30-ply doesnt count).

am i the only one who would be shocked if rybka 30-ply achieves 30 pts (out of 50 games) against rybka 26-ply?
Parent - - By lkaufman (*****) Date 2008-03-12 14:16
I would be shocked if the result were that close. I would expect 30 ply vs. 26 ply to be something like 15 ply vs. 13 ply (I think the ratio is the best guide). A 60% score is 70 Elo, which is about what I would expect for only a 1 ply difference around the 14 ply depth.
Parent - - By Nelson Hernandez (Gold) Date 2008-03-13 20:14
If you use a ratio that implies a linear relationship, when in fact the relationship almost certainly has to be curvilinear.  In your example you may well be right for those two data points, but the same ratio would most likely not apply to, let's say, ply 45 vs. ply 39.  It expect at that level the ELO difference between the two would be close to (but not quite) zero.
Parent - - By lkaufman (*****) Date 2008-03-13 21:45
We can only guess, but my opinion is that even at 45 vs 39 ply the result would be much closer to the same as 30 vs 26 than to zero. Only when the number of ply approaches the typical length of a chess game would I expect that the results would really start to trend towards zero. 
Parent - - By tomski1981 (*****) Date 2008-03-13 22:04
i'm no expert, so let me see if i understand this

you give 30vs26 figure.. let me suggest something close, but that will allow me to grasp this more easily:
30vs25

if i understand you correctly, 30vs25 would be the same roughly as 60vs50, right? so going backwards should be the same as well, same as 24vs20; 12vs10; 6vs5?

if so, we can do a quick test.. let's see if 12vs10 yields similar results to 6vs5.
Parent - By Nelson Hernandez (Gold) Date 2008-03-13 22:53
While that comparison might be interesting, Larry's last point about the extremes reaching the length of a chess game is actually critical, and it is the same point Vas made in a different way, and I have also made in a different way yet.  We're all saying basically the same thing: that in fact the relationship is curvilinear out to some point, beyond which the playing limits of chess exert a different influence on the curve.  Unfortunately current technology does not permit a practical experiment to prove this hypothesis.  If you stop the analysis at 17 vs. 16 or even earlier you do not reach a point where the engine is analyzing the endgame straight out of book.
Parent - - By lkaufman (*****) Date 2008-03-14 02:05
Actually 12 beats 10 worse than 6 beats 5 ply. This ratio idea is only approximate, but it is reasonably accurate in the range where real games are played by today's computers.
Parent - - By Uri Blass (*****) Date 2008-03-14 07:19
I think that the problem is also that plies are different and I believe that rybka earns less from one ply than other programs because of more agressive pruning.

I made some comparisons between rybka and movei at fixed depth(40 games every match)

opening used 1.a3 1.a4 1.b3 1.b4,...1.h3 1.h4 1.Na3 1.Nc3 1.Nf3  1.Na3 1.Nh3

fresults are against movei
Rybka 1 plies-Movei 3 plies 30.5-9.5(+27 =7 -6)
Movei 4 plies-Rybka 1 plies 2 25-15 (+22 =6 -12)

When rybka says 1 ply then it means that Rybka efective depth is something close to 3.7 in movei terms.

Rybka 5 plies-Movei 7 plies 2 19-21 (+18 =2 -20)

If you translate rybka to depth to depth of another program like movei then you have something like the following.

f(1)=3.7
f(5)=6.9
f(x)=0.8x+2.9

f(6)/f(5)=7.7/6.9
f(12)/f(10)=12.5/10.7

f(12)/f(10) is higher so it is no surprise that 12 beats 10 by bigger margin.

If you want similiar proportion to  6 vs 5 plies you need to go to 16 vs 14 plies

f(16)/f(14)<f(6)/f(5)<f(15)/f(13)

Uri
Parent - By lkaufman (*****) Date 2008-03-14 14:19
Yes, good point as usual. If the comparison is made as you suggest, my idea about the depth ratio being a good indicator would look much more accurate.
Parent - - By AsosLight (***) Date 2008-03-14 15:47
Very clever logistic indeed.
Your observation strengthen my opinion that with modern engines prunning destroy any mean of results and you can't measure anything not to mention the ply count problem.
By the way you end up with a linear equation:  f(x)=0.8x+2.9 but the phenomenon can't be linear both theoretical (if we asure that movei is a perfect brute forse and evaluation program) and practical. i can prove that even for the theoretical relation but the text window is too small :-)
Parent - By Uri Blass (*****) Date 2008-03-14 17:23
Movei is not perfect brute force program.
It does late move reduction and null move pruning and other pruning but rybka clearly does more pruning.

Uri
Parent - By Hamlet (**) Date 2008-03-13 23:03
What is difference between the point when we leave opening phase and enter to tablebases?

Right now it is something about 30-40 moves or 60-80 ply but quite soon in some lines it might be
20-30 moves and we are already seeing how randomvisitor is doing really deep analysis (around 30 ply in the middlegame):
http://www.chessgames.com/~RandomVisitor

=> I agree with Uri that top level correspondence chess is dead quite soon.

during next ten years we will see 7 men tablebases, it will solve certain endgames that are difficult for computers right now like
two rooks against two rooks + pawn or two minor pieces + pawn against rook + minor piece or rook + pawn against rook and two pawns.
+ top program that is able to search 35+ ply during the cc game ( I assume that we will get one ply more every 18th month => 6-7 ply more in ten years).

here is one top level cc tournament:
http://www.iccf-webchess.com/EventCrossTable.aspx?id=10699

quite many draws and only two decisive games. 
Parent - - By tomski1981 (*****) Date 2008-03-13 23:21
larry,

when rybka is set up to play ply vs ply, does she still prune, or is it brute force?
Parent - By lkaufman (*****) Date 2008-03-14 02:07
There is no "brute force" Rybka. All tests are with normal selectivity.
Parent - - By 8lrr8 (***) Date 2008-03-14 01:56
this would imply u feel the draw window in chess is extremely narrow, on par w/ the likes of shogi.  i really dont think this is the case.

i think what will happen in a rybka 30-ply vs. rybka 26-ply match, is that for every game, the 30-ply version will achieve an advantange throughout the game.  and by the endgame, it may have, say, a +0.4 score, but the 26-ply will be able to hold nonetheless.
Parent - - By lkaufman (*****) Date 2008-03-14 02:14
Well, I don't think that the draw window is "extremely narrow", but I believe it is narrower than is generally believed. Sometimes the weaker side can draw when the score is +0.4 (with perfect play), but sometimes not. I believe that many times a seemingly very small advantage is enough to win; it all depends on the type of position, in ways that no one understands. I think that a 30 ply program will score plenty of wins with White, and hardly ever lose with Black, against a 26 ply program. White's advantage, although not (probably) decisive, is big enough that many serious defenses may turn out to lose by force. If it turns out that in general the bishop pair is a winning advantage (which is quite possible), Black may have a tough time defending in the opening.
Parent - - By 8lrr8 (***) Date 2008-03-14 02:27
"I think that a 30 ply program will score plenty of wins with White, and hardly ever lose with Black, against a 26 ply program."

wont this be quite hard if black adopts a "play for a draw" mentality?  like playing the petroff, or french, or berlin wall?  basically locking up the position, or choosing a super passive/quiet opening, hoping to "rope-a-dope" white until liquidation of pieces is forced.  and then getting into a marginally worse endgame but nonetheless able to hold the position?
Parent - - By lkaufman (*****) Date 2008-03-14 03:33
My impression from watching top level human games is that White can often (not always) put serious pressure on Black and retain some winning chances deep into the endgame, no matter what defense is chosen. I think this will still be true with 26 ply chess programs. It's true that many (maybe most) of these endings may be theoretically drawn, but if you have a serious advantage in a complex endgame and are also outsearching your opponent by 4 plies you are going to win a decent number of them. It may be that trying to force things with 1e4 won't work at this level and perhaps 1d4 will be the main move, aiming for less of an advantage but in general avoiding forced simplification before achieving an advantage. We'll see in a few more years I guess. 
Parent - - By 8lrr8 (***) Date 2008-03-14 04:13
"My impression from watching top level human games..."

are u referring to correspondence games or OTB (at classical times controls)?  because in the latter the ratings of humans are far below that of the machine, as u know.

"...and are also outsearching your opponent by 4 plies you are going to win a decent number of them."

just to clarify, the 30-ply vs. 26-ply comparison only holds true in the middlegame.  once the endgame is reached, the 30-ply ryba is probably searching around 36 or 40-ply, in which case for this test to be fair, the shallower searching rybka will need to be searching 32 or 36-ply (i.e. maintaining the 4-ply difference).
Parent - - By tomski1981 (*****) Date 2008-03-14 04:16
i'm not sure if i understand you're reasoning that a 30-ply rybka searches deeper at any point in the game (in this case the endgame). i was under the impression that 30-ply rybka is capped off at 30 ply and will not search deeper
Parent - - By 8lrr8 (***) Date 2008-03-14 04:27
i think 59-plies is the max.
Parent - By Uly (Gold) Date 2008-03-14 14:00

> i think 59-plies is the max.


I think that with extensions, the 59ply Rybka may extend a specific variation way beyond that, maybe up to 80ply or more, while most other variations are pruned and reduced and not searched more than, say 8ply.
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka elo per ply
1 2 3 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill