Rybka Chess Community Forum
Would you rather have Rybka 3 ASAP or wait for persistent hash to be effectively implemented? (Closed)
|Persistent hash can wait--give me Rybka 3 ASAP!
|I can wait for persistent hash
Given the choice, would you rather have Rybka 3 released soon without persistent hash, or do you see it as worth the wait to have a reliable persistent hash put in?
As for me, I'd rather wait for an effective implementation of persistent hash, especially since I like to analyze openings and early middlegame positions, and this would be an incredibly valuable analysis tool for such things. I would guess that it would also be very helpful for correspondence players.
Of course, it would be natural to hear what Vas has to say about this, i.e. how much extra time it would take for an effective implementation of this. It might not take all that long...
We're definitely planning this. It's not a huge task, maybe a week (including all tinkering, etc).
If a week is all it takes, then by all means let's wait the extra week. Those of us who use Rybka for deep analysis and annotation will find this a godsend.
Like voting for Dan Quayle, it's a no-brainer ...
Just out of curiousity what difference would it make in terms of elo strength in actual play, roughly?
On the rating lists, 0. In a thematic match, up to several hundred points (in the course of LOTS of games). On Playchess? Depends on your book.
> On the rating lists, 0.
Yes, but this is a flaw on the rating lists themselves. For example, Pro Deo already comes with learning from +6000 games, and Ed said that this amounts to about 30 ELO points. Also, you can switch this learning on while avoiding new learning. But the rating lists just disable learning even though any user downloading Pro Deo fresh will have a stronger version than the one on the rating lists.
I think that there should be a different rating list, that includes engines with learning ON.
> For example, Pro Deo already comes with learning from +6000 games, and Ed said that this amounts to about 30 ELO points.
1. That's far too few games to get _anywhere_ near 30 Elo. Ever. The testing groups use too many different books for that to work. Unless it's eval weight learning (though it'd still be close to unbelievable).
2. If it's worth 30 points, why not just build it into the executable?
3. I tried and failed to find your source. Mind posting a link?
Ed claimed that the 6700 games are only Pro Deo games, and top quality. It was actually better than trying 3'000'000 games, if Pro Deo didn't play on them.
> Unless it's eval weight learning (though it'd still be close to unbelievable).
Yes, eval weight based on the results of games. So, Pro Deo will avoid moves that lost in the past and will keep playing winning moves.
> 2. If it's worth 30 points, why not just build it into the executable?
For simplicity. Also, the learner is in the executable, but the eoc files are of course outside, so you can change them and make yours.
> 3. I tried and failed to find your source. Mind posting a link?
Ok, I apologize since I should have said "20-30" ELO, but here's the link:http://www.top-5000.nl/prodeo.htm"Since the EOC approach is an open source engine programmers can easily add the algorithm to their own engine and add 20-30 elo points without too much of a hassle."
Another question about the persistent hash: will it understand such things as reversed positions? For example, if we let it sit for a few years on Suj's machine looking at the position after 1.e4 e5 and concludes that white has mate in 46 moves, will it come back with the correct evaluation from the persistent hash if we put in the position 1.e3 e5 2.e4 as black having a forced mate in 46 moves?
And don't forget mirror positions, but these only appear after both sides have lost castling rights.
I don't see any way to handle this, at least without a massive loss in efficiency.
After castling rights disappear, the same position has 4 versions:
r6N/ppp1k2p/2n1bn2/6Bq/8/3P1NpP/PPP2bK1/R2Q3R w - - 0 19
r3q2r/1kB2ppp/pPn1p3/8/Qb6/2NB1N2/P2K1PPP/n6R b - - 0 1
N6r/p2k1ppp/2nb1n2/qB6/8/PpN1P3/1Kb2PPP/R3Q2R w - - 0 1
r2q3r/ppp2Bk1/3p1nPp/8/6bQ/2N1BN2/PPP1K2P/R6n b - - 0 1
Mirror position with reversed colors.
So, when you analyze a position, you're actually analyzing 4 of them. The real question is how likely are these 3 later positions to appear? If we're never going to see them on the board, having their analysis is just wasting space.
HashKey[Piece][Square] = ~HashKey[Piece][Mirror(Square)];
I guess you're right, this should work.
For black/white mirroring, it's actually something like:
HashKey [Color_Mirror (Piece)] [Mirror (Square)];
Black/white mirroring seems worthwhile, you just keep two hash keys as you search. Left-right mirroring would only apply to endgames and probably isn't worthwhile.
Anyway, if there are no gotchas, we'll probably do this.
Let's get 3.0 out soon! Save the persistent hash for release 3.1 (unfortunately, I don't think this poll will influence the actual release date...)
That's a good point about 3.1. However, there is also the question of how much has Rybka 2.3.x actually improved? I know that it's something like 50 elo points in the rapid tests at 2000 elo strength, but just looking at the few games played in Paderborn, I don't get the impression that its objective strength has really improved significantly--it seems like the improvement is not more than of what I had the impression with 2.3.2a compared with 2.3.1, and that improvement has turned out to be about 15 elo (when it was actually stated that the improvement in tests was something like 50 elo). Thus, there might not really be much of a reason for us to want the newest version so soon.
> and that improvement has turned out to be about 15 elo (when it was actually stated that the improvement in tests was something like 50 elo).
I think it's because the improvement of Rybka is about 50 ELO under laboratory conditions.
However, Rybka will have to go and play outside the laboratory, in where the improvement isn't as much.
I really don't know why I'm in a hurry. I couldn't even tell the difference between two programs that are only 100 elo apart anyway. :-)
> I couldn't even tell the difference between two programs that are only 100 elo apart anyway.
I couldn't tell the difference from players that are +750 ratings point apart, if they played individually against weaker opponents with the same rating difference...
Rybka 2.3.2a is enough good for me, but it lacks persistent hash.
I'd wait for persistent hash to be on Rybka 3 (Rybka is already the strongest chess engine in most hardware, so I don't see why to rush it.)
Persistent hash is a biggie for me. It's worth the wait.
For the benifit of the other poor people reading this post *cough* could you explin what persistant hash is and why it's useful.
Might change the vote stats as well.
With Persistent Hash, Rybka remembers the score attained on all the moves she analyzes. This means:
That Rybka will not play the same blunder twice.
That Rybka will be the best thing for any kind of analysis (like backwards). Even on positions that she doesn't understand, because after enough interactive analysis, she'll be able to understand them.
That all analysis is useful. Right now, since Rybka forgets her hash, analyzing a position might mean lost time the next time you look at it. But with Persistent Hash, you can analyze a position for 6 hours, switch off the computer, and then, when you switch on the computer you can continue with the analysis as if you didn't switch it off. You can keep analyzing positions a week after (Or a month after, etc.) you started analyzing them, meaning that you never lose time because Hash entries are never overwritten.
And other benefits I've forgotten. But a Rybka 3 without this would be a big let down, since I remember Vas confirming it.
That's remarkable Vytron - so the analysis is stored on a temp file on the hard disk? Isn't that slow?
I think that there would be good, efficient ways of cataloging it so that it's not all that slow. After all, once there is a "hit" in this file, no more calculating needs to be done with that line until a higher depth than what is stored, and so all of the time is spent calculating lines resulting in positions that are not in the file.
Shredder and Hiarcs have used a similar system for several Years - and it works well.
I have used the one with Hiarcs, and I avoid it when I can. I guess the file size is very small or something, because when I go back to the position later, sometimes it's there, sometimes not. Shredder is very fast and very good--as long as you leave the engine running and don't restart the computer or something, in which case it seems that everything gets erased. We need something better...
I think there are 2 things going on here one is the learn file which is written to and saved on the hard disk and retained hash tables which are lost when the machine is re-booted or the engine is reloaded. There needs to be a way of disabling this feature as in Shredder and Hiarcs so when playing matches using test suites that the engine does not remember the last time it played that test set. CEGT and CCRL I am pretty sure disable learning for their tests.
Come to think of it, since the rating lists are meant also for consumers and such, learning should be enabled in tests (and yes, it's typically disabled in tests). After all, the program they will buy will eventually acquire learning...
I think its ok to start with a new learn file at the start of each match. But say they play engine x with learning on using the same test suite v 10 other engines then a new version of engine y is released. is it fair to play engine y with no learning against engine x who has played the test suite 10x?
Well, then again, I disagree with the idea of using test suites for rating lists--the engines could have been tuned to said test suites during their making.
The only major raings list that uses opening books is SSDF - but in a new engine release these can also be tuned to beat the opponents previous book to get an artificially high rating.
CEGT definitely uses opening books--it just doesn't use the programs' OWN opening books--the same opening books are used for all programs in an engine tournament. Same with CCRL.
Yes but i think they are short generic books that play only the first few moves - so again if learning is kept between matches this could have a huge influence on results. Only SSDF uses books released with the product in their 'out of the box' testing policy. That is why a uci engine cant be tested by them with a ctg book as you do not get a CB GUI in the box. Rybka.ctg cant be used as there is no GUI so Rybka has to be tested in arena as does Hiarcs uci but the Hiarcs book is portable to any GUI.
What about if the UCI engine had a book built in that couldn't be disabled? That would give it an unfair advantage?
just delete the book! But I know of none that do this. same could be said of an engine where you cant disable learning - but again none do this. If your not sure move the own book to another folder and as soon as the learn file is created make it read only when it is empty that is what I do with shredder.
> just delete the book!
Or just delete the obnoxious engine (or make the book read-only).
People are free to use whichever books, generally speaking. I often use a book that is actually reasonably strong for playing in its own right, but that is non-engine specific, and let it go up to 12 moves deep. With shorter books, such as the 8-move books often used, there are many repeat games among the same engines. Check out the games between the quads--lots of repeat games.
yes but that is book learning not engine learnong.
Well, I was just responding to your statement about the books being so short--I wasn't talking about learning of any sort, be it book or engine. The CEGT quad games to which I refer are repeats because the book lines are the same between the same engines.
But the point i am trying to make is that if you use the same short books over and over for engine tests with position learning enabled this will lead to the engine that has played the lines most getting an advantage.
Yes, and the point that I'm trying to make is that such short general books shouldn't be used for engine testing :-).
Well it is engine testing not book testing. :)
If small books are used, then it is still book testing, but with more emphasis placed on the initial positions given by the books, which makes things worse because there is less variance in this.
I can wait this feature for the updated Rybka 3.1 but I want my Rybka 3 now plus 6 months full developpment time from Vas, same formula as for Rybka 1 & 2 series. :)
1. Will all analysis (every game, every move) be stored on hard disc ?
2. If yes, wouldn't that file be very BIG after some time ?
3. Won't it be slow to access all previous analysis from that file at every move ?
4. Will we be able to share some learning files just like opening books ? (eg. learning files for some fortresses, for some known blunders, for some known bugs, etc)
1. Yes, every analyzed move from every game would be stored (assuming that the feature is set to "ON")
2. It could potentially be a very big file after some time, but it takes longer to get large than one would think. I seem to recall from someone that continuous play using the same feature in Spike over a period of a number of weeks made the file on the order of 30 megabytes.
3. Not if the cataloging is good--after all, it's only one datum that is being accessed in a reasonable amount of pondering, even if there are many similar positions stored (in which case there are several data).
4. That has been said to be the definite plan.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill