Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Poll: Persistent hash or early Rybka 3 release?
1 2 Previous Next  
Poll Would you rather have Rybka 3 ASAP or wait for persistent hash to be effectively implemented? (Closed)
Persistent hash can wait--give me Rybka 3 ASAP! 10 24%
I can wait for persistent hash 31 76%
- - By turbojuice1122 (Gold) Date 2008-01-02 22:08
Given the choice, would you rather have Rybka 3 released soon without persistent hash, or do you see it as worth the wait to have a reliable persistent hash put in?
Parent - - By turbojuice1122 (Gold) Date 2008-01-02 22:10
As for me, I'd rather wait for an effective implementation of persistent hash, especially since I like to analyze openings and early middlegame positions, and this would be an incredibly valuable analysis tool for such things.  I would guess that it would also be very helpful for correspondence players.

Of course, it would be natural to hear what Vas has to say about this, i.e. how much extra time it would take for an effective implementation of this.  It might not take all that long...
Parent - - By Vasik Rajlich (Silver) Date 2008-01-03 15:47
We're definitely planning this. It's not a huge task, maybe a week (including all tinkering, etc).

Vas
Parent - - By herakleitos (***) Date 2008-01-03 16:09
If a week is all it takes, then by all means let's wait the extra week. Those of us who use Rybka for deep analysis and annotation will find this a godsend.

Like voting for Dan Quayle, it's a no-brainer ...
Parent - - By Carl Bicknell (*****) Date 2008-01-03 17:06
Just out of curiousity what difference would it make in terms of elo strength in actual play, roughly?
Parent - - By Vempele (Silver) Date 2008-01-03 17:40
On the rating lists, 0. In a thematic match, up to several hundred points (in the course of LOTS of games). On Playchess? Depends on your book.
Parent - By Carl Bicknell (*****) Date 2008-01-03 20:22
thanks!
Parent - - By Uly (Gold) Date 2008-01-03 20:52

> On the rating lists, 0.


Yes, but this is a flaw on the rating lists themselves. For example, Pro Deo already comes with learning from +6000 games, and Ed said that this amounts to about 30 ELO points. Also, you can switch this learning on while avoiding new learning. But the rating lists just disable learning even though any user downloading Pro Deo fresh will have a stronger version than the one on the rating lists.

I think that there should be a different rating list, that includes engines with learning ON.
Parent - - By Vempele (Silver) Date 2008-01-04 18:39

> For example, Pro Deo already comes with learning from +6000 games, and Ed said that this amounts to about 30 ELO points.


1. That's far too few games to get _anywhere_ near 30 Elo. Ever. The testing groups use too many different books for that to work. Unless it's eval weight learning (though it'd still be close to unbelievable).
2. If it's worth 30 points, why not just build it into the executable?
3. I tried and failed to find your source. Mind posting a link?
Parent - By Uly (Gold) Date 2008-01-04 20:42
Ed claimed that the 6700 games are only Pro Deo games, and top quality. It was actually better than trying 3'000'000 games, if Pro Deo didn't play on them.

> Unless it's eval weight learning (though it'd still be close to unbelievable).


Yes, eval weight based on the results of games. So, Pro Deo will avoid moves that lost in the past and will keep playing winning moves.

> 2. If it's worth 30 points, why not just build it into the executable?


For simplicity. Also, the learner is in the executable, but the eoc files are of course outside, so you can change them and make yours.

> 3. I tried and failed to find your source. Mind posting a link?


Ok, I apologize since I should have said "20-30" ELO, but here's the link:

http://www.top-5000.nl/prodeo.htm

"Since the EOC approach is an open source engine programmers can easily add the algorithm to their own engine and add 20-30 elo points without too much of a hassle."
Parent - - By turbojuice1122 (Gold) Date 2008-01-07 00:51
Another question about the persistent hash: will it understand such things as reversed positions?  For example, if we let it sit for a few years on Suj's machine looking at the position after 1.e4 e5 and concludes that white has mate in 46 moves, will it come back with the correct evaluation from the persistent hash if we put in the position 1.e3 e5 2.e4 as black having a forced mate in 46 moves?
Parent - - By Uly (Gold) Date 2008-01-07 22:05
And don't forget mirror positions, but these only appear after both sides have lost castling rights.
Parent - - By Vasik Rajlich (Silver) Date 2008-01-08 14:24
I don't see any way to handle this, at least without a massive loss in efficiency.

Vas
Parent - By Uly (Gold) Date 2008-01-08 21:47
After castling rights disappear, the same position has 4 versions:

r6N/ppp1k2p/2n1bn2/6Bq/8/3P1NpP/PPP2bK1/R2Q3R w - - 0 19


Real position.

r3q2r/1kB2ppp/pPn1p3/8/Qb6/2NB1N2/P2K1PPP/n6R b - - 0 1


Reversed colors.

N6r/p2k1ppp/2nb1n2/qB6/8/PpN1P3/1Kb2PPP/R3Q2R w - - 0 1


Mirror position.

r2q3r/ppp2Bk1/3p1nPp/8/6bQ/2N1BN2/PPP1K2P/R6n b - - 0 1


Mirror position with reversed colors.

So, when you analyze a position, you're actually analyzing 4 of them. The real question is how likely are these 3 later positions to appear? If we're never going to see them on the board, having their analysis is just wasting space.
Parent - - By Vempele (Silver) Date 2008-01-09 06:32
HashKey[Piece][Square] = ~HashKey[Piece][Mirror(Square)];
Parent - By Vasik Rajlich (Silver) Date 2008-01-10 22:47
I guess you're right, this should work.

For black/white mirroring, it's actually something like:

HashKey [Color_Mirror (Piece)] [Mirror (Square)];

Black/white mirroring seems worthwhile, you just keep two hash keys as you search. Left-right mirroring would only apply to endgames and probably isn't worthwhile.

Anyway, if there are no gotchas, we'll probably do this.

Vas
Parent - - By Mark (****) Date 2008-01-02 22:13
Let's get 3.0 out soon!  Save the persistent hash for release 3.1  (unfortunately, I don't think this poll will influence the actual release date...)
Parent - - By turbojuice1122 (Gold) Date 2008-01-02 22:22
That's a good point about 3.1.  However, there is also the question of how much has Rybka 2.3.x actually improved?  I know that it's something like 50 elo points in the rapid tests at 2000 elo strength, but just looking at the few games played in Paderborn, I don't get the impression that its objective strength has really improved significantly--it seems like the improvement is not more than of what I had the impression with 2.3.2a compared with 2.3.1, and that improvement has turned out to be about 15 elo (when it was actually stated that the improvement in tests was something like 50 elo).  Thus, there might not really be much of a reason for us to want the newest version so soon.
Parent - By Uly (Gold) Date 2008-01-02 22:25

> and that improvement has turned out to be about 15 elo (when it was actually stated that the improvement in tests was something like 50 elo).


I think it's because the improvement of Rybka is about 50 ELO under laboratory conditions.

However, Rybka will have to go and play outside the laboratory, in where the improvement isn't as much.
Parent - - By Mark (****) Date 2008-01-02 22:27
I really don't know why I'm in a hurry.  I couldn't even tell the difference between two programs that are only 100 elo apart anyway.  :-)
Parent - By Uly (Gold) Date 2008-01-02 22:53

> I couldn't even tell the difference between two programs that are only 100 elo apart anyway.


I couldn't tell the difference from players that are +750 ratings point apart, if they played individually against weaker opponents with the same rating difference...
Parent - - By Uly (Gold) Date 2008-01-02 22:17
Rybka 2.3.2a is enough good for me, but it lacks persistent hash.

I'd wait for persistent hash to be on Rybka 3 (Rybka is already the strongest chess engine in most hardware, so I don't see why to rush it.)
Parent - - By billyraybar (***) Date 2008-01-02 22:24
Persistent hash is a biggie for me.  It's worth the wait.
Parent - - By Carl Bicknell (*****) Date 2008-01-02 22:58
For the benifit of the other poor people reading this post *cough* could you explin what persistant hash is and why it's useful.

Might change the vote stats as well.
Parent - - By Uly (Gold) Date 2008-01-02 23:32
With Persistent Hash, Rybka remembers the score attained on all the moves she analyzes. This means:

That Rybka will not play the same blunder twice.

That Rybka will be the best thing for any kind of analysis (like backwards). Even on positions that she doesn't understand, because after enough interactive analysis, she'll be able to understand them.

That all analysis is useful. Right now, since Rybka forgets her hash, analyzing a position might mean lost time the next time you look at it. But with Persistent Hash, you can analyze a position for 6 hours, switch off the computer, and then, when you switch on the computer you can continue with the analysis as if you didn't switch it off. You can keep analyzing positions a week after (Or a month after, etc.) you started analyzing them, meaning that you never lose time because Hash entries are never overwritten.

And other benefits I've forgotten. But a Rybka 3 without this would be a big let down, since I remember Vas confirming it.
Parent - - By Carl Bicknell (*****) Date 2008-01-03 09:24
That's remarkable Vytron - so the analysis is stored on a temp file on the hard disk? Isn't that slow?
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 13:40
I think that there would be good, efficient ways of cataloging it so that it's not all that slow.  After all, once there is a "hit" in this file, no more calculating needs to be done with that line until a higher depth than what is stored, and so all of the time is spent calculating lines resulting in positions that are not in the file.
Parent - - By Harvey Williamson (*****) Date 2008-01-03 13:41
Shredder and Hiarcs have used a similar system for several Years - and it works well.
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 13:45
I have used the one with Hiarcs, and I avoid it when I can.  I guess the file size is very small or something, because when I go back to the position later, sometimes it's there, sometimes not.  Shredder is very fast and very good--as long as you leave the engine running and don't restart the computer or something, in which case it seems that everything gets erased.  We need something better...
Parent - - By Harvey Williamson (*****) Date 2008-01-03 13:49
I think there are 2 things going on here one is the learn file which is written to and saved on the hard disk and retained hash tables which are lost when the machine is re-booted or the engine is reloaded. There needs to be a way of disabling this feature as in Shredder and Hiarcs so when playing matches using test suites that the engine does not remember the last time it played that test set. CEGT and CCRL I am pretty sure disable learning for their tests.
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 13:52
Come to think of it, since the rating lists are meant also for consumers and such, learning should be enabled in tests (and yes, it's typically disabled in tests).  After all, the program they will buy will eventually acquire learning...
Parent - - By Harvey Williamson (*****) Date 2008-01-03 13:55
I think its ok to start with a new learn file at the start of each match. But say they play engine x with learning on using the same test suite v 10 other engines then a new version of engine y is released. is it fair to play engine y with no learning against engine x who has played the test suite 10x?
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 16:26
Well, then again, I disagree with the idea of using test suites for rating lists--the engines could have been tuned to said test suites during their making.
Parent - - By Harvey Williamson (*****) Date 2008-01-03 16:28
The only major raings list that uses opening books is SSDF - but in a new engine release these can also be tuned to beat the opponents previous book to get an artificially high rating.
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 16:32
CEGT definitely uses opening books--it just doesn't use the programs' OWN opening books--the same opening books are used for all programs in an engine tournament.  Same with CCRL.
Parent - - By Harvey Williamson (*****) Date 2008-01-03 17:05
Yes but i think they are short generic books that play only the first few moves - so again if learning is kept between matches this could have a huge influence on results. Only SSDF uses books released with the product in their 'out of the box' testing policy. That is why a uci engine cant be tested by them with a ctg book as you do not get a CB GUI in the box. Rybka.ctg cant be used as there is no GUI so Rybka has to be tested in arena as does Hiarcs uci but the Hiarcs book is portable to any GUI.
Parent - - By Carl Bicknell (*****) Date 2008-01-03 17:09
What about if the UCI engine had a book built in that couldn't be disabled? That would give it an unfair advantage?
Parent - - By Harvey Williamson (*****) Date 2008-01-03 17:09 Edited 2008-01-03 17:12
just delete the book! But I know of none that do this. same could be said of an engine where you cant disable learning - but again none do this. If your not sure move the own book to another folder and as soon as the learn file is created make it read only when it is empty that is what I do with shredder.
Parent - - By Vempele (Silver) Date 2008-01-03 17:42

> just delete the book!


Or just delete the obnoxious engine (or make the book read-only).
Parent - By Harvey Williamson (*****) Date 2008-01-03 17:44
yep!
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 18:55
People are free to use whichever books, generally speaking.  I often use a book that is actually reasonably strong for playing in its own right, but that is non-engine specific, and let it go up to 12 moves deep.  With shorter books, such as the 8-move books often used, there are many repeat games among the same engines.  Check out the games between the quads--lots of repeat games.
Parent - - By Harvey Williamson (*****) Date 2008-01-03 19:07
yes but that is book learning not engine learnong.
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 19:22
Well, I was just responding to your statement about the books being so short--I wasn't talking about learning of any sort, be it book or engine.  The CEGT quad games to which I refer are repeats because the book lines are the same between the same engines.
Parent - - By Harvey Williamson (*****) Date 2008-01-03 19:25
But the point i am trying to make is that if you use the same short books over and over for engine tests with position learning enabled this will lead to the engine that has played the lines most getting an advantage.
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 19:27
Yes, and the point that I'm trying to make is that such short general books shouldn't be used for engine testing :-).
Parent - - By Harvey Williamson (*****) Date 2008-01-03 19:33
Well it is engine testing not book testing. :)
Parent - By turbojuice1122 (Gold) Date 2008-01-03 19:40
If small books are used, then it is still book testing, but with more emphasis placed on the initial positions given by the books, which makes things worse because there is less variance in this.
Parent - - By Bouddha (****) Date 2008-01-03 00:02
I can wait this feature for the updated Rybka 3.1 but I want my Rybka 3 now plus 6 months full developpment time from Vas, same formula as for Rybka 1 & 2 series.  :)
Parent - - By onursurme (***) Date 2008-01-03 11:20
1. Will all analysis (every game, every move) be stored on hard disc ?
2. If yes, wouldn't that file be very BIG after some time ?
3. Won't it be slow to access all previous analysis from that file at every move ?
4. Will we be able to share some learning files just like opening books ? (eg. learning files for some fortresses, for some known blunders, for some known bugs, etc)
Parent - - By turbojuice1122 (Gold) Date 2008-01-03 13:44
1.  Yes, every analyzed move from every game would be stored (assuming that the feature is set to "ON")
2.  It could potentially be a very big file after some time, but it takes longer to get large than one would think.  I seem to recall from someone that continuous play using the same feature in Spike over a period of a number of weeks made the file on the order of 30 megabytes.
3.  Not if the cataloging is good--after all, it's only one datum that is being accessed in a reasonable amount of pondering, even if there are many similar positions stored (in which case there are several data).
4.  That has been said to be the definite plan.
Up Topic Rybka Support & Discussion / Rybka Discussion / Poll: Persistent hash or early Rybka 3 release?
1 2 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill