Shredder 12 UCI GUI
Houdini 2.0c UCI engine
Rybka 4.1 UCI engine
It takes some extra work to save/load hash files when playing correspondence chess with a couple of different engines. It can be done but it's tedious. So there must be a significant advantage gained for it to be worth the trouble.
Houdini works pretty good to save/load hash from a file named "hash.dat" and resume analysis. It doesn't retain everything but usually gives significant advantage. It doesn't matter whether the game is resumed exactly where it was stopped. It can be restarted a move further on, after opponent's actual move is entered. It also works OK with multi-PV mode. Of course it works best when opponent's reply was correctly predicted, but usually remembers something advantageous in any case. It can save several hours of analysis time versus restarting the analysis from the beginning. So it's worth the extra work required, in my experience.
However Rybka doesn't work right... Yes, I have the Preserve Analysis checkbox selected. I save/load the hash to a file named "hash.dat". But when Rybka resumes analysis with a loaded hash file, it doesn't remember anything and starts the analysis from the beginning. Is there a trick? After searching here I see mention of a .rsh file for use with Rybka. Is that extension required?
I also use a learn.dat file with Houdini but don't understand the distinction between this type of knowledge and saving/loading hash. How can I tell where the knowledge comes from when I resume an analysis - the learn.dat file or the hash.dat file?
Houdini 2.0c UCI engine
Rybka 4.1 UCI engine
It takes some extra work to save/load hash files when playing correspondence chess with a couple of different engines. It can be done but it's tedious. So there must be a significant advantage gained for it to be worth the trouble.
Houdini works pretty good to save/load hash from a file named "hash.dat" and resume analysis. It doesn't retain everything but usually gives significant advantage. It doesn't matter whether the game is resumed exactly where it was stopped. It can be restarted a move further on, after opponent's actual move is entered. It also works OK with multi-PV mode. Of course it works best when opponent's reply was correctly predicted, but usually remembers something advantageous in any case. It can save several hours of analysis time versus restarting the analysis from the beginning. So it's worth the extra work required, in my experience.
However Rybka doesn't work right... Yes, I have the Preserve Analysis checkbox selected. I save/load the hash to a file named "hash.dat". But when Rybka resumes analysis with a loaded hash file, it doesn't remember anything and starts the analysis from the beginning. Is there a trick? After searching here I see mention of a .rsh file for use with Rybka. Is that extension required?
I also use a learn.dat file with Houdini but don't understand the distinction between this type of knowledge and saving/loading hash. How can I tell where the knowledge comes from when I resume an analysis - the learn.dat file or the hash.dat file?
![[mx]](/mwf/v12/flags/mx.png "Mexico")
There are two different kinds of learning that is happening here. The first one, Position Learning, works by saving important data to harddisk so that the engine remembers important lines of analysis, and doesn't need to do researching, this is very useful and effective for Interactive Analysis of positions.
The second one is just saving the entire hash to disk, and only has the same effect of never unloading the engine (the hash is preserved as if the engine wasn't unloaded), but otherwise has no benefit.
Shredder 12, Houdini 2.0, Rybka 3 and Critter 1.4 support Position Learning, so it's recommended that for these engines, you use this feature instead:
Under Shredder, make sure "Position Learning" is ticked.
With Houdini, make sure "Learning" is enabled and properly configured.
With Rybka 3, it's Persistent Hash=ON, and you have to specify settings like the filename and size.
With Critter 1.4, it's "Session File", and and Defaults should work fine after enabling it.
Houdini 2 and Rybka 4.1 support Save/Load hash, which is a feature only recommended for long analysis sessions in where you sit on the same position for several hours, and want to interrupt analysis and have the engine continue after interruption, to see it in action you also have to tick the engine option that protects hash contents so they don't disappear. This is called "Preserve Analysis" in Rybka 4, if you don't tick this box, then the saved hash contents will be overwritten and you wouldn't see what you expect to see. Houdini should have a similarly named option to keep important info on hash (I don't have Houdini.)
The name of the file doesn't matter, .rsh is just how you could name the file (and the extension that would be used to avoid confusion in case users want to exchange hashes) but it can have any name and the engine will use it.
After using Save/Load Hash of Rybka 4, even after setting everything right, I have to say I never saw any significant advantage, not even a small advantage, so I stopped using it. It's much more effective to have an engine around with Position Learning and to feed it the important variations that other engines suggest, unless it's a weak engine such as Shredder.
The second one is just saving the entire hash to disk, and only has the same effect of never unloading the engine (the hash is preserved as if the engine wasn't unloaded), but otherwise has no benefit.
Shredder 12, Houdini 2.0, Rybka 3 and Critter 1.4 support Position Learning, so it's recommended that for these engines, you use this feature instead:
Under Shredder, make sure "Position Learning" is ticked.
With Houdini, make sure "Learning" is enabled and properly configured.
With Rybka 3, it's Persistent Hash=ON, and you have to specify settings like the filename and size.
With Critter 1.4, it's "Session File", and and Defaults should work fine after enabling it.
Houdini 2 and Rybka 4.1 support Save/Load hash, which is a feature only recommended for long analysis sessions in where you sit on the same position for several hours, and want to interrupt analysis and have the engine continue after interruption, to see it in action you also have to tick the engine option that protects hash contents so they don't disappear. This is called "Preserve Analysis" in Rybka 4, if you don't tick this box, then the saved hash contents will be overwritten and you wouldn't see what you expect to see. Houdini should have a similarly named option to keep important info on hash (I don't have Houdini.)
The name of the file doesn't matter, .rsh is just how you could name the file (and the extension that would be used to avoid confusion in case users want to exchange hashes) but it can have any name and the engine will use it.
>So there must be a significant advantage gained for it to be worth the trouble.
After using Save/Load Hash of Rybka 4, even after setting everything right, I have to say I never saw any significant advantage, not even a small advantage, so I stopped using it. It's much more effective to have an engine around with Position Learning and to feed it the important variations that other engines suggest, unless it's a weak engine such as Shredder.
Thanks for the helpful info and advice.
I'm continuing to save hash with Houdini 2.0c effectively. It doesn't always restart the analysis at exactly the same ply level, and may back off by 1-2-3 ply. So if I save hash after 30 ply had been completed, typically it might restart analysis at the 27-28-29 ply level instead. I haven't figured out why this difference, except my impression is that it could depend on the special characteristics of the position. Some positions save to hash better than others. Occasionally it backs off a lot... So if I saved hash at ply level 30, it might only restart at ply level 20, and forgets the other info for unknown reasons. I recently noticed this happening when an en passant pawn capture was involved. However overall it is a big benefit when doing long infinite analysis (several hours or days) with Houdini 2.0c that's interrupted/restarted such as in correspondence chess play.
I've continued to test save hash with Rybka 4.1 but apparently it just doesn't work - or I'm missing some vital operating instructions/details. You mention it should work, so I wonder - what's the trick? I'm fairly sure I have all the engine settings correct. The "Preserve Analysis" box is checked. Could it be the GUI? I'm using Deep Shredder 12 UCI GUI.
I'm also intrigued by your suggestion: "It's much more effective to have an engine around with Position Learning and to feed it the important variations that other engines suggest..." Houdini 2.0c has this feature, which is automatic after the appropriate checkbox and filename are entered. But I'm curious how I would go about feeding it important variations that other engines suggest(?). I'm not aware of any way to enter position learning directly into the learn.dat file. Insofar as I'm aware, it only gets learning from having analyzed positions in the past.
It might be an advantage if it were possible to build in all of the reliable chess opening lines into the learn.dat file.
It would also be useful if there were some way to combine the learn.dat files from different computers. Like most serious correspondence chess players, I have several computers running simultaneously on my home network, usually assigning particular games to a particular computer - but occasionally trading off. If I could somehow use a common network learn.dat file, or merge learn.dat files from the different computers, that should offer an advantage.
It's not a quantum leap from that type of thinking to the idea of creating an ever growing, ever improving centralized learn.dat file which could be accessed and/or added-to over the internet. In other words, why analyze a position if it has already been analyzed and stored in a central learn.dat file? Just look it up, like any table of data, over the internet. Sounds possible, but is it practical(?)...
I'm continuing to save hash with Houdini 2.0c effectively. It doesn't always restart the analysis at exactly the same ply level, and may back off by 1-2-3 ply. So if I save hash after 30 ply had been completed, typically it might restart analysis at the 27-28-29 ply level instead. I haven't figured out why this difference, except my impression is that it could depend on the special characteristics of the position. Some positions save to hash better than others. Occasionally it backs off a lot... So if I saved hash at ply level 30, it might only restart at ply level 20, and forgets the other info for unknown reasons. I recently noticed this happening when an en passant pawn capture was involved. However overall it is a big benefit when doing long infinite analysis (several hours or days) with Houdini 2.0c that's interrupted/restarted such as in correspondence chess play.
I've continued to test save hash with Rybka 4.1 but apparently it just doesn't work - or I'm missing some vital operating instructions/details. You mention it should work, so I wonder - what's the trick? I'm fairly sure I have all the engine settings correct. The "Preserve Analysis" box is checked. Could it be the GUI? I'm using Deep Shredder 12 UCI GUI.
I'm also intrigued by your suggestion: "It's much more effective to have an engine around with Position Learning and to feed it the important variations that other engines suggest..." Houdini 2.0c has this feature, which is automatic after the appropriate checkbox and filename are entered. But I'm curious how I would go about feeding it important variations that other engines suggest(?). I'm not aware of any way to enter position learning directly into the learn.dat file. Insofar as I'm aware, it only gets learning from having analyzed positions in the past.
It might be an advantage if it were possible to build in all of the reliable chess opening lines into the learn.dat file.
It would also be useful if there were some way to combine the learn.dat files from different computers. Like most serious correspondence chess players, I have several computers running simultaneously on my home network, usually assigning particular games to a particular computer - but occasionally trading off. If I could somehow use a common network learn.dat file, or merge learn.dat files from the different computers, that should offer an advantage.
It's not a quantum leap from that type of thinking to the idea of creating an ever growing, ever improving centralized learn.dat file which could be accessed and/or added-to over the internet. In other words, why analyze a position if it has already been analyzed and stored in a central learn.dat file? Just look it up, like any table of data, over the internet. Sounds possible, but is it practical(?)...
> I've continued to test save hash with Rybka 4.1 but apparently it just doesn't work - or I'm missing some vital operating instructions/details. You mention it should work, so I wonder - what's the trick? I'm fairly sure I have all the engine settings correct. The "Preserve Analysis" box is checked.
Try setting the engine to 1CPU, and fixed depth, so results are reproducible, then see if there's a difference in output after loading hash. If there's a difference, then the hash loading is working, it's just that it's not very effective at all.
>But I'm curious how I would go about feeding it important variations that other engines suggest(?).
You just play the moves into the GUI, and let Houdini analyze them. If it works correctly and the variation is better that what Houdini liked originally, you may be saving hours of work.
Why would you allow the engine to reach depth 30? Because chances are there's a change in move choice at high depth. It looks like:
Depth 25 d4
Depth 26 d4
Depth 27 e4
But reaching depth 27 takes too long. More effective is to fire up a second engine, which likes e4 at early depth, and then play e4 on the board and show it to Houdini. Houdini will show a score better for e4 than d4 much faster than it would have taken it to reach depth 27, so you save time. And the saved time goes up exponentially as you repeat this in future positions, and keep track of the scores of the moves, helping the engine to find the best moves without having to wait hours for it to reach high depth. Strong OTB player may not need a second engine to point out the move and they'll force it and the continuations themselves.
I don't let my Rybka reach depth 19, as the next depth is a waste of time. This is the power of interactivity, but it only works if you are willing to do the job (or you can, i.e. if you are going to be away from the computer for 8 hours then you can't interact at all, though it still could be argued that leaving automated interaction methods like IDEA or DPA would be a better usage of time since when you come back you have more than just a PV line), otherwise forget all this.
>It might be an advantage if it were possible to build in all of the reliable chess opening lines into the learn.dat file.
I guess you'd need Aquarium for this. You get a tree with all the positions you want to have in the dat file, you set learning on with Houdini, and then you tell the GUI to analyze all the positions. After it's done you'll have your dat file will all the positions in it.
>It would also be useful if there were some way to combine the learn.dat files from different computers.
That's a feature you'd need to request to Houdini's author on the forum. His nick is Stonehenge.
> Sounds possible, but is it practical(?)...
I think Fritz already has such a feature called "Let's check", that not only stores analysis of Fritz, but stores analysis of any engine the user has on the position. So it's practical and the problem is that people wouldn't be willing to share their analysis with their competition of correspondence chess (I'm one of them.)
It would indeed be practical to have a "floating" learning file on a server or shared among users via a tracker like in uTorrentor or emule (instead of requesting a file, you require a FEN score). A problem here would be that a lot of the analysis would become obsolete once the next engine version comes around or when it's significantly stronger than the version that stored the analysis originally (e.g. if this was implemented for Rybka 2.3.2a, the floating learning file would be mostly garbage once Rybka 3 appeared and showed how relatively poor was 2.3.2a's analysis.)
Thanks for your helpful discussion.
I finally got Rybka to load hash and resume analysis at the same point (30 ply in my test). The trick is to clear hash first - then load the saved hash file and resume analysis. I don't know if this is a peculiarity of the GUI (Deep Shredder 12 UCI) or the engine (Deep Rybka 4.1), but I suspect the engine. There is no similar problem with the Houdini 2.0c engine in the same GUI.
I finally got Rybka to load hash and resume analysis at the same point (30 ply in my test). The trick is to clear hash first - then load the saved hash file and resume analysis. I don't know if this is a peculiarity of the GUI (Deep Shredder 12 UCI) or the engine (Deep Rybka 4.1), but I suspect the engine. There is no similar problem with the Houdini 2.0c engine in the same GUI.
Good job!
I'll report this bug so hopefully Rybka 5 doesn't have this problem.
I'll report this bug so hopefully Rybka 5 doesn't have this problem.
Thanks. I hope Vas is making new code successfully.
FYI, it worked that one time, but I can't get it to work again. I haven't the slightest idea why. I've quit trying.
I got it to work again with Rybka. It's necessary to do the following two things:
(1) When saving hash, do it while the engine is running. Don't stop engine first.
(2) Before loading the saved hash, you must first clear hash.
I'm assuming that you're starting a new session and maybe have turned off computer, or other games have intervened. You may be able to continue the analysis without loss of hash if it was simply interrupted somehow.
Houdini is not so temperamental. Almost any reasonable way works.
This is with the Deep Shredder 12 UCI GUI. Possibly it works differently with other GUIs(?), but I haven't tried any others.
(1) When saving hash, do it while the engine is running. Don't stop engine first.
(2) Before loading the saved hash, you must first clear hash.
I'm assuming that you're starting a new session and maybe have turned off computer, or other games have intervened. You may be able to continue the analysis without loss of hash if it was simply interrupted somehow.
Houdini is not so temperamental. Almost any reasonable way works.
This is with the Deep Shredder 12 UCI GUI. Possibly it works differently with other GUIs(?), but I haven't tried any others.
Reported.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill