![[mx]](/mwf/v12/flags/mx.png "Mexico")
Don't merge Persistent Hash unless you trust it :)
Messing a Persistent Hash is a straightforward process, just use move ignoring for a long time and Rybka will learn the wrong move as best.
I created this Persistent Hash to show an example. Put it on your engine folder and tell Rybka to use Damaged.rph, and set the size to 16.
Now, put her to analyze the starting position and she'll think 1.f3 is the best move up to ply 19. That's as high as I can get in my ancient hardware but someone malicious with time can really mess up a persistent hash so that you'd need a really high depth to fix it.
The catch is that there's no way to distinguish a legitimate persistent hash from a messed one like this, I could have made a messed one that supposedly refutes a variation of the Najdorf when I all did was setting Rybka to ignore all the other moves.
The file is just 21KB zipped, it will grow to 16MB when unzipped.
Messing a Persistent Hash is a straightforward process, just use move ignoring for a long time and Rybka will learn the wrong move as best.
I created this Persistent Hash to show an example. Put it on your engine folder and tell Rybka to use Damaged.rph, and set the size to 16.
Now, put her to analyze the starting position and she'll think 1.f3 is the best move up to ply 19. That's as high as I can get in my ancient hardware but someone malicious with time can really mess up a persistent hash so that you'd need a really high depth to fix it.
The catch is that there's no way to distinguish a legitimate persistent hash from a messed one like this, I could have made a messed one that supposedly refutes a variation of the Najdorf when I all did was setting Rybka to ignore all the other moves.
The file is just 21KB zipped, it will grow to 16MB when unzipped.
Attachment: Damaged.zip (20k)
![[de]](/mwf/v12/flags/de.png "Germany")
I guess this very much kills the idea of a shared hash database - something which I already anticipated to happen. But I didn't think it would be possible to mislead her so dramatically. As well this renders persistent hash being completely useless at the moment, because even in analysis it would be not advised. I guess Vas would need to rethink the whole concept...
![[fi]](/mwf/v12/flags/fi.png "Finland")
> I guess Vas would need to rethink the whole concept...
What? It's a simple bugfix: just don't hash the root position if you're told not to analyze some of the moves.
The good news is, the bug is much worse than all of the others combined. We will definitely see a Rybka 3.0.1 (or whatever) now. :)
Okay, if it would work this way. Anyway, I'm going to discard my hash files so far, since they are probably, well, not maliciously messed up, but not "neutral" anymore, either...
Vas is going to render this bug as minor with the argument: "Just don't use Persistent Hash when you're ignoring moves" :)
:(
:(
>> I guess Vas would need to rethink the whole concept...
> What? It's a simple bugfix: just don't hash the root position if you're told not to analyze some of the moves.
>
> The good news is, the bug is much worse than all of the others combined. We will definitely see a Rybka 3.0.1 (or whatever) now. :-)
Hi guys,
Please correct me if I am wrong but; why would this be a bug? Is this not inevitable anytime if you exclude some moves, so even pruning, if Rybka actually does "hard" pruning anywhere, - In my recollection Vas has stated in the past that Rybka does not really prune moves "forever" -, would then have to be considered a bug?
Just suppose in the starting position you move 1. e4 h6 and analyze this position for a week, likely it will still be very close to a draw (Ignoring contempt effects, I don't know about that). Would not this very deep analysis of 1... h6 suggest to Rybka that it is the best reply to 1. e4, if the value is 0.03 or something for White? In this case I don't think you actually need to ingnore any moves and it is just the natural result of doing a deep analysis and remembering the result. Possibly this kind of effect could also contribute to making tablebases so very ineffective in raising elo but that is just a theory ;)
Regards, Eelco
> Please correct me if I am wrong but; why would this be a bug?
Because you get utterly incorrect data! Hard pruning the best move (as IDeA forces Rybka to do) is the surest way to completely wreck the analysis (worse best move + worse score). When the result is later fetched from PH, shit happens.
You got PM.
If you let 1...c5 to be analyzed for a week it will get better evaluation than h6. If you try this experiment yourself, you will notice that Rybka will notice this and once you analyze the previous position Rybka'll be aware that h6 isn't best before reaching the ply that h6 reached. But if you use this bug, you'd need to analyze the position for more than one week for Rybka to analyze this.
I'll put a more clear example:
Ok, now make Rybka analyze this position but ignore everything but 3.Qxf7 for a week. After that, Rybka will learn that Qxf7 is best, and will need to reach the next ply to find an alternative. What is really bad about this is that if the person that is messing the hash does the same for Qf3, Rybka will learn that it is the only move, and this position will get blocked from the search because it got a really bad score, so when analyzing a previous position Rybka will think that e4 loses, because after e5! Qf3(only move) Nc6! Qxf7(only move) White is losing.
This is just an example of what can go wrong, and it's bad because with Persistent Hash you need Depth+1 to fix it.
I'll put a more clear example:
r1bqkbnr/pppp1ppp/2n5/4p3/4P3/5Q2/PPPP1PPP/RNB1KBNR w KQkq - 0 3
Ok, now make Rybka analyze this position but ignore everything but 3.Qxf7 for a week. After that, Rybka will learn that Qxf7 is best, and will need to reach the next ply to find an alternative. What is really bad about this is that if the person that is messing the hash does the same for Qf3, Rybka will learn that it is the only move, and this position will get blocked from the search because it got a really bad score, so when analyzing a previous position Rybka will think that e4 loses, because after e5! Qf3(only move) Nc6! Qxf7(only move) White is losing.
This is just an example of what can go wrong, and it's bad because with Persistent Hash you need Depth+1 to fix it.
Hi Vytron,
Thanks for your reply, I think I follow the explanation a bit! The solution for any analysis with Exclude moves I think should indeed be the simple thing Vempele mentioned in his post above:
What? It's a simple bugfix: just don't hash the root position if you're told not to analyze some of the moves.
The good news is, the bug is much worse than all of the others combined. We will definitely see a Rybka 3.0.1 (or whatever) now.
This is not new I think, for instance Rebel relies heavily on analysis where the first best move can be ignored so the other moves can get exact scores. Rebel never had multi-PV though! Users would have liked that feature and asked for it in countless e-mails I reckon but Ed Schröder always said it was too slow! Sorry Ed I am exaggerating terribly and sometimes a programmer has to stand firm! But for learning and EOC, Rebel programmer Ed would have had to come up with a solution what to do with the excluded moves too!
I am not quite sure the problem with deep searches like after 1.e4 h6 is totally avoidable, I don't have Rybka 3 so I can't check it, the process you describe Vytron seems plausible. I think when analyzing the starting position, the position before 1. e4, Rybka will do some interpolation between a shallow result it gets returned from the search for the position after 1.e4 h6 and the stored deep result for 1. e4 h6. Either that or Rybka would only use the Permanent Hash stored reply-move or the stored PV if available, but not the evaluation for 1.. h6, until the other moves in the starting position reached the same depth. I think that last solution would be mathematically correct/safe, but the interpolation would only ever be an approximation although it would be really tempting to try to get a more precise score that way! But it all depends on Rybka's search too which I do not really know of course! We will have to hear what Vas thinks about the bug when he is finished unpacking and sets up the Pringles can antenna again for contacting the forum :)
Regards, Eelco
Thanks for your reply, I think I follow the explanation a bit! The solution for any analysis with Exclude moves I think should indeed be the simple thing Vempele mentioned in his post above:
What? It's a simple bugfix: just don't hash the root position if you're told not to analyze some of the moves.
The good news is, the bug is much worse than all of the others combined. We will definitely see a Rybka 3.0.1 (or whatever) now.
This is not new I think, for instance Rebel relies heavily on analysis where the first best move can be ignored so the other moves can get exact scores. Rebel never had multi-PV though! Users would have liked that feature and asked for it in countless e-mails I reckon but Ed Schröder always said it was too slow! Sorry Ed I am exaggerating terribly and sometimes a programmer has to stand firm! But for learning and EOC, Rebel programmer Ed would have had to come up with a solution what to do with the excluded moves too!
I am not quite sure the problem with deep searches like after 1.e4 h6 is totally avoidable, I don't have Rybka 3 so I can't check it, the process you describe Vytron seems plausible. I think when analyzing the starting position, the position before 1. e4, Rybka will do some interpolation between a shallow result it gets returned from the search for the position after 1.e4 h6 and the stored deep result for 1. e4 h6. Either that or Rybka would only use the Permanent Hash stored reply-move or the stored PV if available, but not the evaluation for 1.. h6, until the other moves in the starting position reached the same depth. I think that last solution would be mathematically correct/safe, but the interpolation would only ever be an approximation although it would be really tempting to try to get a more precise score that way! But it all depends on Rybka's search too which I do not really know of course! We will have to hear what Vas thinks about the bug when he is finished unpacking and sets up the Pringles can antenna again for contacting the forum :)
Regards, Eelco
Rybka would store h6 as best because it assumes the other moves were searched to the same depth, and h6 was best, when in fact they were excluded.
> Rybka would store h6 as best because it assumes the other moves were searched to the same depth, and h6 was best, when in fact they were excluded.
Hi Vytron,
I meant the possible problem that you have just storing a week long analysis of the position after 1. e4 h6. That should give a PH result still close to 0.00 + contempt factor. Not using Ignore moves or Exclude moves in the process. Because the evaluation for White should drop with increasing depth, closer to a drawscore with every iteration, you can not directly use this result anymore (when in a search of the starting position, that is one ply earlier) to compare the score with other responses to 1. e4 that have only shallower searches. Some correction for the drop (internally by Rybka in the search when using PH results) has to be applied, or alternatively only the stored moves have to be used. That was the potential search problem I was referring to in my first post.
Regards, Eelco
If h6 is actually a loss for black the score of white will grow. If h6 is so bad you need a lot more than a week for the score to drop towards draw score. Assuming branching factor of 2 Rybka will reach a depth of 30 in a week and still think that white is winning and that other moves besides h6 are better when you go back one move.
> If h6 is actually a loss for black the score of white will grow.
There are no forced losing moves this close to the starting position. I'm assuming 1..h6 is just a draw
>If h6 is so bad you need a lot more than a week for the score to drop towards draw score. Assuming branching factor of 2 Rybka will reach a depth of 30 in a week and still think that white is winning and that other moves besides h6 are better when you go back one move.
If Rybka thinks that other moves are better than 1. h6 which is evaluated as a draw or almost a draw it is only because Rybka already applies a correction to the draw score, otherwise 1. h6 would go directly to the top of the move list.
It should be possible to verify this correction in score assuming it is not obfuscated in some way. I have seen this type of correction in HIARCS output. But, sorry to say, I do not trust the manipulated evaluation output of Rybka very much. Not trusting it as information about how the program works that is. But I should have no real reason to complain because I have not bought the program, and I am happy with the free versions at the moment. Thanks Vas for Rybka 2.2n2! Sorry if the free versions only appear to be some shooting target for a lot of people. But I hope they are also a source of other fun for a lot of people and a reason to buy the commercial version.
Eelco
Rybka is calculating winning chances. A score of 0 means 50% winning chances, a score of 0.16 means 51% winning chances (or something like that.) That both h6 and c5 draw with perfect play doesn't mean you should play h6, because you should play the move that makes it the hardest for your opponent to draw (Unless you're losing, in such case it's the opposite.)
In a sense Rybka is playing stronger than tablebases (in the cases in where tablebases give away material that could have won against an non perfect opponent), at least in the cases in where it is playing perfect chess (Whether the "perfect" line is broad or narrow isn't known.)
In a sense Rybka is playing stronger than tablebases (in the cases in where tablebases give away material that could have won against an non perfect opponent), at least in the cases in where it is playing perfect chess (Whether the "perfect" line is broad or narrow isn't known.)
![[pl]](/mwf/v12/flags/pl.png "Poland")
> What? It's a simple bugfix: just don't hash the root position if you're told not to analyze some of the moves.
Indeed, I completely forgot about this scenario. Next-best/exclude-move are pretty important. Let me think a bit.
Vas
> Now, put her to analyze the starting position and she'll think 1.f3 is the best move up to ply 19.
Well, having a best move at the starting position is theoretically hard. f3 is worst just from a human point of view, because the game resulting is much more difficult to play by a human. But, there are some very clever theoretical insights about this. Pachman in the last chapter of his Modern Strategy says black can equalize any game without errors of any player regardless of white's first move. If that's true then at starting position there is not a best move at all.
This was just an example, you can teach Rybka that any move is the best, even those leading to her being mated.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill