Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Aquarium / Evaluation Error IA and IDeA tree
- - By sbm (*) Date 2020-05-12 07:55
I have not seen error like this before :roll:

16.Kb1 at +1.03 ???

I noticed this only because it was right in front of me...wondering if this kind of errors are hidden somewhere deep in the tree..unnoticed...
In this project I have the engine set to Add moves and eval to Idea tree in IA analyse mode...at depth of 30.



The question is from where and how AQ managed to get that erroneous evaluation and inserted it in the tree!? Of course that after I noticed the error, I put the engine to Analyse Selected moves...leaving only Kb1 ..but of course nothing, not even at the beginning on some low depths... it is a huge blunder actually, drops a piece instantly...but puts a poker face on like nothing happened :cool:
I manually minimaxed the tree to see if it disappears ...nope, I had to let IA run again and pass through depth 30 to write over that nightmarish eval.
I have seen a lot of garbage-like evaluations similar to this at some distant leafs along an IdeA analysis (mostly when Automatic tree expansion is ON), but this is not the case, this is IA face value
Parent - - By pawnslinger (****) Date 2020-05-12 08:34
This is what I have complained about in the past.  It is what I call a "singleton".   In other words, a single incorrect evaluation that throws off the entire tree and is very hard to find (it is buried deep in the tree).  Once found, it is not hard to correct, just manually prolong the variation and minimax will erase the erroneous evaluation.  I consider it a serious bug in the program.  I think the FA parameter was added to try and address this issue, but actually it just creates more chances for singletons to occur... in my opinion... so I have FA disabled.  And I try everything I know to confirm moves in my intended variation of play.  Backsliding is one technique that has been discussed in this forum, and I use it often.
Parent - - By dickie (**) Date 2020-05-12 11:40
The aberrant evaluation complained of here is a false positive, and we do not know if it is the engine or Aq that is responsible, though I suspect more likely the engine. In any event, when working in automatic IDeA, a significant false positive like this would be overwritten at the next prolongation.
Parent - By pawnslinger (****) Date 2020-05-12 15:47
Yes... if it is automatically prolongated.  I have seen instances, many, where the evaluation is far enough off (from the root score) that a quick prolongation is unlikely.  Sometimes it can take a long time for prolongation to occur. 

I agree the responsibility for the amount of the evaluation is the engine.  But is it not the purpose of FA to prevent such?  Yet, I find it does not really do a good job of doing so.  I believe that is due to the fact that FA generates only 1 candidate at each prolongation step, and the final prolongation step could be erroneous.
Parent - - By centipawn (*) Date 2020-05-12 15:31
It is odd. The SF column says Stockfish evaluated this to +1.03 at depth 30. I tried it and could not get SF to evaluate this position (after Kb1) as anything positive for white at all.

I can only guess - here is my theory: one of the previous moves was 'unexpected' to Aquarium in the sense that it was not one of the highest rated moves. The screenshot shows no N value for 16. Kb1. I imagine Aquarium asked Stockfish to evaluate an alternative after, say, move 14, and Stockfish came up with a line that included 16. Kb1 followed by something other than 16...exd4 and Stockfish reported this at a depth of 34 or so with a rating of +1.03. Aquarium put the first moves of the line in the tree, and the last move it copied was 16. Kb1 with the corresponding depth of 30.

I'm inclined to think this is due to the horizon effect most engines exhibit, not a bug in Aquarium. I think this occurs only after a move that was not considered likely by Aquarium has been made (otherwise, why does 16. Kb1 not have an N value in the tree?), and that goes away when the position is a made a root and Aquarium ponders on it leisurely for a while.
Parent - By pawnslinger (****) Date 2020-05-12 15:57
Could it be that the OP is feeding IA analysis directly into the tree?  If so, it would be highly likely that end points of such variations are not very reliable.

Hence the importance of using a reasonably large value for "Min Depth".... to prevent such end points from being recorded in the tree.
Parent - By sbm (*) Date 2020-05-14 10:29 Edited 2020-05-14 10:38
That is why I'm almost 100% sure that it is not engine related error. SF does not exhibit errors like this no matter what are the conditions. As it shows in the screenshot, it does not even needed to get to depth 30 to clarify the evaluation, and this is obviously not some kind of complex situation where such a sharp contrast might emerge after a depth-wise deeper analysis...on the contrary.

There is no N value because it was totally new, "freshly" added variation, and for the time being without further branching...by the way N value...I have very large and deeply analysed trees, where one of the main lines at some point shows no N value at all, but entering in that very variation, right at the next move it shows tens of thousands! Obviously it has nothing to do with minimaxing, for these trees are several years old and have been minimaxed many times. It is just a bug.... (out of the question is trying to resolve it using Fill Subtree Power, as AQ crashes doing that with way smaller projects where aren't several hundred thousand positions in the tree...)

Btw, in my opinion AQ does not ponder at all...in anyway :evil: Actually, if we think a little deeper, it really does nothing more than that it concatenates some results (using simple String like functions), compares the data in some way using maybe a pinch of Boolean logic (but sorts the results according to our own settings!), count the variations if one asked for (N value) and finally sorts&presents the data in those columns but again, based on our settings.

All the bells and whistles like Prolongation, Analysis Quality, Insert this Insert that, tree expansion, tree width, limit white limit black limit variation length, relative to this score or that score, shapes so on so fort (including of course all the TGS-s too!) are nothing more than influencing, limiting and channeling the calculation process - made of course solely by the engine - within certain limits based on the specified parameters. And... at the end of the day...same goes for the calculation :wink: All those variations "produced" by custom tasks, root nodes, TGS-s are nothing more then tiny fractions of a calculation started by the engine as "infinite", only that is limited&truncated by time, depth, or value parameters
Up Topic Rybka Support & Discussion / Aquarium / Evaluation Error IA and IDeA tree

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill