Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Aquarium / Idea- Infinite transposition
- - By SchackMatt (**) [se] Date 2017-05-26 13:43
I have got a problem in some of my trees using Aquarium2017 (build 950) and Houdini 5.

In Idea it seams that a transposition is repeated infinite times and either Aq or Houdini does not stop analysing the line. I have a second tree (same Aq and same engine) in the Scandinavian left over night where it just went on and when I stopped it in the morning it had analysed 15000 position – exactly the same position and sequence (3-5 moves).

I have run the same Houdini in another GUI and it was fine – stopped analysing the same position/sequence propperly. The same goes for Infinite analysis in AQ2017 - no problem.
I have not run with another engine in the same position in AQ due to limited time. The chess must go on. Unfortunately the support desk didn’t recognized it as a problem.[img][/img]

Has any one encountered the same problem? The first question is of cource – Aq2017/Idea problem or Houdini problem?
Parent - - By pawnslinger (****) [us] Date 2017-05-27 01:27
Yeah, this has been discussed several times before... transpositions are a tough nut to crack.  Different engines handle them differently... I use Stockfish for most of my work (at this time) and it usually assigns an eval of 0 to a transposition, which causes problems too.  Usually I am forced to manually deal with them myself.  Using multiple roots beyond the transposition is my current method of choice.  But sometimes there are many transpositions, not easily stepped beyond.  Sometimes I just throw up my hands in disgust and start a new tree with Fill SubTree Power.  At other times I have to find other means around the problem.  It is a real problem with no easy answers.
Parent - - By SchackMatt (**) [se] Date 2017-05-27 01:39
Thanks for you input Pawnslinger. Would you classify it as an Idea problem or an engine problem? As I see it it is Idea that sends the tasks to the engine?!
In e.g Colle where there is a lot of transpositions it is a very big problem.
Parent - - By pawnslinger (****) [us] Date 2017-05-27 04:10
It is both an engine problem and an IDeA problem.  Of course, we deal most directly with IDeA, so that's what most directly affects Aquarium users.

Also, almost any flexible modern type opening, i.e. flank openings in general, have A LOT of transpositions.  So they are hard to deal with for computers.  15,000 evals that you mentioned is a very small number of evals, if you want to effectively deal with these positions.  Some of my trees are extremely old and key positions have 100s of days, not hours, days worth of evals.  When preparing for a trounament, I generally do at least 50,000 new evals in key positions.  The more transpositions possible, the better the opening, because more chances for the opponent to go wrong... victory goes to the player who puts in the work to overcome these problems.

One of the biggest mistakes a player can make is to cutoff analysis before it has sufficiently progressed.  And that is a human judgement call.

The more clear cut the variations, the less chance for mistake, the easier it is for the opponent (or you) to find the correct continuation.
Parent - - By SchackMatt (**) [se] Date 2017-05-27 22:16 Edited 2017-05-27 22:20
I usually use Aq to find interesting positions I haven't played before in corr. and my opening trees are also rather big as well. Much of my time is spent on analysing "impossible" moves to a rather deep depth. I use a lot of manaual steering in IDeA while infinite analysis is running in parallell.

When I mentioned 15000 evals I ment:
One node looked like this (just started on this particular node and IDeA was runing over night):

Move       IDeA      N
6…cxd4   +0,36     474
6…bxc4   +0,46     109

6…Bb7    +0,53 15000

And the line starting with 6…Bb7 looked like this: 6... Bb7  7. Be2 Bc8 8. Bf1  Bb7 9. Be2 Bc8 10. Bf1 Bb7 11. Be2 Bc8 12. Bf1 Bb7 13. Be2 Bc8 14. Bf1 Bb7 15. Be2 Bc8 16. Bf1 Bb7 …. And continued.
Parent - - By pawnslinger (****) [us] Date 2017-05-27 22:54 Edited 2017-05-27 22:57
I would be tempted, in this case, to set roots beyond the repeating move.  Or if that position is not of further interest, color Bb7 red (bad move) so the analysis of the other moves will be focused upon.

To give you an idea of what some of my analysis looks like, here is a key position from my Najdorf tree... lots of transpositions both before an after this spot...
Parent - - By SchackMatt (**) [se] Date 2017-05-28 11:20
The number of evals in your tree is way beyond mine. I have only one standard laptop 4 core (+4) on which I do anything else as well. Leaving it running over a couple of days doesn't expand my tree overwhelmingly. In the opening I usually run 30sec/23moves. It does vary a bit though depending on my experiene in the particular opening/position.

Thank you for the suggestions. I have actually done exactly as you suggested and I will extract the subtree after the second repetition.
Parent - - By pawnslinger (****) [us] Date 2017-05-28 18:04
As you can see, this tree has developed over a number of "days".  And actually even more than that, because the file(s) containing the counts was lost several times.  So the day counter is just the most recent iteration.  I now use a desktop based on the new Ryzen cpu from AMD, but for most of this work, I was using a 4 core Intel cpu.  And it wasn't running continuously, it has just been working on this tree off and on for more than 5 years.  You can see from the eval counts, that many dead-ends have been followed, as many bad moves have quite large counts.  This is the method of IDeA, it follows a line of play until it is proven bad, then tries a different line.  This is very time consuming, so the best advice I can give is minimize analysis of bad moves.  Use whatever tools to do that that you can, but do not abandon IA... many tactical shots are found thru IA and verified thru IDeA.
Parent - - By SchackMatt (**) [se] Date 2017-05-29 01:19
I totally agree with you concerning IA. Actually IA is my main method and I always feed IDeA with IA result before starting IDeA. Often I use another GUI than Aq when running IA even if I "always" in Aq run 1 core IA in parallell with IDeA and then IA  usually is running a couple of plies deeper than the IDeA-node(s).

I also to some extent use the statistics when deciding where I should go deeper. I say "to some extent" because I don't always really understand how to inteprete the statistics.

Right now I use Komodo 10 (have not converted to Komodo 11 yet) in IA and Stockfish in IDeA. In interesting positions I also could run all of the three big ones (Komodo, Stockfish and Houdini) in IA/Aq before starting IDeA.

I think your suggestion to increase depth in IDeA to sort out "bad" moves earlier is very interesting. As I mention before I often use 30sec/23 plies in the opening (depending on among other things the position/opening/my knowledge etc.). When I get the time I will run some tests and increase to about 30 plies to compare.
Parent - By pawnslinger (****) [us] Date 2017-05-29 02:41
Of course, it is a trade-off... the greater the depth, the longer each step in the analysis will take.  I also use a high minimum in the Sandbox, often feeding only the very best moves to IDeA.  No matter what the eval depth is in IDeA, I am convinced that spending time only on the best moves is the correct idea, read "How to Think Like a Grandmaster" by Kotov for more about this.  So I routinely use 5-7 variations, min time of '1' and min depth of '28' (or more) in the Sandbox.   Sometimes I let the analysis there proceed for quite awhile and navigate thru the main variations BEFORE even starting IDeA.  In extremely difficult positions I have a tendency to turn off the automatic prolongation of IDeA.  Of course, when you do that you must be very careful and make sure that all bases (significant variations) are explored in the Sandbox... to get all moves fit for consideration into the IDeA queue.

It goes almost without saying, but perhaps I should anyway... all parameters should be tailored per position.  No parameters are fixed in concrete.  For example, generally the further along you are in the game, the deeper the analysis should be, both overall and at each step.  Also some games, like variations of the Kings Indian (that I have discussed here before) are extremely trap prone... and engines often fail to give correct analysis even at great depth and over many days.  I am currently playing in one final (The Veterans World Cup) and just started a semi-final of the WCCC... I say that not to brag, but to point out that only after many games of the Kings Indian, do I feel that I am starting (just starting) to have enough experience to play the opening at a high level.  And I am still losing games in it.  :-<
Up Topic Rybka Support & Discussion / Aquarium / Idea- Infinite transposition

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill