Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Aquarium / Generating Tasks Bottleneck
- - By Horned1 Date 2020-02-26 04:04
I am using Aquarium 2019.

When running in automatic analysis over long periods of time for opening development, once I've hit a few hundred thousand positions in the tree the "generating tasks" begins to take up the majority of the time.
I have a min depth of 40 to add to the tree, am analyzing with 3s OR 13 ply and have an FA check of 13. I am currently at 750K positions and have no problems with min-maxing the tree.

I apologize if this is a noob question, I am quite new to this.

EDIT: In case it was not clear, I'd like to find a way to avoid this bottlenecking, my CPUs are idling 60-70% of the time.

Thanks for the help,
Horned1
Parent - By pawnslinger (****) Date 2020-02-26 06:33 Upvotes 1
Just imagine what it is like when you have 5000k positions!!  Several of my trees are that big or bigger (yep, 5 million).  I don't know if bigger trees exist (but I bet they do!).  Even minimax takes a good deal of time.  I think a lot of folks just turn off "auto prolongation"... in other words, they don't do that "generation" mode any longer.  I do, but I go watch a movie or 2 when I run IDeA on it (and I do a lot of manual additions to the trees).

Periodically, I prune (delete) the infinite analysis trees.  This can speed things up somewhat, but not permanently and even then things are still a lot like watching grass grow.  Myself and others have mentioned this before, nobody paid much attention.  At least no one that could do anything about it.

Not so far as I can tell, anyways!
Parent - - By rov (*) Date 2020-02-26 07:02
I am rather new in IDEA, but 3s and 13 ply is not too low ? I see that the tree has a lot of nodes, but is it better to have quantity than quality ?
Parent - By pawnslinger (****) Date 2020-02-26 09:12
Sure, that seems low, but I don't think it is a contributing factor to the OP's question.  I could be wrong... been known to happen.
Parent - By Uwak (***) Date 2020-02-27 17:49
My approach is similar to Ghengis-Kann's.
Automatic prolongation is switched off. One root node only. 30 seconds and 30 ply depth settings (max 600 seconds to reach 30 ply deep). Generate moves using a custom strategy of:
3 alternatives at 5 plies deep:

<?xml version="1.0" encoding="UTF-8"?>
<strategy description="GMB.Generate 3 alt @ dif50cp, dpt5ply">
  <white maxply="5" ply1="3=50"/>
  <black maxply="5" ply1="3=50"/>
</strategy>

This is not as thorough as Ghengis-Khan's strategy but it also minimizes unnecessary "bloatage"

I generate the initial nodes by repeatedly using this strategy in interesting positions around the root.

After sometime, I switch on the automatic prolongation to clean up loose ends around the node then I stop it again and repeat the above process.

For a small IDeA project say analysis of a middle-game or endgame position, quantity > quality and use fast IDeA settings 5 secs & 10 plies or so to get the job done faster.

But for opening repertoires which are developed over time and analysis is done at multiple points, quality > quantity because of the bloatage.
Parent - - By Ghengis-Kann (***) Date 2020-02-26 18:03 Upvotes 1
Automated expansion inevitably creates a bloated mess.

My advice is to take a more personal hand in steering the analysis by placing Infinite Analysis nodes at and around the tabiya positions, then flesh out the promising lines with appropriate Tree Strategies, and finally walk through the best variations by hand with a powerful IA engine.
Parent - - By rov (*) Date 2020-02-27 09:25
I tested this approach and it is really good and allow to limit time consuming.
Additionally I am testing which engine is best for IDEA.
I read http://talkchess.com/forum3/viewtopic.php?t=70151 about engine stability.
I see that original stockfish is not as well as I thought. (Contempt off).
What I see is that stockfish evaluation change even 0.2 - 0.4  in the next depth - to 40 depth. Higher than 40 is more stabled.
If last node's evaluation is done incorrectly, all IDEA analysis is bad - after minimax process.
Parent - By pawnslinger (****) Date 2020-02-27 15:06
Yes.

Funny though, I kind of like the fact that Stockfish jumps around.  It doesn't always.  And this gives me a feeling for how confident I am in the evaluation, and which positions are critical.  If it didn't jump around, it would be especially hard for me to spot those critical spots that need extra care.
Parent - By Ghengis-Kann (***) Date 2020-02-27 17:51 Upvotes 1
MCTS engines seem to "jump around" less than alpha/beta search.

They rely more on positional considerations and compute a win percentage that doesn't always assume the opponent will make the best move (actually what they currently think the best move is, which might not be correct).

I Crazy K to prepare for OTB play, but it would probably be suicidal in correspondence competition.

Crazy K is Komodo MCTS with the dynamism value ramped up to 200.
My theory is that piece activity is more important in human versus human games than computer versus computer, and I am building an opening repertoire that takes that into account.

From a purely theoretical standpoint it seems that the best approach may be to start at the tabiya position with an MCTS engine like Leela or Komodo MCTS (not crazy), then follow the chosen line with an alpha beta engine like regular Komodo or Stockfish. This is analogous to the balance between explore and exploit that is a setting in Komodo MCTS, but in 2 stages.
The MCTS engines have good positional evaluation skills, so let them do the exploring of which line to follow, then turn on the a/b engine to find all the tactical tricks.
Up Topic Rybka Support & Discussion / Aquarium / Generating Tasks Bottleneck

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill