Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Aquarium / IDEA running zombie tasks without any job assigned to
- - By sbm (*) Date 2018-03-05 07:21
I started to experience a weird issue regarding the tasks in IDEA.

As it is visible in the picture, all the engines are idle (of course IDEA is running, project is active), non of them analyzing so they finished they job, but there are two (sometimes are two, sometimes only one...) tasks stuck in the Stage Status window, tasks that do nothing! (and the "Tasks:"  shows some erroneous information too 5+0..but there are only 2 green blocks)
It is very frustrating to find Aquarium in this state, just staying doing nothing, even though a set of tasks with those settings it takes only a few minutes to finish and to start another cycle. Huge waste of time and computational power of several hours in a row in this state! You can not finish those tasks even manually, as they do not have delete, finish or something when one clicks on them (I tried from "Manage" too, but from there I can not delete tasks when the project is active and Idea is running anyway), You have to stop Idea and make the project inactive to be able to clear those zombies!
Any "Idea" what might be the cause of this? No error message of any kind..
Parent - - By chesspjn (*) [se] Date 2018-03-05 19:01
I have noticed that. Click on manage and do a minimax and the engine will start again. It looks like it happens when there is a long que or have been.
Parent - - By sbm (*) Date 2018-03-06 07:08
I use frequently the manual Minimax from Manage, tried that in this situation too but had no effect. I tried the "Unload All Inactive" too, to clear the engines out from memory but nothing has changed.
Parent - - By dickie (**) [gb] Date 2018-03-06 07:32
Have you tried deleting your infinite analysis tree files? There used to be problems with older Aquarium versions when using very short analysis times (10 secs here) exacerbated by the high engine count.
Parent - - By sbm (*) Date 2018-03-06 18:42
No, I did not. Now that You mentioned..I rather moved them away from AQs folder (as I need them)...and tried again. Same story, but I think that I found what was the culprit! As the position was more toward endgame many of the positions where very near to Tablebase or Lomonosov query.
And now when I started the project again, right at the beginning I had that classical error message regarding that can not connect to TB7 Lomonosov server..all tasks will be deleted (don't know the exact text but something similar). I clicked OK that message, the project continued, but when the tasks where finished now I had 1 zombie which has blocked the whole process at that state! And I recall that when I had more zombies, I got at the start of the job more Lomonosov related messages too ... in a row..surely as many as zombies I got at the end!
Funny though, in the message say that the task will be deleted..well not quite.:confused:
Parent - - By Ghengis-Kann (***) [us] Date 2018-03-06 22:52
I have also seen this and am not using tablebases.

I think I just manually stopped IDEA and they either went away or I deleted all tasks.

There are certain conditions that can put an engine in a state where it is impossible to complete the task.
I'm not sure if this is still true with the current version, but I recall seeing engines that were set to a depth limited search sit and do nothing until max time is reached when they were asked to calculate to a depth that goes beyond mate by force.
That may not even be an issue with Aquarium, but rather with the engine.

This is one of the reasons I switched to time limited parameters.
Parent - - By sbm (*) Date 2018-03-08 06:58
The issue is that You are not even aware that the whole project is in that state...the computer might running for hours without actually doing nothing! First, one has to notice this, and yes, You have to manually stop IDEA and delete the tasks. I did not noticed (yet) issues like this regarding depth limited search.

Actually I've started to experience some Analysis settings like this "10s AND depth 17" because I wanted to quickly populate that tree...I noticed that this new pc I built reaches depth of 23-24 in a few seconds (in this specific position, and with "wait for next depth" in less then 10s) so I thought that the aforementioned settings are fair enough to have a decent tree to browse through it..this way I intend to have some kind of "base", a broader three to work with, and later when I found that it is enough what I got with this settings, I would raise the depth to higher level. This way the lines that IDEA considers main lines will have higher depths, but the so called side lines will have decent depths too. And for the lines that I consider crucial intersection, I will put in IA too.
Parent - - By Ghengis-Kann (***) [us] Date 2018-03-08 17:55
Your settings seem reasonable, and it is certainly some kind of bug, but maybe a bug in the engine instead of Aquarium.
Does the same thing happen if you use a different engine?

I have moved away from automated analysis because of things like this and because it creates huge trees that are plagued by poor evaluations at the leaf nodes.

Now I use IA in multi-variation mode to generate the IDEA tasks, and add lines from my intuition to those the computer likes. Suspicious evaluations are followed out to the leaves and subjected to further analysis.

It's a lot more time consuming than automatic expansion, but I learn from it and the quality of the analysis is vastly improved.
Parent - - By pawnslinger (****) [us] Date 2018-03-09 21:09
I still use automatic expansion in addition to IA technique similar to what you describe... but I have made 1 change recently that has helped tremendously with automatic expansion... I have changed most of my projects to time controls like this: 120 seconds AND depth 26 with maximum time of 9999 seconds.  Now these values are just examples, my actual values vary all over the place depending on what the project/position seems to call for.  I have used 60 seconds as a lower limit and 600 as an upper, 23 ply as a low depth limit and 30+ as an upper limit... just depends... on how I feel about the position.  I find that using these limits (especially the higher lows of time) gives Aquarium a better chance at getting good alternative selection early on.

On a separate note, I just finished my last norm for the CCM title, and my rating is back at its high (after a dip down a bit).  So I am feeling good about Aquarium and Chess in general...
Parent - - By mist (**) [gb] Date 2018-03-10 08:14

>I just finished my last norm for the CCM title

Well done, welcome to the for the next one?! :smile:
Parent - - By pawnslinger (****) [us] Date 2018-03-11 01:44

Hehehe, thank you.  Yes, they keep getting harder.  But I remember when 2300 seemed an unreachable goal... now I am feeling like I might be able to reach 2500 before I am done.  My 73rd birthday is coming up pretty soon <GRIN>.

Looks like I will miss out on the Veteran's World Cup, I am playing in a final now, and thought I had a chance, with 2 wins likely, but now it is looking like 3 wins will be needed to take 1st place.  Oh well, it will be my best finish so far (a 4th in the U.S. Golden Knights was my previous best finish).
Parent - - By mist (**) [gb] Date 2018-03-11 09:00
Just ahead of you on the birthday side of things.!  Not so sure about myself ever getting to the 2500!!!
Parent - - By pawnslinger (****) [us] Date 2018-03-11 09:11
I am not sure either, just a bit more optimistic than I used to be.  I can recall thinking that 2300 was out of reach, but I am nearly to 2400 (only 9 below)... and 2500 no longer seems insurmountable... just another goal to work toward.  My elo has been on a steady, if somewhat slow, upward trend lately.
Parent - - By dickie (**) [gb] Date 2018-03-11 15:41
Congratulations on your title! I am also heading for 73 this year - we are still young! If you want to target 2500 I suggest you take your recent experience and apply a similar thought process to the foundation stage of your trees. Getting better piece placement coming out of the opening will help you a lot against tougher opposition. Good luck!
Parent - - By pawnslinger (****) [us] Date 2018-03-11 18:05
Against tough opponents (any opponent really) I find that "unclear" yet "sound" positions offer the most "dynamic" chances.

Man, I am full of cliches this morning!

But the idea is still true.  And I find that tough opponents do the same thing.  By gosh, they seem to know where all the unclear dynamic positions are to be found.  There is this one really strong player that I have gone up against twice now, a guy with a Spanish name, but I don't want to name him.  I don't think he is from Spain, but his name "looks" Spanish, whatever that means.  Anyway this guy, who always seems just a bit ahead of me rating wise, always finds the edgy unclear positions.  Makes it darn hard to hold the draw.  Darn hard.  I think he is an IM now, but I have easier times playing SIM and GMs.... this guy is a shoo-in for GM and maybe more.  Who knows.  But he is obviously a "class" above me.

Another fellow I have played only once, is really really strong.  He beat me cold.  Just cold as you please.  And every time lately I see a new rating list, he is always getting a +40 or something.  I don't know how he does it.  I don't know his title, but at the rate he is going, he is going to be WC.  If he doesn't burn out.  I think he is a pretty young guy and they tend to do that, burn bright and fast.
Parent - - By Ghengis-Kann (***) [us] Date 2018-03-12 17:50
Setting max time to 9999s is asking for trouble.

Your whole analysis will be paused if one engine gets itself into a state where the task does not complete (Which is the topic that started this thread).

I generally set max time to about 5X min time.
Parent - By pawnslinger (****) [us] Date 2018-03-12 22:15
The problem is, that the engine I use usually quits before it gets to the first time limit, if the max is not set high enough.  I have only run into the zombie problem once and I was able to clear it by just making the project inactive and exiting the program (after deleting all tasks and doing minimax).  The zombies disappeared and never came back (yet)... that was about a month ago.  I used to have problems with the max time too large because the engine I was using at that time would over analyze lost positions.  Spending too much time on bad moves when it should have been using more time on good moves.  This engine was Houdini, and I stopped using it after I pointed this problem out to the author (he blamed Aquarium).  Anyway Houdini has been revisited since, but only briefly... I'll stick with my latest engine (of appox 2 years usage).  And so far the 9999 time limit has not been a big problem here.  Keeping my fingers crossed.

In my mind, the theory of having a extra high time limit is that I don't really care how long it takes to get to the required depth.  As long as it is working on a reasonable alternative move, because I usually let it run for "a long time" with 10-13 engines plowing thru.  If 1 or 2 get stuck for awhile, I really don't notice (unless the queue empties and these engines are still stuck).
Parent - - By sbm (*) Date 2018-04-17 06:02
I just find out this:
where this statement looks that invalidates my Analysis setup:

I started IDeA with "10 seconds AND 1 ply" and after generating 1000 positions from the base position,  I decided to do "30 seconds AND 15 plies" starting from the same position. How will this affect the evals for the 1000 positions in the existing tree?
The tree will be preserved so existing positions will not be reevaluated. Some of the evaluations will be replaced by the 15-ply ones due to minimax. If you want new evaluations you must clear the analysis tree or switch to another tree.

Anybody knows if this is still valid?!
Parent - - By pawnslinger (****) [us] Date 2018-04-17 06:28 Edited 2018-04-17 16:17
As far as I know, yes, it has always been like this.  Same holds true when you upgrade or change engines.  The old evals only get replaced when a minimax updates them.  Naturally, this means that the tree is basically "contaminated" (in some sense) by the old evals.  For a long time now, I have requested an easy way to overcome this, by allowing some easy way to process the leaf nodes with new evals, if you could do this, then it would not be a problem.  With all leaf nodes re-evaluated, minimax would update everything.  But nobody seemed to like my suggestion.  So I plod on...
Parent - By sbm (*) Date 2018-04-17 11:03
Sad. I started to think that I found a proper way to fine tune my analysis setup....and it looked pretty logical.
Up Topic Rybka Support & Discussion / Aquarium / IDEA running zombie tasks without any job assigned to

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill