Just a quick update:
Tactics testing is getting harder due to the improvements in hardware and, especially, software. My 100 position testsuite is showing its age with Critter 1.6a getting 91% and less than 10 seconds solving time. I've also discovered that a few positions were cooked. Furthermore I (foolishly) sent my testsuite out to about a dozen people who asked for it and I'm seeing some of the positions all round over net.
So, out with the old, in with the new. Over the last year I have started to compile some SERIOUSLY difficult tactical positions. This time each position is IDeA checked to weed out errors & cooks. This is very time consuming, it takes about a day to check a position and many positions are not suitable. So bear with me. There won't be any updates for a while but hopefully by October I'll be ready to roll with a new testsuite which will be very hard, a 100% error free and totally private so programs that cheat (by using databases) can't cheat on these.
If you want to help please PM me if you think you've got a position which is seriously hard...and not too well known.
Anyway, most of the positions are too easy for modern engines, so it needs replacing.
If I'm happy with the new suite I can always make the old one public.
> It is assumed that any private copy of software given in trust should not be transferred into a third parties hands without first the authors authorization.
yep, although it's partly my fault - I got a load of emails from people I'd never heard of and I just emailed them without thinking.
> One engine added a load of positions to its database and 'solved' them instantly.
Chess engine developers in general are real scumbags! You wouldn't invite them over for dinner; they'd steal the silverware and make off with your hard drives.
> Chess engine developers in general are real scumbags! You wouldn't invite them over for dinner; they'd steal the silverware and make off with your hard drives.
I guess some are ok, but the ones which put their paw print on a well known piece of code and sell it off as a new engine...I'll hide the silverware
People who are serious about testing a chess engine should develop their own test position suites; something I have been doing for many years!
> I would recommend you DO NOT accept positions from other people; keep everything in your test suite private and original to you. This way you can be sure as you possibly can that no programmer will every know what your test suite comprises.
> People who are serious about testing a chess engine should develop their own test position suites; something I have been doing for many years!
Point taken. I'm only accepting positions from people I know are totally reliable. If possible I may try to alter some positions without changing the best line of play. This could be done by moving an irrelevant pawn or something.
are you still sharing your old test set?
But there are loads of testsets out now which are quite good.
when creating such a test, how is it possible to avoid a bias in favor of a particular engine since presumably an engine was used to locate the problems in the first place?
> I am looking for a test that is<=1year old and that is ~ as hard as this test. any links?
Try over at Talk chess. Jon Dart has posted something recently.
> when creating such a test, how is it possible to avoid a bias in favor of a particular engine since presumably an engine was used to locate the problems in the first place?
I don't bother with that. I just find positions that look challenging and see if the engines can solve them. When I find a position that can't be solved in a few minutes I setup an IDeA project to test the position for soundness. If it turns out to be sound...I add it to my testsuite. Once you have 100 positions or so it balances out so it's fair for all the engines.
What does cook mean?
Some positions were cooked?
> Well some moves might be OK for the side to move, but the important thing is that there's only one winning move
By OK does that mean it could be a draw?
So 1 winning move yet there is some moves that hold a draw?
> By OK does that mean it could be a draw?
> So 1 winning move yet there is some moves that hold a draw?
Yes that's possible. The intention behind a tactical testsuite is to give the engine an opportunity to find one move that is far better than all the rest. It could be that this move is the only one which wins (while all the rest are only ok, or only draw) or it could be that this move is the only one which draws (while all the rest lose).
The only other consideration is that the move ought to be tactical in nature (as opposed to being positional).
So the only problems an engine will have in the tactical category is positions that have zugzwang moves, or positions with many quiet moves (no checks or threats).
In theory positional themes are long term things in computer chess.
tactical themes are short term things are short term things in computer chess.
With the extensions in modern engines any engine can see long term tactics that don't require zugzwang moves or many quiet moves.
So good luck with the tactical suite. I'm not sure how you will pull it off with some controversial positions. Also I'm not sure how you will know if an engine gets a move right because of built in positional knowledge vs. because the engine saw all the relevant lines to the end.
to stop the opening book helping to the match,both engines will play with a neutral book,or without book
since you are the suspicious person,are you going to arrange that match between the cluster and a 12,24 or 32 (the maximum cores that Houdini can use) core Houdini?of course,you must pay all the expenses
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill