Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Test Positions / Alternative Chess Test 2
- - By dorszcz (**) Date 2019-09-20 21:29 Upvotes 1
Alternative Chess Test 2 (ACT2) - is a collection of 100 hard chess positions. The test differs from other popular test suites. It mostly contains tricky endgame puzzles and difficult mates. The results do not always correlate with Elo rating. The test is similar to the first ACT problem set, but it is not a replacement; all positions are new. The set is quite difficult. Creating the test I paid special attention to keep it in a shape that proper solution given by engines by luck and random factor is reduced to minimum.

Conditions:
Time limit: 15 seconds per position
CPU: i5-3570 (MP engines: 4x3600 MHz, SP engines: 1x3800MHz)
Tablebases: 6-pieces Syzygy, 5-pieces Gaviota, 5-pieces (and ~80 GB of most popular 6-pieces) Nalimov, all on SSD
Strongest version of engine is used: 64 bit, POPCNT, SSE4.2 etc.
256 MB hash, no books, no learning, 50 move rule is enabled

Results:
72 out of 100 = 72%   -   Sting 14
72 out of 100 = 72%   -   Crystal 260819
69 out of 100 = 69%   -   MateFinder 260819
61 out of 100 = 61%   -   Critter 1.6a
58 out of 100 = 58%   -   Houdini 1.5a
58 out of 100 = 58%   -   Eman 4.00
56 out of 100 = 56%   -   CorChess 3.1 260819
56 out of 100 = 56%   -   Crystal-Honey X5i
52 out of 100 = 52%   -   Honey X5i
44 out of 100 = 44%   -   IvanHoe 9.46b
44 out of 100 = 44%   -   Cfish 240719
44 out of 100 = 44%   -   Stockfish 10
39 out of 100 = 39%   -   Rybka 3 Dynamic
39 out of 100 = 39%   -   AsmFishWCP_2019-07-23
39 out of 100 = 39%   -   SugaR NN 130819
36 out of 100 = 36%   -   Rybka WinFinder 2.2
33 out of 100 = 33%   -   Naum 4.6
33 out of 100 = 33%   -   Spike 1.4
33 out of 100 = 33%   -   Sting SF 3 VE
32 out of 100 = 32%   -   Deep Rybka 4
29 out of 100 = 29%   -   Fire 2.2+ xTreme GH
27 out of 100 = 27%   -   Spark 1.0
27 out of 100 = 27%   -   Wasp 3.75
26 out of 100 = 26%   -   Equinox 3.30
25 out of 100 = 25%   -   Booot 6.3.1
24 out of 100 = 24%   -   Komodo 10
22 out of 100 = 22%   -   SmarThink 1.98 (1CPU)
20 out of 100 = 20%   -   Alfil 13.1
19 out of 100 = 19%   -   Gull 3
18 out of 100 = 18%   -   Nemorino_5.00
18 out of 100 = 18%   -   RubiChess 1.5
15 out of 100 = 15%   -   Fire 7.1
15 out of 100 = 15%   -   Texel 1.08a11
14 out of 100 = 14%   -   Wasp 3.60
14 out of 100 = 14%   -   Xiphos 0.5.6
14 out of 100 = 14%   -   RofChade 2.2
13 out of 100 = 13%   -   Topple 0.7.3
13 out of 100 = 13%   -   Andscacs 0.95
13 out of 100 = 13%   -   Xiphos 0.5.3
13 out of 100 = 13%   -   Arasan 21.3
11 out of 100 = 11%   -   Stockfish 5
10 out of 100 = 10%   -   Pedone 1.9
10 out of 100 = 10%   -   Rodent III 0.273
9 out of 100 = 9%   -   Protector 1.9.0
9 out of 100 = 9%   -   Fritz 11 SE (1CPU)
8 out of 100 = 8%   -   Laser 1.7
7 out of 100 = 7%   -   Senpai 2.0
6 out of 100 = 6%   -   Marvin 3.4.0
6 out of 100 = 6%   -   iCE 3.0 (1CPU)
5 out of 100 = 5%   -   Nirvanachess 2.4
4 out of 100 = 4%   -   Amoeba 3.0
4 out of 100 = 4%   -   Bobcat 3.25
4 out of 100 = 4%   -   RofChade 2.1
1 out of 100 = 1%   -   Demolito 2019-10-29

I want to thank Dann Corbit for help, all suggestions and spending really a lot of time to verify the set.

There are 3 files:
ACT2.epd - test suite
ACT2-Analysis.epd - analyzed positions
ACT2.pgn - popular format, some analysis

http://www.mediafire.com/file/i7u0kz1t5nm9h5l/Alternative_Chess_Test_2_20.09.2019.zip/file

Have fun!
http://dorszcz.blogspot.com
Parent - - By user923005 (****) Date 2019-09-21 02:21 Upvotes 1
I tried a few engines I did not see in your list.
I used ten threads for 7 seconds at 3.2 GHz, which should be about the same.

I got:


65 out of 100 Crystal-x64-mingw
59 out of 100 Sting16
52 out of 100 Houdini 6 Tactical 1
38 out of 100 Stockfish-x64-mingw-dann
31 out of 100 Sugar
27 out of 100 Deep Rybka 4.1 SSE42 x64
17 out of 100 Komodo-13.2a-x64


For the individual problems, I saw:

problem 1: solvers=1, time spent = 108
problem 2: solvers=4, time spent = 65
problem 3: solvers=5, time spent = 51
problem 4: solvers=4, time spent = 57
problem 5: solvers=5, time spent = 52
problem 6: solvers=3, time spent = 77
problem 7: solvers=1, time spent = 108
problem 8: solvers=4, time spent = 63
problem 9: solvers=4, time spent = 63
problem 10: solvers=3, time spent = 79
problem 11: solvers=1, time spent = 114
problem 12: solvers=2, time spent = 91
problem 13: solvers=6, time spent = 31
problem 14: solvers=3, time spent = 78
problem 15: solvers=2, time spent = 99
problem 16: solvers=3, time spent = 81
problem 17: solvers=6, time spent = 37
problem 18: solvers=5, time spent = 48
problem 19: solvers=0, time spent = 126
problem 20: solvers=1, time spent = 108
problem 21: solvers=4, time spent = 63
problem 22: solvers=5, time spent = 55
problem 23: solvers=2, time spent = 92
problem 24: solvers=2, time spent = 99
problem 25: solvers=4, time spent = 65
problem 26: solvers=3, time spent = 80
problem 27: solvers=4, time spent = 65
problem 28: solvers=4, time spent = 61
problem 29: solvers=3, time spent = 80
problem 30: solvers=1, time spent = 108
problem 31: solvers=0, time spent = 126
problem 32: solvers=1, time spent = 114
problem 33: solvers=3, time spent = 78
problem 34: solvers=3, time spent = 79
problem 35: solvers=1, time spent = 108
problem 36: solvers=4, time spent = 64
problem 37: solvers=4, time spent = 73
problem 38: solvers=4, time spent = 61
problem 39: solvers=7, time spent = 22
problem 40: solvers=3, time spent = 78
problem 41: solvers=3, time spent = 81
problem 42: solvers=6, time spent = 29
problem 43: solvers=3, time spent = 76
problem 44: solvers=4, time spent = 66
problem 45: solvers=1, time spent = 114
problem 46: solvers=3, time spent = 78
problem 47: solvers=0, time spent = 126
problem 48: solvers=1, time spent = 113
problem 49: solvers=6, time spent = 38
problem 50: solvers=3, time spent = 76
problem 51: solvers=5, time spent = 50
problem 52: solvers=4, time spent = 60
problem 53: solvers=3, time spent = 83
problem 54: solvers=2, time spent = 94
problem 55: solvers=1, time spent = 112
problem 56: solvers=2, time spent = 91
problem 57: solvers=4, time spent = 66
problem 58: solvers=5, time spent = 56
problem 59: solvers=5, time spent = 56
problem 60: solvers=5, time spent = 40
problem 61: solvers=1, time spent = 108
problem 62: solvers=3, time spent = 83
problem 63: solvers=2, time spent = 97
problem 64: solvers=2, time spent = 98
problem 65: solvers=2, time spent = 91
problem 66: solvers=1, time spent = 112
problem 67: solvers=6, time spent = 26
problem 68: solvers=2, time spent = 95
problem 69: solvers=3, time spent = 74
problem 70: solvers=1, time spent = 112
problem 71: solvers=4, time spent = 57
problem 72: solvers=1, time spent = 112
problem 73: solvers=1, time spent = 108
problem 74: solvers=0, time spent = 126
problem 75: solvers=1, time spent = 110
problem 76: solvers=1, time spent = 109
problem 77: solvers=5, time spent = 42
problem 78: solvers=4, time spent = 71
problem 79: solvers=2, time spent = 97
problem 80: solvers=4, time spent = 67
problem 81: solvers=2, time spent = 96
problem 82: solvers=4, time spent = 63
problem 83: solvers=3, time spent = 81
problem 84: solvers=1, time spent = 108
problem 85: solvers=2, time spent = 93
problem 86: solvers=3, time spent = 76
problem 87: solvers=3, time spent = 82
problem 88: solvers=1, time spent = 108
problem 89: solvers=2, time spent = 94
problem 90: solvers=3, time spent = 77
problem 91: solvers=5, time spent = 49
problem 92: solvers=5, time spent = 42
problem 93: solvers=3, time spent = 83
problem 94: solvers=2, time spent = 99
problem 95: solvers=3, time spent = 84
problem 96: solvers=3, time spent = 82
problem 97: solvers=3, time spent = 75
problem 98: solvers=3, time spent = 84
problem 99: solvers=0, time spent = 126
problem 100: solvers=1, time spent = 113

Five were not solved by any engine at the stated time control. (19, 31, 47, 74, 99)
Only 1, problem 39, was solved by all seven engines.
time spent is the cumulative time to solution for all engines for that problem, with 7 seconds added if the problem was not solved.
Parent - By Akbarfan (***) Date 2019-09-21 08:46 Upvotes 1

> 17 out of 100 Komodo-13.2a-x64


Komodo 13.2a is not out yet
Parent - - By MrKris (**) Date 2019-10-14 07:26
Alternative Chess Test 2
#31

Stockfish & Lc0 have 1.Be1??, Sf 0.00.

Stockfish PV=2 has the solution 1.Bxa5! but 2nd best and flawed with 2.Qd5+?? so it says 0.00 also.

For black after 1.Bxa5!, engine restarted, Stockfish finds black lost in only 1 second.

[Event "Position #31"]
[Site "Alternative Chess Test 2"]
[Date "2019.10.14"]
[Round ""]
[White "Stockfish_19100908c0"]
[Black "Stockfish_19100908c0"]
[Result "1-0"]
[BlackElo "2200"]
[TimeControl "120+2"]
[SetUp "1"]
[FEN "1qNQ4/2p3p1/1p6/n7/1B6/1k6/8/K7 w - - 0 1"]
[Termination "adjudicated"]
[PlyCount "33"]
[WhiteType "program"]
[BlackType "program"]

1. Bxa5 {Stockfish PV=2 0.00= ?? at 02:24, 46 plies; 2nd best to 1.Be1??
0.00.} (1. Be1 {?? PV=1 at 02:28, 47 plies, 0.00= Stockfish_19100908c0} Qa8
2. Qd1+ Kc4 3. Qa4+ Kd3 {--- obvious failure of search: black's king is
much safer here on d3 than in the solution a4, see below ---} 4. Qd7+ Ke4 5.
Ne7 Nc4+ 6. Kb1 Ne3 7. Nc6 Qa3 8. Qe6+ Kd3 9. Bb4 Qa4 10. Qg6+ Kc4 11. Qe4+
Kb5 12. Nd4+ Kxb4 13. Qxe3 Kc4 14. Qe2+ Kc3 15. Qf3+ Kc4 16. Qf7+ Kc3 17.
Qxc7+ Qc4 18. Ne2+ Kb4 19. Qe7+ Qc5 20. Qh4+ Qc4) 1. .. bxa5 {If solution,
1.Bxa5!, is played then Stockfish_19100908c0 finds that black is losing at
1 second!!} 2. Qd3+ Ka4 {--- obvious failure of search, black's king is in
clear danger, quasi-'stalemate' all surrounding squares are blocked or
white controlled, see above ---} 3. Ka2 Qb7 4. Qc4+ Qb4 5. Nb6+ cxb6 6. Qd3
g6 7. Qd7+ Qb5 8. Qd4+ Qb4 9. Qd3 g5 10. Qd7+ Qb5 11. Qd4+ Qb4 12. Qd3 g4
13. Qd7+ Qb5 14. Qxg4+ Qb4 15. Qd7+ Qb5 16. Qd4+ Qb4 17. Qd3 {Zugzwang +-,
Mate in 10} 1-0
ACT2 #31 (solution 1.Bxa5 bxa5 2.Qd3+! +-)
FEN: 1qNQ4/2p3p1/1p6/n7/1B6/1k6/8/K7 w - - 0 1

Stockfish_19100908c0: PV=2 best at bottom; i7-4770K threads=6 ---------------------------------------------------------------------------
 46/15  02:24   1,048,802k  7,276k   0.00  1.Bxa5 bxa5 2.Qd5+?? Kc2 3.Qe4+ Kc3 4.Qe1+ Kc2
 46/29  02:24   1,048,802k  7,276k   0.00  1.Be1 Qa8 2.Qd1+ Kc4 3.Qa4+ Kd5 4.Qd7+ Ke4 5.Ne7 Nc6+ 6.Kb2 Nxe7 7.Qxe7+ Kd3 8.Qd7+ Kc4 9.Qg4+ Kc5 10.Qb4+ Kd5 11.Qb3+ Kd6 12.Qg3+ Ke6 13.Qh3+ Kf6 14.Qf1+ Ke6

Lc0-0.22.0_42872: PV=1 threads=2 + RTX 2060
 24/48  02:27   5,087k  22k  +0.23  1.Be1 Qa8 2.Qd1+ Kc4 3.Qa4+ Kd5 4.Qd7+ Ke4 5.Qg4+ Kd3 6.Qd7+ Ke4 7.Qg4+ Kd3 8.Qg3+ Ke2 9.Qf2+ Kd3 10.Qf1+ Kd4 11.Qf2+ Kd3 12.Qd2+ Kc4 13.Qb4+ Kd3 14.Qc3+ Ke2 15.Qd2+ Kf3 16.Qf2+ Kg4 17.Qh4+ Kf5 18.Qh7+ Kf4 19.Bd2+ Kg3 20.Qxg7+ Kf2 21.Qd4+ Kg2 22.Qg4+ Kf2 23.Qh4+


After 1.Bxa5, engines restarted, both PV=1:
FEN: 1qNQ4/2p3p1/1p6/B7/8/1k6/8/K7 b - - 0 1

Stockfish_19100908c0:
... 31/12  00:01   9,474k  7,665k   0.00  1. ... bxa5 2.Qd3+ Ka4 3.Ka2 Qb7 4.Qc2+ Kb5 5.Qb3+ Ka6 6.Qe6+ Kb5
>>> 32/46  00:01   13,371k  7,743k  -148.95  1. ... bxa5 2.Qd3+ Ka4 3.Ka2 Qb7 4.Qc4+ Qb4 5.Nb6+ cxb6 6.Qd3 g6 7.Qd7+ Qb5 8.Qd4+ Qb4 9.Qd3 g5 10.Qd7+ Qb5 11.Qd4+ Qb4 12.Qd3 g4 13.Qd7+ Qb5 14.Qxg4+ Qb4 15.Qd7+ Qb5 16.Qd4+ Qb4 17.Qd3 Qd2+ 18.Qxd2 Kb5 19.Qd7+ Kc4 20.Qc6+ Kd4 21.Qxb6+ Ke4 22.Qxa5 Kf4 23.Kb2 Kg4
... 58/56  00:03   29,928k  8,635k  -148.95  1. ... bxa5 2.Qd3+ Ka4 3.Ka2 Qb7 4.Qc4+ Qb4 5.Nb6+ cxb6 6.Qd3 g6 7.Qd7+ Qb5 8.Qd4+ Qb4 9.Qd3 g5 10.Qd7+ Qb5 11.Qd4+ Qb4 12.Qd3 g4 13.Qd7+ Qb5 14.Qxg4+ Qb4 15.Qd7+ Qb5 16.Qd4+ Qb4 17.Qd3 Qd2+ 18.Qxd2 Kb5 19.Qd5+ Ka6 20.Qd7 b5

Lc0-0.22.0_42872:
 12/26  00:03   97k  28k  -0.15  1. ... bxa5 2.Qd3+ Ka4 3.Ka2 Qb7 4.Qd1+?? Kb5 5.Qe2+ Kc5 6.Qe3+ Kc4 7.Qe2+ Kd4 8.Qf2+ Ke5 9.Qg3+ Kf6 10.Qf4+ Ke6


#31 FEN: 1qNQ4/2p3p1/1p6/n7/1B6/1k6/8/K7 w - - 0 1
Komodo MCTS-13.1: PV=6 6ths ---------------------------------------------------------------------------
 20/19  09:10   15k  26  -1.52  1.Qd4
 20/19  09:10   15k  26  -1.44  1.Qd2
 20/19  09:10   15k  26   0.00  1.Be7 Kc2 2.Qd5 Qxc8 3.Qa2+ Kd3
 20/19  09:10   15k  26   0.00  1.Bf8 Kc2 2.Qd5 Qxc8 3.Qg2+ Kc3 4.Qb2+
 20/19  09:10   15k  26   0.00  1.Bxa5 bxa5 2.Qd3+ Ka4 3.Qc4+?? Qb4 4.Qxc7
 20/19  09:10   15k  26  +0.04  1.Be1 Qa8 2.Qd3+ Ka4 3.Ne7 Qh1 4.Qe2 Qh3 5.Qe4+ Kb5 6.Nd5 Qa3+ 7.Kb1 Qb3+ 8.Kc1 Qc4+ 9.Nc3+ Kc5 10.Qe7+ Kc6

Lc0-0.22.0_J20-460 2ths + RTX 2060:
 29/65  09:07   19,128k  35k  +0.29  1.Be1 Qa8 2.Qd1+ Kc4 3.Qa4+ Kd3 4.Qd7+ Ke4 5.Qg4+ Kd3 6.Qg3+ Ke2 7.Qf2+ Kd3 8.Qd2+ Kc4 9.Qb4+ Kd5 10.Qb5+ Kd4 11.Qb2+ Kc5 12.Qe5+ Qd5 13.Qxc7+ Nc6 14.Qxb6+ Kc4 15.Qa6+ Kd4 16.Nb6 Qb3 17.Bf2+ Ke4 18.Qe2+ Kf5 19.Nc4 Qc3+ 20.Kb1 Qb3+ 21.Kc1 Qc3+ 22.Kb1 Qb3+ 23.Kc1 Qc3+ 24.Kd1 Qa1+ 25.Kd2 Qa2+ 26.Ke1 Qxe2+ 27.Kxe2
Parent - By dorszcz (**) Date 2019-10-15 10:16
Did you test Lc0 with whole set?
Up Topic The Rybka Lounge / Test Positions / Alternative Chess Test 2

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill