Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / R4 on the Minimal Book Test
- By Uly (Gold) [mx] Date 2010-05-26 03:21 Edited 2010-05-26 03:25
I like this test because it's extremely consistent and stable despite the low number of games, in fact, I got this exact same result two times before with the Betas, including a test at 1+'10 time controls, and in another I got a 8.5/16 result but only because Rybka blew a drawn game. That Rybka hasn't improved only means that the new TC defaults aren't helping her in these conditions, and indeed she seems to be playing too slow at the start of the game. A main difference from the Betas is that the draw% has dropped from 50-56% to 37.5% with a long string of decided games. When in doubt, Rybka seems to go wild.

Minimal books is a test proposed by Larry Kaufman where you force as few moves as possible in each game. While he suggested each move one by one (a4, b4, c4...) I chose to run these positions: Opening position, 1.e4, 1.d4, 1.Nf3, 1.c4, 1.e4 c5, 1.d4 Nf6 and 1.e4 Nf6.

Quad @2.33Ghz - 128MB RAM
Rybka: 4 Proccesses. 64bit Ponder OFF. Default setting.
Opponent: Stockfish 1.7 4CPU 64bit.
1 Minutes/Game + 3 Seconds/Move
Engine Match, 2010.05.25
                                  Score     1234567890123456
--------------------------------------------------------------
1: Deep Rybka 4 x64              9.0 / 16   1==11001010=1===
2: Stockfish 1.7 JA 4CPU 64bit   7.0 / 16   0==00110101=0===
--------------------------------------------------------------
16 games: +6 =6 -4
56.25%
+44 ELO


(for comparison, Rybka 3 got -14 in this test against Stockfish 1.6)

Even with bad time management, and not TB, that seems to make Rybka run into drawn engames while scoring them +2.00 and so on, I think testing with 3men tablebases (for KPK endings) is next, to see if it improves results linearly (4men would come later, then the relevant positions with 5).

Games attached at the end. Shortest wins by both sides:

[Event "1 Minutes/Game + 3 Seconds/Move"]
[Site "Engine Match"]
[Date "2010.05.25"]
[Round "11"]
[White "Stockfish 1.7 JA 4CPU 64bit"]
[Black "Deep Rybka 4 x64"]
[Result "1-0"]

1. e4 c5 2. Nc3 {+0.44/19 4s } Nc6 {+0.23/15 7s } 3. Bb5
{+0.36/19 6s (Nf3) } e5 {+0.12/15 7s (e6) } 4. d3 {+0.44/19
4s } a6 {+0.10/15 6s } 5. Bxc6 {+0.48/20 6s (Bc4) } dxc6
{+0.07/17 13s } 6. f4 {+0.52/20 4s } exf4 {+0.07/16 4s }
7. Bxf4 {+0.36/21 13s } Nf6 {+0.09/16 8s } 8. Nf3 {+0.36/19
3s } Be7 {+0.07/15 4s } 9. Qd2 {+0.44/18 4s (O-O) } O-O
{+0.03/14 7s (c4) } 10. O-O-O {+0.24/16 4s (O-O) } b5
{+0.05/14 8s (Be6) } 11. Kb1 {+0.44/17 3s (Rhe1) } a5
{+0.04/14 5s (Be6) } 12. h3 {+0.28/16 6s (d4) } a4
{-0.11/14 8s } 13. g4 {+0.40/17 2s (Ne2) } Qb6 {-0.15/15 7s
} 14. Ne2 {+0.16/16 3s } a3 {-0.14/14 4s (c4) } 15. b3
{+0.20/17 2s } c4 {-0.12/13 2s } 16. Qe3 {+0.16/17 4s } Qb7
{-0.04/14 7s (Qxe3) } 17. d4 {+0.32/16 2s } cxb3 {-0.51/12
4s (Re8) } 18. cxb3 {+0.28/17 2s } c5 {-0.55/13 0s } 19. d5
{+0.24/17 4s } c4 {-0.50/14 4s (Re8) } 20. Ng3 {+0.12/15 3s
(Ned4) } Re8 {-0.56/11 3s } 21. Be5 {0.00/15 2s (Rhe1) } h6
{-0.47/11 2s (Ra6) } 22. Rhe1 {+0.32/15 6s } Bd7 {-0.41/12
6s (Nh7) } 23. g5 {+1.49/16 5s (Nf5) } hxg5 {-0.08/10 1s }
24. Qxg5 {+1.01/17 3s } cxb3 {+0.15/11 2s (Ra6) } 25. Rg1
{+2.74/16 2s (axb3) } bxa2+ {+0.51/9 3s (Qb6) } 26. Ka1
{+3.31/18 3s } g6 {+1.88/10 3s } 27. Bxf6 {+3.87/16 1s }
Bxf6+ {+2.32/10 3s } 28. Qxf6 {+3.75/13 0s } Ra6 {+2.32/10
0s } 29. Qf4 {+3.87/18 3s (d6) } Qb8 {+1.74/10 2s (Qa7) }
30. d6 {+7.27/16 4s (Qxb8) } Re6 {+3.81/9 3s (Qc8) }
31. Ng5 {+11.35/17 3s } Qf8 {+4.59/10 1s (Qe8) } 32. Nf5
{+29.65/17 3s (Nxe6) } Rxe4 {+13.26/8 1s } 33. Nxe4
{+84.08/19 3s } Bxf5 {+M11/11 5s } 34. Qxf5 {+M11/13 1s }
Qh6 {+M10/10 2s } 35. d7 {+M10/12 0s } Qg7+ {+M9/11 4s }
36. Nf6+ {+M9/10 0s } Qxf6+ {+M8/11 3s } 37. Qxf6 {+M8/10
0s } Rxf6 {+M7/14 3s } 38. d8=Q+ {+M7/9 0s } Kg7 {+M6/13 3s
} 39. Rdf1 {+M6/16 1s (Rgf1) } Rc6 {+M5/11 3s (Rxf1+) }
40. Qe7 {+M5/6 0s (Rxf7+) } Rc1+ {+M4/10 1s } 41. Rxc1
{+M4/6 0s } b4 {+M3/10 3s } 42. Rc6 {+M3/6 0s (Rc7) } Kh8
{+M2/10 3s (Kg8) } 43. Qxf7 {+M2/6 0s } b3 {+M1/11 3s (g5)
} 44. Rc8# {+M1/6 0s} 1-0

[Event "1 Minutes/Game + 3 Seconds/Move"]
[Site "Engine Match"]
[Date "2010.05.25"]
[Round "10"]
[White "Deep Rybka 4 x64"]
[Black "Stockfish 1.7 JA 4CPU 64bit"]
[Result "1-0"]

1. c4 e5 {0.00/19 3s } 2. Nc3 {-0.02/17 10s } Nf6 {+0.12/21
15s (Nc6) } 3. Nf3 {0.00/16 10s } Nc6 {+0.12/20 4s } 4. e4
{+0.02/15 4s } Bb4 {+0.24/20 9s } 5. Be2 {0.00/16 4s (d3) }
Bxc3 {-0.08/19 4s (O-O) } 6. dxc3 {+0.02/16 4s } Nxe4
{-0.12/20 2s } 7. Nxe5 {+0.04/16 3s } Nxe5 {-0.32/20 2s }
8. Qd5 {+0.01/16 7s (Qd4) } d6 {-0.20/20 4s (O-O) } 9. Qxe4
{+0.03/15 1s } O-O {-0.20/19 2s } 10. O-O {+0.02/15 5s }
Re8 {-0.20/19 3s } 11. Re1 {+0.01/15 10s (Be3) } Bd7
{-0.04/17 3s } 12. Bf4 {+0.01/15 11s } Bc6 {-0.08/18 3s }
13. Qd4 {+0.06/14 5s } b6 {-0.04/18 4s (Qf6) } 14. Bg3
{+0.06/13 5s } Qd7 {0.00/18 5s (a5) } 15. b4 {+0.11/13 8s }
Rab8 {0.00/18 5s (Bb7) } 16. a4 {+0.13/12 3s (Rad1) } a5
{+0.24/16 3s } 17. bxa5 {+0.05/12 2s } bxa5 {+0.28/17 3s }
18. c5 {+0.17/13 2s } f6 {+0.20/17 3s (Rb2) } 19. cxd6
{+0.18/14 6s } cxd6 {+0.16/18 3s } 20. f4 {+0.15/14 5s
(Ra2) } Nf7 {+0.32/17 3s } 21. Bd3 {+0.21/14 4s (Bc4) }
Rxe1+ {+0.12/16 3s } 22. Rxe1 {+0.21/15 3s } Rb2 {+0.12/17
2s (Bxa4) } 23. Bf2 {+0.30/12 1s } h6 {+0.12/18 4s (Nd8) }
24. h3 {+0.45/13 2s } Kf8 {+0.44/18 3s (Qb7) } 25. Bg6
{+0.37/14 2s (c4) } f5 {+0.48/18 6s (Qb7) } 26. c4
{+0.68/12 3s } Rb4 {+1.05/17 5s } 27. Qc3 {+0.80/12 1s }
Rxa4 {+1.09/17 4s } 28. c5 {+0.92/12 1s } d5 {+1.25/17 3s
(Rxf4) } 29. Bd4 {+1.34/13 3s } Kg8 {+1.13/18 3s } 30. Re3
{+1.38/14 3s (Rb1) } Rc4 {+1.73/16 4s } 31. Qd2 {+1.50/15
10s (Qd3) } a4 {+1.85/16 3s } 32. Rg3 {+1.99/12 1s (Kh2) }
Nd8 {+2.42/17 4s } 33. Qd3 {+2.22/11 0s } Qe7 {+2.78/18 3s
(Bb5) } 34. Bxf5 {+2.51/11 1s } Rxd4 {+3.07/18 2s }
35. Qxd4 {+2.60/11 0s } Kf8 {+3.11/19 3s (Qe1+) } 36. Re3
{+2.84/14 2s (Kf2) } Qf6 {+3.63/17 2s (Qf7) } 37. Qxf6+
{+3.45/13 2s } gxf6 {+3.55/15 0s } 38. Kf2 {+3.57/14 1s }
Kf7 {+3.71/19 3s (h5) } 39. Bc2 {+3.89/16 6s (Bc8) } d4
{+4.44/20 4s (Nb7) } 40. Ra3 {+4.13/15 2s } Ne6 {+4.56/20
3s } 41. Bxa4 {+4.14/15 0s } Bd5 {+5.01/21 3s } 42. c6
{+4.35/17 2s } f5 {+5.13/22 3s } 43. g4 {+4.50/16 4s (Rd3)
} Kf6 {+5.37/17 2s } 44. Rd3 {+4.82/17 3s (Bc2) } Be4
{+5.53/19 2s } 45. Rd2 {+5.09/16 3s } Bd5 {+5.65/20 3s
(Ke7) } 46. Kg3 {+5.09/15 5s (c7) } Be4 {+6.90/19 2s }
47. Bc2 {+6.18/14 1s } Bxc6 {+7.67/20 2s } 48. Bxf5
{+6.23/15 1s } Nd8 {+7.91/22 3s (Ng7) } 49. Rxd4 {+6.95/15
2s } Nf7 {+8.24/21 3s } 50. Bd7 {+7.51/15 4s (Rd2) } Bh1
{+9.81/23 3s } 51. Be8 {+9.38/17 4s } Ke7 {+12.92/23 3s
(Nh8) } 52. Bxf7 {+10.64/13 0s } Kxf7 {+11.31/16 0s (Bc6) }
53. f5 {+10.64/16 5s (Rd7+) } Ke7 {+22.06/22 2s (Kg7) }
54. Kf4 {+13.30/14 0s (Rd1) } Bg2 {+24.68/21 2s } 55. h4
{+13.30/14 0s (Rd3) } Bf1 {+89.39/21 2s } 56. g5 {+16.01/14
0s } Bb5 {+M37/22 4s (hxg5+) } 57. gxh6 {+M80/13 0s } Kf6
{+M9/18 4s (Kf7) } 58. Rd8 {+M11/17 15s } Ke7 {+M8/16 0s }
59. Rb8 {+M7/14 3s } Bc4 {+M7/10 0s (Kd6) } 60. h7 {+M7/14
3s } Kd6 {+M6/9 0s } 61. h8=Q {+M6/14 3s } Be6 {+M5/7 0s
(Kc5) } 62. Qd8+ {+M4/11 1s (fxe6) } Kc5 {+M3/7 0s }
63. Ke5 {+M3/12 3s (fxe6) } Bd7 {+M2/6 0s } 64. Qxd7
{+M2/12 3s (Qb6+) } Kc4 {+M1/6 0s } 65. Qd4# {+M1/11 2s} 1-0
Attachment: R4.pgn (71k)
- - By Uly (Gold) [mx] Date 2010-06-18 16:49
Something probably went wrong in this test, as Rybka performed as I'd expect Rybka 5 to perform, or something :yell: But it showcases some nice Rybka playing style, what I'd expect to see once Rybka plays with optimal time controls, the speculative moves seem to shine!

Positions used: Opening position, 1.e4, 1.d4, 1.Nf3, 1.c4, 1.e4 c5, 1.d4 Nf6 and 1.g4 (an idea inspired from Vempele's wacky test that seems to produce nice games).

Everything seemed fine, then Rybka went on a killing spree and won 5 games in a row. PGN attached at the end.

18.75% drawing rate.

Quad @2.33Ghz - 128MB RAM
Rybka: 4 Proccesses. 64bit. Ponder OFF. Normal Move: 55. KPK tablebases.
Opponent: Stockfish 1.7 4CPU 64bit.
1 Minutes/Game + 3 Seconds/Move
                                   Score     1234567890123456
--------------------------------------------------------------
1: Deep Rybka 4 NM55            11.5 / 16   1011===011011111
2: Stockfish 1.7 JA 4CPU 64bit   4.5 / 16   0100===100100000
--------------------------------------------------------------
16 games: +10 =3 -3
71.875%
+163 ELO


Sample positions:

3rr1k1/1p1q1pp1/pp5p/3nP2Q/P1b1R2N/1P6/4nPPP/1BB1R1K1 w - - 0 24


6k1/4q1p1/2b1r2p/2p2p2/1p1rPQPP/pP1P4/P3B1K1/3R1R2 w - - 0 115


1k1r1b1r/3q2p1/1p4n1/1NpppbPp/Q1n5/P1P1BN1P/3RPPB1/5RK1 b - - 0 22


2r5/ppr1qpk1/6p1/2b4p/2Rp1n1P/1QNP1B2/PP2PP2/5K1R b - - 0 30

Attachment: R4NM55.pgn (64k)
Parent - - By dr_zied_haddad (**) [fr] Date 2010-06-21 06:40
Nice test. Do you réel that adding table bases of 3 pièces helped rybka? I don't think so. I have to analyse these games More seriously before drawing définitive conclusions.
Anyway it's an
interesting way of testing.
Parent - By Uly (Gold) [mx] Date 2010-06-21 09:04
I'm pretty sure Stockfish had a problem at the end of the match and even Drunken Rybka would have scored that well :)

I need to rerun the tourney, still no time for that.
Up Topic Rybka Support & Discussion / Rybka Discussion / R4 on the Minimal Book Test

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill