Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / New: Noomen Low draw rate testset v2
- - By Venator (Silver) Date 2021-06-02 05:58 Upvotes 2
Hi all,

Here is a new testset for engine-engine testing: Noomen_LowDrawRate_v2.  This is a selection of TCEC superfinal opening lines (S9 through S20) with the following criteria:

* Leela Chess Zero book exit of +0.35 or higher
* Good chance of a 1.5-0.5 result between the strongest engines

As the previous two superfinals (S19 and S20) showed, Leela book exits below +0.30 are a certain draw under TCEC superfinal conditions. To get interesting games it is necessary to give the engines something to play with. Hence the minimum book exit of +0.35.

Using this special book for engine-engine testing will lead to:

* Exciting games
* More decisive results 

Enjoy testing!
Attachment: Noomen_LowDrawRate_v2.pgn (27k)
Parent - By MrKris (****) Date 2021-06-03 06:35
Thank you very much!

I tried it here: http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=588492

G/1'+1" 4 threads/engine
Honey-13.2  110.5-113.5  ShashChess_17.1-C
49.3%, +46 -49 =129 (draws 57.6%, white +92 -3 =129)

The last finished event at CCC seemed to have a lot of draws, screenshot below.
https://www.chess.com/computer-chess-championship#event=belgian-stew
Random sample (done very quickly from the crosstable, very unreliable) of Lc0 book exits:
0.29  0.32  0.05  0.11  0.04  0.19  0.29  0.30  0.14  0.02
 
Parent - - By MrKris (****) Date 2021-06-04 06:27

> Leela book exits below +0.30 are a certain draw under TCEC superfinal conditions.


CCC just played an event with all games beginning 1.c3.

Lc0 there said 0.00.

In the 120 games among Lc0, Sf, SfClassic and Dragon (20 each) all were 19 or 20 draws!

https://www.chess.com/computer-chess-championship#event=saragossa
Parent - - By Venator (Silver) Date 2021-06-05 06:21
CCC just played an event with all games beginning 1.c3.

That is a recipe for lots of draws and uninteresting games.

Any chance for a LC0 vs Stockfish Dev match with my new Low Draw Rate book? I expect the win rate to be 40% or maybe even better!
Parent - - By MrKris (****) Date 2021-06-06 03:33

> Any chance for a LC0 vs Stockfish Dev match with my new Low Draw Rate book? I expect the win rate to be 40% or maybe even better!


That was here (5 hrs. before your question) http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=588497 :
"Stockfish and the forks are changing too much for my original idea for this.
Apex Elimination Matches
Nooman Low Draw Rate v2 http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=33922
112 openings, each colors reversed.
G/30"+0.5"  256Mh  Sy.6
R7 2700X 16ths | RTX 2060
Previous:
ShashChess_17.1-C   113.5-110.5  Honey-13.2  (draws 57.6%)
New:
Stockfish_21060401  125.5-98.5  Lc0-0.27-net-69272
Score of Stockfish_21060401 vs Lc0-0.27-net-69272: 64 - 37 - 123 [0.560]
...      Stockfish_21060401 playing White: 58 - 5 - 49  [0.737] 112
...      Stockfish_21060401 playing Black: 6 - 32 - 74  [0.384] 112
...      White vs Black: 90 - 11 - 123  [0.676] 224
Elo difference: 42.1 +/- 30.5, LOS: 99.6 %, DrawRatio: 54.9 %
224 of 224 games finished. ...{games at the link}"


The newer T60 nets don't seem too bad, though still not a match on my hardware (stock CPU & GPU, unadjusted for any LeelaRatio).

Your 40% range is here:
Nooman LDRv2: 1 Komodo13.3, 2 Ethereal13.00std, 3 Igel3.0.5
http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=33925

Thanks again for the book!!
Low drawrate and very interesting games, I tried to show a few in my posts.
(My Sf-Lc0/Sf-forks drawrates looks good to me: my Ko-Eth-Igel had a lot of same-color-wins.)
Parent - By Venator (Silver) Date 2021-06-06 18:13 Upvotes 1
Thanks again for the book!!

You're welcome! Glad you enjoy it. And thanks for the interesting matches.
Parent - - By The Truth (**) Date 2021-06-10 11:16
But how many openings can be won with both colors?

This result here kinda indicates that a lot are one sided wins:
...      Stockfish_21060401 playing White: 58 - 5 - 49  [0.737] 112
...      Stockfish_21060401 playing Black: 6 - 32 - 74  [0.384] 112


which is like having high draw rate to begin with since you're winning or losing at the start of the game. Openings with decisive results with both sides are the real gem with these test set (or a draw and win though less valuable) for engine matches. Because that means one engine is mistaken.
Parent - By MrKris (****) Date 2021-06-11 07:07

> This result here kinda indicates that a lot are one sided wins:
> ...      Stockfish_21060401 playing White: 58 - 5 - 49  [0.737] 112
> ...      Stockfish_21060401 playing Black: 6 - 32 - 74  [0.384] 112


First lets look at the event you quoted.
Then my 2nd reply for your "lot of one sided wins" and other points.

After 40+ years of home computers and that long computer chess on home computers (see Sargon, Chessmaster etc.) there is no software to help with analyzing same opening colors reversed game pairs.

So, manually analyzing the event you quoted:
Nooman Low Draw Rate v2, 112 openings, 224 games; the 112 game pairs were as follows:

57 or 51% : 1.5/2 or 0.5/2 - win/loss and draw

19 or 17% : 1.0/2 by win & loss

33 or 29% : 1.0/2 by both games draws

3 or 3%   : 2.0/2 or 0.0/2 same engine won/lost with both colors.


The 51% 1.5/2 or 0.5/2 is outstanding - an outstanding book!

Even more so considering the very low 17% 1.0/2 by win & loss and the very low 29% 1.0/2 by both draws - they combine to only 46%!

There are practical difficulties in reducing draws and simultaneously eliminating 1.0/2 win & loss.
Here the 17% 1.0/2 win & loss is only 1/3 of the large 51% portion of 1.5/2 or 0.5/2 win/loss and draw.
 
Scid vs PC crosstable, spaced added in the top lines, color-reversed game pair analysis added.
Here I 'showed my work' with the tallies. The original .pgn: http://rybkaforum.net/mwf/rybkaattach/97/588497/Sf-Lc_Nooman.pgn
                          Score                                                                                                                                                                                
 1: Stockfish_21060401  125.5 / 224   1= 10 == 10 1= 10 == 1= =0 == 1= 1= 0= 1= =1 1= 0= 1= =0 == == =0 == 1= == 10 =1 1= 10 1= == == 10 == 1= 1= 1= == 1= =1 == =0 == == 1= 10 10 1= == == 10 1= 1= 1= 1= == 10 == 1= 00 =1 00 == == == =1 1= 10 == 1= 10 1= 0= == == 10 10 1= =0 == 1= == == 10 =0 1= =0 10 == 1= =0 == 1= =0 1= 1= 10 == 10 1= 1= == 10 1= 11 1= == 1= == =0 =0 1=  (+64 -37 =123)
 2: Lc0-0.27-net-69272   98.5 / 224   0= 01 == 01 0= 01 == 0= =1 == 0= 0= 1= 0= =0 0= 1= 0= =1 == == =1 == 0= == 01 =0 0= 01 0= == == 01 == 0= 0= 0= == 0= =0 == =1 == == 0= 01 01 0= == == 01 0= 0= 0= 0= == 01 == 0= 11 =0 11 == == == =0 0= 01 == 0= 01 0= 1= == == 01 01 0= =1 == 0= == == 01 =1 0= =1 01 == 0= =1 == 0= =1 0= 0= 01 == 01 0= 0= == 01 0= 00 0= == 0= == =1 =1 0=  (+37 -64 =123)
112 col.rev. op. pairs:                a  s  d  s  a  s  d  a  a  d  a  a  a  a  a  a  a  a  a  d  d  a  d  a  d  s  a  a  s  a  d  d  s  d  a  a  a  d  a  a  d  a  d  d  a  s  s  a  d  d  s  a  a  a  a  d  s  d  a  b  a  b  d  d  d  a  a  s  d  a  s  a  a  d  d  s  s  a  a  d  a  d  d  s  a  a  a  s  d  a  a  d  a  a  a  a  s  d  s  a  a  d  s  a  b  a  d  a  d  a  a  a
      a=win|draw/loss|draw 57 = 51%    1           2        3  4     5  6  7  8  9 10 11 12 13       14    15       16 17    18             19 20 21    22 23    24       25       26          27 28 29 30          31    32             33 34       35    36 37             38 39    40          41 42 43       44 45    46 47 48 49          50 51       52    53    54    55 56 57 
      s=win|loss/loss|win  19 = 17%       1     2     3                                                           4        5           6                                      7  8           9                10                               11       12             13 14                   15          16                         17    18          19           
      d=draw|draw          33 = 29%
      b=win|win/loss|loss   3 =  3%                                                                                                                                                                                         1     2                                                                                                                                3
224 games, +90 -11 =123
Parent - By MrKris (****) Date 2021-06-12 15:21

> But how many openings can be won with both colors?


Here the point of Nooman's and Pohl's ( https://www.sp-cc.de/anti-draw-openings.htm ) 'low draw rate' genre:
Chess engines have only 3 choices for each opening:
1) near 0.00 evaluation = most games will be a draw;
2) white has an advantage;
3) black has an advantage.

If an engine "wins with both colors" that means the loser is losing with both colors meaning only 2 possibilities:
1) with an equal opening its losing both instead of drawing both, its author(s) need to fix both cases;
2) its losing with an disadvantage and with an advantage so its author(s) need to fix the with advantage case.
Engines are not humans.

CCC shows what happens with ordinary human 'chances for both sides' openings - top 3 only (Lc0/KD2/Sf in various place orders):
https://www.chess.com/computer-chess-championship#event=sicilian-najdorf-special
60 games, 55 draws
https://www.chess.com/computer-chess-championship#event=evans-gambit-madness
60 games, 53 draws
https://www.chess.com/computer-chess-championship#event=dutch-defense-special
88 games, 78 draws
https://www.chess.com/computer-chess-championship#event=kings-indian-defense-special
90 games, 83 draws
https://www.chess.com/computer-chess-championship#event=carokann-special
90 games, 81 draws

For example consider this opening from an online DB after 1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. Bg5 e6 7. f4 Qb6 8. Qd2 Qxb2
rnb1kb1r/1p3ppp/p2ppn2/6B1/3NPP2/2N5/PqPQ2PP/R3KB1R w KQkq - 0 9
2200+ players 9.Rb1:
white won +30.4% | draws 40.9% | black won 28.7%

The higher the human level the more "win with both colors" become mostly draws: 2600+ players 9.Rb1:
white won +26.7% | draws +58.3% | black won 15%

The latest Stockfish evaluation for 9.Rb1 is 0.00 so top engines with have a very high draw rate.
If there are more than a small percentage of wins the evaluation is wrong or the loser needs fixing.
The author(s) try to avoid wrong evaluations and losing.
FEN: rnb1kb1r/1p3ppp/p2ppn2/6B1/3NPP2/2N5/PqPQ2PP/R3KB1R w KQkq - 0 9
Stockfish_21060820: 2700X 14ths
Found 510 tablebases
NNUE evaluation using nn-7e66505906a6.nnue enabled
...
 39/31+  01:39   1,086,498k  10,898k  +0.08  Ra1-b1
 39/31+  01:40   1,094,500k  10,898k  +0.16  Ra1-b1
 39/52  01:46   1,164,071k  10,888k  +0.18  Ra1-b1 Qb2-a3 f4-f5 Nb8-c6 f5xe6 f7xe6 Nd4xc6 b7xc6 Bf1-e2 Bf8-e7 e4-e5 d6xe5 Bg5xf6 Be7xf6 Be2-h5+ g7-g6 O-O Qa3-c5+ Kg1-h1 O-O Rf1xf6 Rf8xf6 Nc3-e4 Qc5-e7 Ne4xf6+ Qe7xf6 Bh5-e2 Qf6-e7 Qd2-e3 Qe7-c7 h2-h4 c6-c5 Be2-f3 Ra8-a7 h4-h5 Bc8-b7 Bf3xb7 Ra7xb7 Rb1-d1 g6xh5 Qe3-e2 Rb7-b8 Qe2xa6 Kg8-f7 Rd1-f1+ Kf7-e7
 40/53+  02:13   1,452,215k  10,855k  +0.26  Ra1-b1
 40/62  03:13   2,091,675k  10,794k  +0.17  Ra1-b1 Qb2-a3 f4-f5 Nb8-c6 f5xe6 f7xe6 Nd4xc6 b7xc6 Bf1-e2 Bf8-e7 e4-e5 d6xe5 Bg5xf6 Be7xf6 Be2-h5+ g7-g6 Nc3-e4 O-O O-O Qa3-e7 Ne4xf6+ Rf8xf6 Rf1xf6 Qe7xf6 Bh5-g4 Qf6-e7 h2-h3 e5-e4 Qd2-c3 Qe7-a7+ Kg1-h1 Bc8-d7 Rb1-d1 Ra8-f8 Qc3-c4 Bd7-c8 Qc4xc6 e4-e3 Bg4xe6+ Bc8xe6 Qc6xe6+ Kg8-h8 Rd1-e1 Kh8-g7 Re1xe3 Rf8-f1+ Kh1-h2 Qa7-c7+ Qe6-e5+ Qc7xe5+ Re3xe5
 41/56+  03:41   2,384,340k  10,764k  +0.25  Ra1-b1
 41/56-  03:43   2,403,441k  10,761k  +0.09  Ra1-b1 Qb2-a3
 41/62+  03:53   2,506,376k  10,750k  +0.21  Ra1-b1
 41/62  03:54   2,515,447k  10,749k  +0.25  Ra1-b1 Qb2-a3 f4-f5 Nb8-c6 f5xe6 f7xe6 Nd4xc6 b7xc6 Bf1-e2 Bf8-e7 e4-e5 d6xe5 Bg5xf6 Be7xf6 Be2-h5+ Ke8-e7 Rb1-d1 Qa3-c5 Nc3-e4 Qc5-d4 Qd2-e2 Qd4-b6 Qe2-d3 Rh8-d8 Qd3-a3+ c6-c5 Rd1xd8 Ke7xd8 O-O Kd8-c7 Ne4xf6 g7xf6 Qa3-f3 Bc8-b7 Qf3xf6 Ra8-d8 Qf6xe5+ Qb6-d6 Rf1-f7+ Kc7-b8 Qe5xd6+ Rd8xd6 Rf7xh7 Rd6-d2 g2-g4 Bb7-e4 Rh7-e7 Rd2-g2+ Kg1-f1 Rg2xh2 Bh5-f7 Rh2xc2 Re7xe6 Be4-d3+ Kf1-g1 Rc2-c1+ Kg1-f2 c5-c4 g4-g5
 42/69-  04:02   2,606,141k  10,744k  +0.17  Ra1-b1 Qb2-a3
 42/69-  04:09   2,683,083k  10,744k  +0.09  Ra1-b1 Qb2-a3
 42/69  04:39   2,997,846k  10,729k   0.00  Ra1-b1 Qb2-a3 f4-f5 Nb8-c6 f5xe6 f7xe6 Nd4xc6 b7xc6 Bf1-e2 Bf8-e7 e4-e5 d6xe5 Bg5xf6 Be7xf6 Be2-h5+ Ke8-e7 Rb1-d1 Qa3-c5 Nc3-e4 Qc5-d4 Qd2-e2 Qd4-b6 Rh1-f1 Rh8-d8 Rd1xd8 Ke7xd8 Ne4xf6 g7xf6 Rf1xf6 Qb6-d4 Qe2-d3 Kd8-c7 Rf6-f7+ Bc8-d7 Rf7xh7 Qd4-g1+ Ke1-d2 Qg1xg2+ Bh5-e2 Qg2-g5+ Kd2-c3 Ra8-d8 Qd3-e4 Qg5-f4 Be2-f3 Kc7-d6 Qe4-d3+ Kd6-c7
 43/66+  04:40   3,010,948k  10,728k  +0.08  Ra1-b1
 43/66  04:45   3,061,497k  10,725k  +0.04  Ra1-b1 Qb2-a3 f4-f5 Nb8-c6 f5xe6 f7xe6 Nd4xc6 b7xc6 Bf1-e2 Bf8-e7 e4-e5 d6xe5 Bg5xf6 Be7xf6 Be2-h5+ Ke8-e7 Rb1-d1 Qa3-c5 Nc3-e4 Qc5-d4 Qd2-e2 Qd4-b6 Rh1-f1 Rh8-d8 Rd1xd8 Ke7xd8 Ne4xf6 g7xf6 Rf1xf6 Qb6-d4 Qe2-d3 Kd8-c7 Rf6-f7+ Bc8-d7 Bh5-g4 Kc7-d6 Rf7xh7 c6-c5 Bg4-f3 Bd7-c6 Rh7-h4 Qd4xd3 c2xd3 Bc6xf3 g2xf3 Ra8-b8 Rh4-a4 Rb8-b2 h2-h4 Kd6-d5 Ke1-f1 Rb2-h2
 44/63  06:43   4,319,667k  10,701k   0.00  Ra1-b1 Qb2-a3 f4-f5 Nb8-c6 f5xe6 f7xe6 Nd4xc6 b7xc6 Bf1-e2 Bf8-e7 e4-e5 d6xe5 Bg5xf6 Be7xf6 Be2-h5+ Ke8-e7 Rb1-d1 Qa3-c5 Nc3-e4 Qc5-d4 Qd2-e2 Qd4-b6 Rh1-f1 Rh8-d8 Rd1xd8 Ke7xd8 Ne4xf6 g7xf6 Rf1xf6 Qb6-d4 Qe2-d3 Kd8-c7 Rf6-f7+ Bc8-d7 Bh5-g4 Kc7-d6 Rf7xh7 c6-c5 Bg4-f3 Bd7-c6 Rh7-h4 Qd4xd3 c2xd3 Bc6xf3 g2xf3 Ra8-b8 a2-a4 Rb8-b2 Rh4-h8 Kd6-d5 Rh8-d8+ Kd5-c6 h2-h4 Rb2-h2 Rd8-e8 Kc6-d5 Re8-d8+
Up Topic The Rybka Lounge / Computer Chess / New: Noomen Low draw rate testset v2

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill