Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / sp-cc.de experiment: Stockfish-Ko.Dragon with multi-PV's
- By MrKris (****) Date 2021-05-24 20:24 Edited 2021-05-24 23:38
https://www.sp-cc.de/experiments.htm
"
2021/05/24 Experimental RoundRobin tournament with 3 engines (Stockfish 13, KomodoDragon 1.0 and KomodoDragon 2.0), each with 4 different MultiPV-settings (1-4, were 1 is the normal, default playing mode)...
"

(Downloading the games from his site includes rating and stats. text files in a folder with the .pgn.)

Thank you very much!!  Excellent showing of the KD1 to KD2 multi-PV improvement, and of course the topical vs. Sf data!
 
- By MrKris (****) Date 2021-05-25 01:24
Here my feeble attempt to further analyze Stefan Pohl's experiment above.

Original sp-cc.de .pgn from the post above except:
all KD 1.0's games removed and all Sf-vs-Sf and KD2-vs-KD2 games removed,
leaving 400 games each with 100 games vs each other-engine's 4 PV's (1-4) :

    1600 games             Elo    +   -   Games   Score   Av.Op.  Draws

  1 Stockfish 13  pv=1  : 2526   22  21   400    72.0 %   2362   54.5 %
  2 Stockfish 13  pv=2  : 2457   21  21   400    63.4 %   2362   59.8 %
  3 KomDragon 2.0 pv=1  : 2435   19  19   400    49.8 %   2436   70.0 %
  4 Stockfish 13  pv=3  : 2415   20  20   400    57.5 %   2362   64.5 %
  5 KomDragon 2.0 pv=2  : 2370   20  21   400    40.5 %   2436   62.5 %
  6 Stockfish 13  pv=4  : 2350   21  21   400    48.2 %   2362   61.5 %
  7 KomDragon 2.0 pv=3  : 2336   23  23   400    35.9 %   2436   53.8 %
  8 KomDragon 2.0 pv=4  : 2311   22  23   400    32.8 %   2436   54.0 %
                                                  (50% would be 2400)


Above copy made with pv #'s removed (Text Editor) then rated: 
 
1 Stockfish 13  pv=  : 2436  1600 (+484,=961,-155), 60.3 % 
2 KomDragon 2.0 pv=  : 2364  1600 (+155,=961,-484), 39.7 %

Same 1600 games as above, Stockfish 13 +72 Elo (+/- 11).
Stockfish Wins=30.25% Losses=9.69% Draws=60.06%.


Summary of the below head-to-head data:
Overall each vs all 4 other-engine PV's (1-4) 100 games each:
  Sf 13 pv1 72.0%/400   KD2 pv1 49.8%/400
   Sf +22.2%
  Sf 13 pv2 63.4%/400   KD2 pv2 40.5%/400
   Sf +22.9%
  Sf 13 pv3 57.5%/400   KD2 pv3 35.9%/400
   Sf +21.6%
  Sf 13 pv4 48.2%/400   KD2 pv4 32.8%/400
   Sf +15.4%

Head-to-head:
Sf 13 pv1 61.5%/100 vs KD2 pv1
Sf 13 pv2 61.5%/100 vs KD2 pv2
Sf 13 pv3 62.5%/100 vs KD2 pv3
Sf 13 pv4 56.0%/100 vs KD2 pv4

 
Individual statistics: sorted by score %
--Note: Sf 13 pv=4 (48.2%) did only slightly worse than KD2 pv=1 (49.8%) :

1 Stockfish 13  pv=1 : 2526  400 (+179,=218,-  3), 72.0 % = 1st

KomDragon 2.0 pv=1   : 100 (+ 25,= 73,-  2), 61.5 %
KomDragon 2.0 pv=2   : 100 (+ 43,= 56,-  1), 71.0 %
KomDragon 2.0 pv=3   : 100 (+ 50,= 50,-  0), 75.0 %
KomDragon 2.0 pv=4   : 100 (+ 61,= 39,-  0), 80.5 %

2 Stockfish 13  pv=2 : 2457  400 (+134,=239,- 27), 63.4 % = 2nd

KomDragon 2.0 pv=1   : 100 (+ 18,= 74,-  8), 55.0 %
KomDragon 2.0 pv=2   : 100 (+ 28,= 67,-  5), 61.5 %
KomDragon 2.0 pv=3   : 100 (+ 40,= 51,-  9), 65.5 %
KomDragon 2.0 pv=4   : 100 (+ 48,= 47,-  5), 71.5 %

4 Stockfish 13  pv=3 : 2415  400 (+101,=258,- 41), 57.5 % = 3rd

KomDragon 2.0 pv=1   : 100 (+ 12,= 73,- 15), 48.5 %
KomDragon 2.0 pv=2   : 100 (+ 26,= 64,- 10), 58.0 %
KomDragon 2.0 pv=3   : 100 (+ 32,= 61,-  7), 62.5 %
KomDragon 2.0 pv=4   : 100 (+ 31,= 60,-  9), 61.0 %

3 KomDragon 2.0 pv=1 : 2435  400 (+ 59,=280,- 61), 49.8 % = 4th

Stockfish 13  pv=1   : 100 (+  2,= 73,- 25), 38.5 %
Stockfish 13  pv=2   : 100 (+  8,= 74,- 18), 45.0 %
Stockfish 13  pv=3   : 100 (+ 15,= 73,- 12), 51.5 %
Stockfish 13  pv=4   : 100 (+ 34,= 60,-  6), 64.0 %

6 Stockfish 13  pv=4 : 2350  400 (+ 70,=246,- 84), 48.2 % = 5th

KomDragon 2.0 pv=1   : 100 (+  6,= 60,- 34), 36.0 %
KomDragon 2.0 pv=2   : 100 (+ 16,= 63,- 21), 47.5 %
KomDragon 2.0 pv=3   : 100 (+ 27,= 53,- 20), 53.5 %
KomDragon 2.0 pv=4   : 100 (+ 21,= 70,-  9), 56.0 %


5 KomDragon 2.0 pv=2 : 2370  400 (+ 37,=250,-113), 40.5 % = 6th

Stockfish 13  pv=1   : 100 (+  1,= 56,- 43), 29.0 %
Stockfish 13  pv=2   : 100 (+  5,= 67,- 28), 38.5 %
Stockfish 13  pv=3   : 100 (+ 10,= 64,- 26), 42.0 %
Stockfish 13  pv=4   : 100 (+ 21,= 63,- 16), 52.5 %

7 KomDragon 2.0 pv=3 : 2336  400 (+ 36,=215,-149), 35.9 % = 7th

Stockfish 13  pv=1   : 100 (+  0,= 50,- 50), 25.0 %
Stockfish 13  pv=2   : 100 (+  9,= 51,- 40), 34.5 %
Stockfish 13  pv=3   : 100 (+  7,= 61,- 32), 37.5 %
Stockfish 13  pv=4   : 100 (+ 20,= 53,- 27), 46.5 %

8 KomDragon 2.0 pv=4 : 2311  400 (+ 23,=216,-161), 32.8 % = 8th

Stockfish 13  pv=1   : 100 (+  0,= 39,- 61), 19.5 %
Stockfish 13  pv=2   : 100 (+  5,= 47,- 48), 28.5 %
Stockfish 13  pv=3   : 100 (+  9,= 60,- 31), 39.0 %
Stockfish 13  pv=4   : 100 (+  9,= 70,- 21), 44.0 %
 
Up Topic The Rybka Lounge / Computer Chess / sp-cc.de experiment: Stockfish-Ko.Dragon with multi-PV's

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill