Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / Lc0 at CCRL and GECT
- - By MrKris (***) Date 2019-04-15 04:13
Lc0 rating detail:
http://ccrl.chessdom.com/ccrl/404/cgi/engine_details.cgi?print=Details&each_game=1&eng=Lc0%200.21.1%20w41800%20GPU#Lc0_0_21_1_w41800_GPU
Lc0 0.21.1 w41800 GPU "Games played on a GTX 1050 GPU with a Leela ratio of 1.65 (after 120 seconds) to Stockfish 8 1CPU on the AMD Phenom II X6 1090T test machine. ..."
selected results:
Stockfish 10 64-bit: Lc0 47.0/100
Houdini 6 64-bit: Lc0 54.0/100
Komodo 11.3.1 64-bit: Lc0 56.5/100

selected 1-cpu from: 40/4 Complete List, http://ccrl.chessdom.com/ccrl/404/rating_list_all.html :
3496 Stockfish 10 64-bit
3474 Lc0 0.21.1 w41800 GPU
3446 Houdini 6 64-bit
3421 Houdini 5.01 64-bit
3421 Komodo 11.2 64-bit

New CCRL/GEGT rankings:
http://ccrl.chessdom.com/ccrl/4040/
                              Rating   +/-     wins           draws  games ; LOS to next
1  SugaR 1.6.1 64-bit 4CPU    3466  +29  −28  72.9%  −143.6  51.7%  385  ; 100.0%
2  Komodo 11.3.1 64-bit 4CPU  3399  +19  −19  67.6%  −120.9  50.1%  870 ; 58.7%
3  Houdini 6 64-bit 4CPU      3396  +15  −14  63.1%  −84.7  60.0%  1439 ; 100.0%
4  Fire 7.1 64-bit 4CPU       3320  +16  −16  57.2%  −44.5  58.5%  1215 ; 95.3%
5  Ethereal 11.00 64-bit 4CPU 3299  +20  −20  48.7%  +7.9  63.9%  714 ; 71.2%
6  Komodo 12.3 MCTS 64bit4CPU 3290  +25  −25  48.9%  +7.2  54.6%  458 ; 64.4%
7  Laser 1.7 64-bit 4CPU      3282  +34  −34  52.1%  −9.7  59.7%  238 ; 53.1%
8  Shredder 13 64-bit 4CPU    3281  +12  −12  48.6%  +8.4  62.1%  2125 ; 58.7%
9  Fizbo 2 64-bit 4CPU        3279  +15  −14  46.6%  +19.9  55.1%  1442 ; 57.1%
10  Andscacs 0.95 64-bit 4CPU 3275  +34  −34  45.6%  +25.6  63.8%  240 ...

http://ccrl.chessdom.com/ccrl/404/
1  Stockfish 10 64-bit 4CPU  3547  +15  −15  72.2%  −144.5  51.7%  1487 ; 99.8%
2  Houdini 6 64-bit 4CPU     3521  +10  −10  66.9%  −120.6  51.8%  3382 ; 96.9%
3  Komodo 11.2 64-bit 4CPU   3503  +16  −16  58.2%  −66.9  55.3%  1158 ; 98.9%
4  Lc0 0.21.1 w41800 GPU     3474  +17  −17  55.7%  −37.6  51.2%  1000 ; 100.0%
5  Fire 7.1 64-bit 4CPU      3423  +15  −15  53.2%  −24.2  55.1%  1347 ; 90.2%
6  Komodo 12.3 MCTS 64b 4CPU 3409  +16  −16  52.7%  −17.7  51.0%  1244 ...

http://www.cegt.net/5Plus3Rating/Purelist/rangliste.html | 5+3 ponder
*(32930) ?GPU                                   wins           draws
1  Stockfish 10.0 x64   3447   15   15   1400   77.4%   3215   39.8%
2  LCZero 0.20.2 Cuda*  3418   14   14   1400   74.3%   3217   42.5%
3  Houdini 6.0 x64      3356   13   13   1400   67.0%   3221   50.1%
4  Komodo 12.2.2 x64    3350   13   13   1400   66.3%   3222   47.4%
Parent - - By gsgs (***) Date 2019-04-15 07:34 Edited 2019-04-15 07:37 Upvotes 1
Wofgang Battig (CEGT) wrote :

In a direct duel LC0 is normally at eye level, but only there.
In our 5'+3" test with Ponder, LC0 (32930, the network from the last TCEC final)
won the match against SF10 with 53,5:46,5, thus approx. +24 ELO. After 1300 games
against 13 opponents Leela is 25 points behind. Effectively it should be rather more,
the Leela ratio is 1.4, so quite friendly

In our ranking with 40 moves / 3 minutes similar. Here the match against SF 10 was lost by a narrow margin.
(- 25 points), but the distance in the list is 44 points (1800 games).
Or against a Stockfish-Development version (from 20.3.): direct match +7 ELO,
Distance but probably around the 60-70 ELO, since the Stockfish-Dev. about 25-30 points
stronger than SF10.
We'll also be using this network with 3'+1" (with Ponder), 40/20 (without Ponder)
and possibly also 40/120 (without ponder). But this takes its time.
we will only test 2 to 3 networks per year to this extent,
Parent - - By MrKris (***) Date 2019-04-17 18:31
Very interesting, thank you!

Good to know the CEGT 'Leela Ratio', and I had not noticed Lc0 had underperformed Sf vs. the 2nd - 13th ranked engines.
Parent - - By gsgs (***) Date 2019-04-29 06:33 Edited 2019-04-29 07:25 Upvotes 1
2019/04/29
----------------------------
CEGT 40/20,LR=1.4 ?
3420,Stockfish 10.0 x64 1CPU
3389,LCZero 0.20.2 Cuda (32930)
3349,Houdini 6.0 x64 1CPU
3326,Komodo 12.2.2 x64 1CPU
------------------------------------------------
CEGT 40/4,LR=1.4 ?
3454,Stockfish 10.0 x64 1CPU
3418,LCZero 0.20.2 Cuda (32930)
3374,Houdini 6.0 x64 1CPU
3343,Komodo 12.3 x64 1CPU
--------------------------------------------
CEGT 5'+3'',LR=1.4 ?
3447,Stockfish 10.0 x64
3418,LCZero 0.20.2 Cuda (32930)
3356,Houdini 6.0 x64
3350,Komodo 12.2.2 x64
--------------------------------------------
CCRL 40/4 , LR=1.65
3546,Stockfish 10 64-bit
3520,Houdini 6 64-bit
3502,Komodo 11.2 64-bit
3473,Lc0 0.21.1 w41800 GPU
3451,Lc0 0.20.2 w32930 GPU

------------------------------------------------

> Games played on a GTX 1050 GPU with a Leela ratio of 1.65 (after 120 seconds)
> to Stockfish 8 1CPU on the AMD Phenom II X6 1090T test machine. For Lc0 fans -
> On a ratings list, Lc0 is NOT disadvantaged by playing 4CPU opponents.
> The Elo ratings system looks solely at the Elo rating of the opponent and
> the performance against it. The number of cores the opponent uses is irrelevant.
[?]

3479,3546,Stockfish 10 64-bit 4CPU,(+10−30=60)
3480,3520,Houdini 6 64-bit 4CPU,(+15−27=58)
3476,3496,Stockfish 10 64-bit 1CPU,(+18−24=58)
3493,3486,Komodo 12.3 64-bit 4CPU,(+24−22=54)
3472,3445,Houdini 6 64-bit,(+28−20=52)
3512,3422,Fire 7.1 64-bit 4CPU,(+42−15=43)
3461,3418,Komodo 11.3.1 64-bit,(+29−16=55)
3449,3389,Ethereal 11.25 64-bit 4CPU,(+32−14=54)
3456,3333,Fire 7.1 64-bit,(+50−13=37)
3461,3304,Xiphos 0.5 64-bit,(+53−6=41)
------------------------------------------------------------------
3474, against top5:3480 , against bottom 5:3468

24 fewer Elos than SF10 (1 core) , to which the LR of 1.65 refers (?) , 75 estimated Elos weaker than SF10 for LR=1

http://ccrl.chessdom.com/ccrl/404/cgi/engine_details.cgi?print=Details&each_game=1&eng=Lc0%200.21.1%20w41800%20GPU#Lc0_0_21_1_w41800_GPU

-------------------------------------
32930 was 32 Elos weaker than SF10 at CEGT (LR=1.4) , so ~ 66 Elos weaker with LR=1
Parent - By MrKris (***) Date 2019-04-30 20:54
Thank you for your interesting data collations!
They certainly clarify their current views on Lc0.

(Just my feeling of " ... The number of cores the opponent uses is irrelevant. ":
They seem to me to attempt to treat Engine A 4CPU and Engine A (1CPU) as different engines and rate them both with their one system/list.
I sometimes wonder if even they have enough time for enough 1-vs-4 combinations, but they are are much better than no attempt I think.)
_ _ _

TCEC 15 Premier, game 15, Stockfish 1/2-1/2 Lc0 (44)
https://cd.tcecbeta.club/archive.html?season=15&div=p&game=21

Maybe I am computing this incorrectly, and its not Sf 8:
First after book moves 9.Nf3 54 Mnps; 9...O-O 46 knps
54,000/46 = 1174 ; 875/1174 = Leela Ratio 0.75
Final moves 44.Qc1 66 Mnps; 44...Nd3 66 knps
66,000/66 = 1000 ; 875/1000 = Leela Ratio 0.88
_ _ _

Interestingly, your quote above:
" Wofgang Battig (CEGT) wrote :
In a direct duel LC0 is normally at eye level, but only there. ...
"
although only 42 games each at TCEC 15P Lc0-Sf 3.5-2.5, but final Sf 27.5, Lc0 25.5.
Up Topic The Rybka Lounge / Computer Chess / Lc0 at CCRL and GECT

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill