Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / CCRL 40/40, 40/4 and FRC lists updated (2nd March 2019)
- - By Graham Banks (*****) Date 2019-03-02 22:23
The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://computerchess.org.uk/ccrl/4040/ (40/40)
http://www.computerchess.org.uk/ccrl/404/ (40/4)
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

Please note that the three lists are often updated separately to each other. The FRC list is only updated when a new engine or engine version is being/has been tested.
Also, please note that the 40/40 and 40/4 lists are constructed from totally different databases and not comparable to each other.

Thanks to all of our currently active testers (Graham, Ray, Tirsa, Gabor and Sergio), plus to Chessdom.com for providing an online server to assist with resources.

40/40 testing this week that I'm aware of will include (with live broadcast port for TLCV noted where applicable):

73rd Amateur Series Division 2 Tournament (continuing 16001)
73rd Amateur Series Division 3 Tournament (continuing 16002)
Jumbo And The Monster Tournament (finishing Tuesday 16053)
Realm Of The Knight Tournament (starting Tuesday 16053)
Asymptote 0.4.1 64-bit Gauntlet (finishing Tuesday 16063)
Topple 0.3.5 64-bit Gauntlet (continuing 16064)
Jumbo 0.6.96 64-bit Gauntlet (continuing 16065)
CT800 1.34 64-bit Gauntlet (starting Tuesday 16063)
Winter 0.5 64-bit Gauntlet (continuing 16091)
Laser 1.7 64-bit Gauntlet (continuing 16092)
KnightX 1.93 64-bit Gauntlet (continuing 16093)
RubiChess 1.3 64-bit Gauntlet (continuing 16094)
The Red Jewel 4CPU Tournament (finishing Thursday 16083)
New 4CPU Tournament (starting Thursday 16083)
Tongues Of Fire 4CPU Tournament (finishing Wednesday 16084)
New 4CPU Tournament (starting Wednesday 16084)
Wasp 3.50 64-bit 4CPU Gauntlet (being run by Tirsa)
Various other 1CPU Gauntlets (being run by Tirsa)
Deuterium 2019.1.36.50 64-bit Gauntlet (being run by Gabor)
Various 1CPU Tournaments (being run by Sergio)

40/4 testing since the last update report has included (thanks to Gabor and Sergio):

Dimitri 4.77
Arasan 21.2 64-bit 4CPU
RubiChess 1.3 64-bit 4CPU
Francesca MAD 0.21 64-bit
Topple 0.3.5 64-bit
Counter 3.2 64-bit
SpaceDog 1.0 64-bit
RubiChess 1.3 64-bit
KnightX 1.93 64-bit
Wasp 3.50 64-bit 4CPU
Minic 0.40 64-bit 4CPU
Various Tournaments (run by Sergio)

FRC 40/4 testing to be completed (thanks to Ray):

Demolito 2018-10-29 64-bit
DanaSah 7.30
Parent - - By Ozymandias (****) Date 2019-03-03 07:33

>   Stockfish   9 64-bit 4CPU  3548  +12  −12  74.1%  −183.3  44.7%  2605
>   Stockfish 10 64-bit 4CPU  3547  +17  −17  73.2%  −153.4  51.3%  1163


Is this correct?
Parent - - By Graham Banks (*****) Date 2019-03-09 23:45

>>    Stockfish   9 64-bit 4CPU  3548  +12  −12  74.1%  −183.3  44.7%  2605
>>    Stockfish 10 64-bit 4CPU  3547  +17  −17  73.2%  −153.4  51.3%  1163
> Is this correct?


Seems so for now. Not sure on the latest list.
Parent - - By Ozymandias (****) Date 2019-03-10 14:58
This is the latest:

>Stockfish 10 64-bit 4CPU  3550  +17  −16  73.5%  −155.5  50.9%  1215
>Stockfish   9 64-bit 4CPU  3548  +12  −12  74.1%  −183.2  44.7%  2605


Even if SF ends up 19 Elo points ahead (the upper end of the error range) this doesn't look enough. Do we know if numbers coming in from different testers are congruent with one another?
Parent - - By gsgs (***) Date 2019-03-12 18:50
Elo(Stockfish 10) - Elo(Stockfish 9) =

+31,CCRL 40/40
+2,CCRL 40/4
+51,CEGT 40/4
+80,CEGT 5+3
+32,CEGT 40/20
+60,CEGT 3+1
+41,fastgm 60
+48,fastgm 600
+47,fastgm 3600
+56,spcc
+65,ipman2670
-----------------------
+47 average(11)
Parent - By gsgs (***) Date 2019-03-15 04:57 Edited 2019-03-15 07:30
Lc0 ratings : (compared to Stockfish 10)

Elo(number of games) , engine , list

-------------------------------------------------------------------------------
3463(678)  Stockfish 10 64-bit 4CPU , CCRL 40/40
3022(458) , Lc0 0.20.1 w36089 64-bit , CCRL 40/40
2842(24) , Lc0 0.20.2 w32930 64-bit , CCRL 40/40
2856(128) , Lc0 0.19.1 w11248 64-bit , CCRL 40/40
2939(142) , Lc0 0.20.1 w11258 64-bit , CCRL 40/40
-----------------------------------------------------------------------------------
3550(1215) , Stockfish 10 64-bit 4CPU , CCRL 40/4
2798(982) , Lc0 0.20.1 w36089 64-bit , CCRL 40/4

-------------------------------------------------------------
3555(1300) , Stockfish 10.0 x64 4CPU
3403(1700) , LCZero 0.20.2 Cuda (32930) , CEGT 4/40
2779(1200) , LCZero 0.9 (229) GTX1060 t2 , CEGT 4/40
---------------------------------------------------------------
3422(1861) , Stockfish 10.0 x64 1CPU
2868(1208) , Lc0 0.20.1 11258-112x9 1CPU , CEGT 40/20
-----------------------------------------------------
3448(17750) , Stockfish 10 64 , fastgm 60+0.6
2494(17750) , Lc0 CPU 11258-112x9-se , fastgm 60+0.6
------------------------------------------------------
3197(6000) , Stockfish 10 , fastgm 10min
2847(2700) , Lc0 CPU 11258-112x9-se , fastgm 10min:
-------------------------------------------------------------------
3529(5000) , Stockfish 190227 bmi2 , spcc
3419(1200) , LF 0.3 32930/Houdini 6 , spcc
3318(1200) , Lc0 0.20.2 N:32930 , spcc
-----------------------------------------------------------------
3450(1319) , Stockfish 10 64 BMI2 , spcc long
3452(304) , LF 0.3 N:32930 A:Houdini 6 , spcc long
3393(308) , Lc0 v0.20.1 N:32890 , spcc long
3385(246) , Lc0 v0.20.1 N:32567 , spcc long
3384(178) , Lc0 v0.21.0 N:41390 , spcc long
3321(283) , Lc0 v0.20.2 N:40695 , spcc long
--------------------------------------------------------------------
3612(36) , Stockfish 181224 , TCEC14P
3519(36) , LCZero v20rc2-32194 , TCEC14P
---------------------------------------------------------
3588(100) , Stockfish 190203 , TCEC14S
3585(100) , LCZero v20.2-32930 , TCEC14S
-------------------------------------------------------------------
3376(120) , Stockfish 190203 , TCEC14B1
3360(120) , LCZero v20.2-32930 , TCEC14B1
-------------------------------------------------------------------
3343(100) , Stockfish 190203 , TCEC14B2
3383(100) , LCZero v20.2-32930 , TCEC14B2
-------------------------------------------------------------------
3422(50) , Stockfish 10 x64 bmi2 , ipman 60+5
3443(300) , Lc0-cuda v21.0 - 32991 , ipman 60+5
---------------------------------------------------------------
3452(100) , Stockfish 10 x64 bmi2 , ipman 300+5
3459(100) , Lc0-cuda v20.0 - 32579 , ipman 300+5
-----------------------------------------------------------------
3655 , Stockfish , cccc
3629 , Lc0 , cccc
--------------------------------------------------
mtgostark , test30
3570 , Stockfish 10 4CPU
3448 , 35867
3448 , 36051
3443 , 32335
3423 , 32450
3418 , 11258
3411 , 11248
3400 , 32390
-----------------------------------------
mtgostark , test 40
3506 , LF 32585
3490 , Stockfish 8 4CPU
3436 , 32335
3435 , 32585
3433 , 41475
3432 , 41450
3430 , 32930
3429 , 41400
3428 , 41500
3414 , 41350
3411 , 11258
----------------------------------
lczero.org , 3000 games tests 40s-net :
41525-41425=+54 (2019,03,14)
41520-41490=+35 (2019/03/13)
41505-41475=+29 (2019/03/12)
41500-41400=+44 (2019/03/12)
41490-41460=+10 (2019/03/12)
41475-41375=+50 (2019/03/11)
41475-41445=+30 (2019/03/11)
41460-41430=+30 (2019/03/10)
41450-41350=+51 (2019/03/09)

========================================================

Elo(best Lc0 -version)-Elo(SF10)  GPU :
-152,-110,+2,-93,-3,-16,+40(hybrid),-26,-21,+7,-122,-164

CPU  :
-441,-752,-554,-994,-350,

Lc0 has better Elo in tournaments with long TC on good hardware
Parent - - By Lukas Cimiotti (Bronze) Date 2019-03-12 21:06 Edited 2019-03-12 21:11
Thank you for your valuable work.
Is there a chance you test lc0 using a graphics adapter?
I'm currently running tests: lc0 latest version on RTX 2060 vs. asmFish. First results show lc0 is a bit better.
The pc has a Core i9 7940X - asmFish gets 10 of 14 cores, lc0 gets 2. 2 cores are for system and GUI. HT is off.
Parent - - By Felix Kling (Gold) Date 2019-03-13 05:37
wow, so lc0 is already about equal to classical engines on top hardware?
Parent - By Lukas Cimiotti (Bronze) Date 2019-03-15 19:11
I'm awaiting götterdämmerung (yes, this is an English word, too :wink:) for conventional chess engines. I ran a test RTX 2060 vs. 10 cores of a 14 core i9. Stockfish on 10 cores was slightly better. Now I'm running a test RTX 2080 vs. 10 cores on that computer. It looks about equal. I built a computer using 2 x RTX 2080. Speedup looks good, but one of the GPUs died after a few hours of CUDA work :roll:
Currently performance per Watt still looks slightly better for conventional engines - but lc0 progress is stunning.
Parent - - By Graham Banks (*****) Date 2019-03-15 02:17
Ray has been testing Lc0 GPU for the 40/4 list.
Look for the next update.  :cool:
Parent - By Lukas Cimiotti (Bronze) Date 2019-03-15 19:32
The only GPUs that make sense are Nvidia RTX I guess. Using FP16 is the only way to get good performance. "Old" GTX graphics adapters don't make it.
Up Topic The Rybka Lounge / Computer Chess / CCRL 40/40, 40/4 and FRC lists updated (2nd March 2019)

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill