Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / CCRL Rating Lists and Statistics update (13th Jan 2007)
- By Graham Banks (*****) [nz] Date 2007-01-13 08:57
The January 13th update of the CCRL Rating Lists and Statistics is now available for viewing at the following link:
http://www.computerchess.org.uk/ccrl/4040/

The links to the various rating lists can be found just beneath the default Best Versions list.

Our standard testing is at 40 moves in 40 minutes repeating while our current blitz testing is at both 40 moves in 4 minutes repeating and 40 moves in 12 minutes repeating, all adjusted to the AMD64 X2 4600+ (2.4GHz).

We are now expressing our benchmark CPU as an AMD64 X2 4600+ (2.4GHz), instead of AMD64 3800+ (2.4GHz) single core.
This is so that we are consistent with other groups who quote an X2 as a benchmark.
The two CPUs are identical speed with the same 512k L2 cache. The AMD 3800+ single core was never a big seller and it has caused confusion with people mistaking it for the X2 3800+.

Our team of testers currently includes:
Graham Banks, Ray Banks, Sarah Bird, Shaun Brewer, Kirill Kryukov, Chinmay Kulkarni, Tom Logan, Andreas Schwartmann, Joshua Shriver, Charles Smith, George Speight, Chris Taylor, Martin Thoresen, Chuck Wilson and Gabor Szots.

We are always on the lookout for reliable testers to help, so please contact one of us if you're interested.

40/40 NEWS

1,172 games were added into our 40/40 database this week, making a grand total of 43,630 games.

Many engines on our list have few games and in many cases their ratings are likely to fluctuate (markedly for some) until a lot more games are played. Therefore no conclusions should be drawn about their strength yet.
To illustrate this point, when an engine has 200 games played, the error margin is still approximately +/-40 ELO, after 500 games +-25 ELO, after 1000 games +-17 ELO and even after 2000 games there is a +-13 ELO error margin!
This of course highlights the importance of looking at other rating lists that are also available in order to draw comparisons and get a more accurate overall picture.

Multi CPU Engines

Rybka 2.2 64 bit 4CPU predictably heads the rating list.

Zap!Chess Paderborn 64 bit 4CPU holds next spot way behind the Rybkas, but is superior to LoopMP 12.32 2CPU, Deep Shredder 10 64 bit 4CPU and Deep Junior 10 4CPU by a 30+ ELO margin.

Hiarcs 11 4CPU has made a promising start, but the number of games is too small to make any worthwhile statements about its strength.

Deep Fritz 10 4CPU has been a disappointment and is below the previously mentioned engines.

Glaurung 1.2.1 64 bit 2CPU is the best of the free engines although we've yet to test Scorpio 1.84.

Single CPU Engines

Rybka 2.2 64 bit tops this list.

The well established quartet of Shredder 10, Loop 10.32f, Toga II 1.2.1 and Hiarcs X50 Hypermodern continue to slug it out for second spot behind the Rybkas with Fritz 10 still in the hunt also.

Loop 12.32 and Hiarcs 11 are still struggling to prove their superiority over  Loop 10.32f and Hiarcs X50 Hypermodern respectively.
However, as noted, marked fluctuations in rating can occur during the early stages of testing, so it's worth keeping an eye on how things progress.

We can make a fairly certain statement that Spike 1.2 Turin, Fruit 2.2.1 and Junior 10 are all similar in strength to each other after a large number of games.

Further engines with a large number of games indicating similar strength to each other are Naum 2.0 32 bit and Ktulu 8.0.

After 314 games, the rating for Chess Tiger 2007 has fallen and it is currently just 50 ELO ahead of Chess Tiger 15.
This puts it at a similar strength to Zap!Chess Paderborn.
However intensive testing continues and the situation could yet change for the better.

Glaurung 1.2.1 is similar in strength to the top Chessmaster settings. This is also based on a large number of games.
CM9000 Enforcer and CM10th Paralyse continue to head off other CM settings in our list, but it's close!

In the 3rd CCRL Amateur Championship, Spike 1.2 Turin holds a narrow lead in Division 1 after 11 rounds.
Booot 4.12.1, Frenzee 3.0 and E.T Chess 18.11.05 have all scored 3/3 to take an early lead in Division 2.
The original intention of running just three divisions is likely to be expanded to include a Division 4 and possibly a Division 5 also, so there's plenty of action for fans of the amateur engines to look forward to!

We have a number of other tournaments in progress also involving amateur engines and most of these can also be followed in our public forum.
You should notice that the amateur engines lower in our list are steadily getting more games. Our aim is that every engine in our list gets 150-200 games at least.

BLITZ NEWS

Our 40/4 rating list has also been updated and can be accessed through the link given at the top of this post.
There are now 74,009 games in the 40/4 database.

Rybka 2.2 64 bit 2CPU is over 100 ELO ahead of the nearest challengers.

Zap!Chess Paderborn 64 bit 4CPU, Deep Fritz 10 4CPU, Hiarcs 11 4CPU and Deep Shredder 10 64 bit 4CPU are all fairly similar in strength.

LoopMP 12.32 2CPU and Deep Junior 10 4CPU are slightly further back.

Aside from Rybka 1.0 64 bit, Toga II 1.3 Beta1 32 bit and Glaurung 1.2.1 32 bit 4CPU are the strongest free engines 30+ ELO ahead of Spike 1.2 Turin.
Naum 2.0 64 bit in turn is a further 20 ELO back.

Chess Tiger 2007 is in 12th spot on the best versions list with a rating of 2796.
Shaun is keeping the Tiger results under different GUIs separate at this stage, but there aren't enough games to draw any conclusions.
It is interesting to note that its best performance currently is under Chessbase GUIs!

We will soon be putting more work into our 40/12 rating list which hasn't been updated for a long while.

FRC NEWS

Ray has reverted back to including only those engines that can play FRC through the Shredder Classic GUI.

At present there are no new engines for Ray to test, so the rating list hasn't changed from the last report!

Hiarcs 11 and Shredder 10 are pretty much equal with each other at the top of the ratings with Loop 10.32f further back in third place.

STATS/PRESENTATION NEWS

All games are available for download and the usual comprehensive stats tables are also available for your perusal through the link given at the top of this post.
Up Topic Rybka Support & Discussion / Rybka Discussion / CCRL Rating Lists and Statistics update (13th Jan 2007)

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill