Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / Critter 1.6 running for the IPON
- - By Ingo (***) Date 2012-06-15 09:35
As usual: http://www.inwoba.de

Right now I can only spare a few cores and have to interrupt for some hours from time to time until Sunday (CET). After that the test will run with the familar speed.

Have fun
Ingo
Parent - - By Ray (****) Date 2012-06-15 12:43
Will be interesting to see the results. I think Richard has mainly concentrated on analysis improvements. I think no more than +20 ELO over Critter 1.4, and probably much less than that, < 10. But, I would be very happy to be proved wrong.
Parent - - By Chaotic Chess (****) Date 2012-06-15 13:31
It will pass Houdini 1.5a
Parent - - By Carl Bicknell (*****) Date 2012-06-15 14:42
I very much doubt it.
Parent - - By turbojuice1122 (Gold) Date 2012-06-15 15:10
It is early, with just over 10% of the games complete, but it is currently ahead of Houdini 2.0c.
Parent - - By Kappatoo (*****) Date 2012-06-15 15:13
After how many games? It's not what I see after 274/2550 games.
Parent - By turbojuice1122 (Gold) Date 2012-06-15 18:49
I have forgotten the exact number, but in any case, the point is moot now--it is definitely below Houdini 2.0c now.
Parent - - By Carl Bicknell (*****) Date 2012-06-15 16:59

> It is early, with just over 10% of the games complete, but it is currently ahead of Houdini 2.0c.


on the list I'm looking at it's 40-50 elo below Houdini 2. Are there two lists?
Parent - - By Eelco de Groot (***) Date 2012-06-15 17:17
Yes actually there are :smile: the site of Clemens Keck, which is well in agreement with Critter 1.6's provisional IPON rating. See http://www.clemens-keck.de/livegames/Critter_1.6.html Critter 1.4 is rated 2960 on that site, so now a few elo better. But I'm envious of all the new learning capabilities in Critter 1.6! It was not really expected either because Richard said he would wait for Komodo MP. But this is a tough act to follow! Not completely compatible with the Chessbase GUIs yet though.


Critter_1.6

Critter 1.6 64-bit x1 - Houdini 1.5a x64 x1 (3016)  10.5 - 10.5  50.00%  Perf=3016
Critter 1.6 64-bit x1 - Houdini 2.0c x64 x1 (3009)  8.5 - 11.5  42.50%  Perf=2957
Critter 1.6 64-bit x1 - Komodo64 SSE Version 4 (2972)  10.0 - 10.0  50.00%  Perf=2972
Critter 1.6 64-bit x1 - Deep Rybka 4.1 SSE42 x64 x1 (2953)  12.0 - 8.0  60.00%  Perf=3023
Critter 1.6 64-bit x1 - Stockfish 2.2.2 JA SSE42 (2945)  9.5 - 10.5  47.50%  Perf=2928
Critter 1.6 64-bit x1 - Fire 2.1 xTreme x64 x1 (2944)  13.0 - 7.0  65.00%  Perf=3051
Critter 1.6 64-bit x1 - Loop 2010 x64 (2849)  12.5 - 7.5  62.50%  Perf=2937
Critter 1.6 64-bit x1 - Naum 4.2 (2837)  15.0 - 5.0  75.00%  Perf=3027
Critter 1.6 64-bit x1 - Deep Sjeng c't 2010 (2803)  15.0 - 5.0  75.00%  Perf=2993
Critter 1.6 64-bit x1 - Deep Shredder 12 UCI x1 (2800)  14.0 - 6.0  70.00%  Perf=2947
Critter 1.6 64-bit x1 - Spike 1.4 T1 (2786)  17.0 - 3.0  85.00%  Perf=3087
Critter 1.6 64-bit x1 - spark-1.0 T1 (2783)  16.5 - 3.5  82.50%  Perf=3052
Critter 1.6 64-bit x1 - HIARCS 13.2 MP T1 (2772)  15.5 - 4.5  77.50%  Perf=2986
Critter 1.6 64-bit x1 - Protector 1.4.0 x64 JA (2763)  15.5 - 4.5  77.50%  Perf=2977
Critter 1.6 64-bit x1 - Deep Junior 13 x1 (2763)  12.5 - 5.5  69.44%  Perf=2905
Critter 1.6 64-bit x1 - Zappa Mexico II x1 (2753)  13.5 - 6.5  67.50%  Perf=2879
Critter 1.6 64-bit x1 - Umko 1.2 x64 x1 (2683)  18.0 - 2.0  90.00%  Perf=3064
Critter 1.6 64-bit x1 - Jonny 4.00 (2638)  17.5 - 2.5  87.50%  Perf=2976

  246.0 - 113.0  68.52%  Perf=2973

359 von 1800 Partien gespielt
Spielstufe: 5 Minuten/Partie + 3 Sekunden/Zug
Parent - By Barnard (Bronze) Date 2012-06-19 18:49
At the time they are "developing" the MP version,if Richard must wait for Komodo MP version,surely he will have his retirement before :smile:
Parent - - By Uly (Gold) Date 2012-06-16 02:25 Edited 2012-06-16 08:57
[Update] - This post is null and void. With LP disabled in the OS, the behavior of Critter is correct.

Please note that Critter now forces the system to use Large Pages, this may produce messing up of other engine's hashes and raise Critter's elo artificially.

In my system, any engine's hash loaded before Critter will be completely obliterated when Critter is loaded, and will refuse to even respond to the GUI.

Since the IPON doesn't show its games, we don't know if this is being done to the engines at the start of games and its results could be invalid.

See Harvey Williamson's post on the Critter 1.6 thread, if two engines ask for Large Pages only one gets them and the other gets handicapped, if both Critter 1.6 and Houdini 2.0 are forcing Large Pages for them and only one of them is getting it, the results would be invalid.
Parent - - By Ray (****) Date 2012-06-16 06:05

> Please note that Critter now forces the system to use Large Pages, this may produce messing up of other engine's hashes and raise Critter's elo artificially.


It can't force large pages, if large page permissions are disabled in Windows. It is impossible. No problems at all here, running very smoothly as with other engines.
Large pages aren't worth the hassle, just disable them in windows and all engines will be fine.
Parent - - By Ingo (***) Date 2012-06-16 07:12 Edited 2012-06-16 07:15

>It can't force large pages, if large page permissions are disabled in Windows. It is impossible. No problems at all here, running very >smoothly as with other engines.


Independent of the engine Large Pages are impossible for an automated engine test. All engines will fail with a constant load and unload. I tried in the past but it is just a matter of time until all the games are frozen or (even worse) are giving VERY strange results. This is the case even if just ONE Engine is using LP (Thats what I tried, two engines using LP in engine testing is a terrible thought). Because of this problems LP is off on my Testing systems. (Actually I think it is windows which is messing up the LP not the engine (at least on my WXP64 - and if I have doubts it is better on W7)

>Large pages aren't worth the hassle, just disable them in windows and all engines will be fine.


LP is fine for an Analysis or a single game (eg on a Server) but not for some serious testing. "Not worth ..." is relative then. It depends what you want to do ... (and if 10, mabye 15% speed increase - in other words around 10 Elo are worth the risk of a crashing system is a personal taste as well)

Bye
Ingo
Parent - By Ray (****) Date 2012-06-16 07:32
Yes we are basically in agreement. Useful for analysis perhaps where with a large hash size you can get 10-15% speedup, assuming you have recently re-booted and have enough unfragmented memory. But for engine testing, should always be disabled in Windows.
Parent - - By Uly (Gold) Date 2012-06-16 08:56
Yes Ray. Sorry about that, I disabled large pages and was still suffering that behavior, it turned out I only needed to restart Windows for the changes to take effect. My above post is null and void for that reason.
Parent - By Ray (****) Date 2012-06-16 21:55
OK, happy you got your issues sorted :-)
Parent - - By magnumpi (**) Date 2012-06-17 19:05
Hello Ingo,
  I always wondered why you don't publish the games.
Am I right if I say that you don't publish the games so that progammers can't tune their programs to the positions you use?

Thanks a lot for your tests, it's always nice to follow them live.
Parent - By Ingo (***) Date 2012-06-18 07:25

>I always wondered why you don't publish the games.
>Am I right if I say that you don't publish the games so that progammers can't tune their programs to the positions you use?


Right, and Strelka 5.5 with its included starting positions proved me to be correct.

There are a few others mostly related to some beta testing.

BYe
Ingo
Parent - - By Ingo (***) Date 2012-06-18 07:23
The full run is finished.

Resutls at http://www.inwoba.de

Bye
Ingo
Parent - - By siah (***) Date 2012-06-18 14:50
Your website is down here. It only can be viewed by proxy servers. Why is that?
Parent - - By Ingo (***) Date 2012-06-18 18:12
Hi siah,

Actually the very first one to ask is your repressive goverment ... (but don't do it, you know why!)

The site is up and running, why is it blocked on your site is beyond anything I and every sane people can understand!

Good luck
Ingo
Parent - - By siah (***) Date 2012-06-19 06:24 Edited 2012-06-19 07:41
It is not blocked. I mean the block page doesn't appear.
Parent - - By Barnard (Bronze) Date 2012-06-19 06:42
siah,if you cant acces with the normal way,and you need proxy servers to acces 'x' site,that site is blocked in your country,as easy as that

if you cant acces the normal way AND also cant acces with proxies,maybe is problem of the 'x' site,BUT if you cant acces AND can acces with a proxy,your country is blocking the site

even msn or a lot of programs to run video-conferences,are blocked...China,is China :smile:
Parent - - By siah (***) Date 2012-06-19 07:56
Why and?
Parent - - By Barnard (Bronze) Date 2012-06-19 17:58
because if you cant acces without proxies and you can acces with them,it isnt problem of the site,is problem of your country/internet provider that has blocked the acces siah
Parent - Date 2012-06-19 18:28
Parent - - By Razor (****) Date 2012-06-20 19:52
Hi Ingo,

Thanks for this - very interesting.

Do you have any plans to run H2z against your current set - I believe the person with the 'Barnard' username has created a version of H2c that has a claimed 13 ELO improvement - the IPON test framework would be an excellent way to confirm this.

What do you think?
Parent - By Ingo (***) Date 2012-06-21 06:07
Hi Razor,

>Do you have any plans to run H2z against your current set - I believe the person with the 'Barnard' username has created a version of >H2c that has a claimed 13 ELO improvement - the IPON test framework would be an excellent way to confirm this.


There once was a R4 setting which was unbelievable good, so I decided to test it. It was even better in my test in the full list. Unfortuntely R4 had many more games and when I looked at identical opponents nothing was left. I test a setting from time to time for years and never was something there that was  that much better that it could be proved beyond any doubt. That does not mean that there is no chance that something is better, but as I would have to compare this 2.0Z setting with the list on my start page and the error bar there is about 10 Elo plus AND minus it is impossible to prove 13 Elo ... therefore I will not test it. Another argument is, that 13 Elo are impossible to distinguis in individual games and especially impossible to see for humans. Really I dont see any need to test that.

But I agree on the excelency of the IPON framework :-)

Regards
Ingo
Up Topic The Rybka Lounge / Computer Chess / Critter 1.6 running for the IPON

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill