Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / Houdini 2.0s vs Houdini 2.0c
- - By RFK (Gold) Date 2012-06-28 21:07
Periodically, I will keep you informed of progress. 50,000 games
tbs off, ponder off. 128 hash. Albert Silvers LargSuite.
Parent - - By keoki010 (Silver) Date 2012-06-29 00:07
OK go for it Robert! Good luck...:cool:
Parent - - By RFK (Gold) Date 2012-06-29 03:24
It is just a matter of making sure the computer doesn't crash because of an electric outage. I'm not sure Arena would boot back up with the match able to resume. This is the time of year we get severe electrical storms up here.
Parent - - By saurus_ (**) Date 2012-06-29 06:56
Is this correct, you are running 50.000 games with a time control of 5min + 3sec? :eek:
Each game will take ~15 min (with endgame tables and resign if score is bigger than x)
Running one game at the time will give ~100 games a day or 500 day's running time.
Assuming you are running Houdini at one core and have a 12core machine you might be able to run 12 games in parallel.
This will still be 42 day's running time.

Wow. Tell me where I'm wrong.
Parent - By Barnard (Bronze) Date 2012-06-29 10:24
not,you arent wrong,you are right

he is running 50.000 games at 5'+3'',to prove at good time control Houdini2.0s against Houdini2.0c

he can also stop the match,at least say,20.000 games,but he is running a 50.000 games match

regards

Salvador
Parent - By RFK (Gold) Date 2012-06-29 14:10
Actually, I'd be happy to see 10,000 games come out of this without interuption. I'll let it run until something unforseen gets in the way.
Parent - - By keoki010 (Silver) Date 2012-06-29 15:30
yes you can resume where it crashed.  You can also stop the tour and restart it when you want just click tournament then click last game under the notation.
Parent - By RFK (Gold) Date 2012-06-29 16:58
This is the first engine match I've ever run with tb's disabled. I personally believe that tb's are very important to Houdini's performance. But, seeing that they are disabled across the board will be interesting.
Parent - By Gaмßito (****) Date 2012-06-29 12:06
Hi Robert,

I am interested in your ''Albert Silvers LargSuite'' file. I have one Silver test file but it only has 50 positions.

Could you please tell me where I can find the one you are using?

Regards,
Gaмßito.
Parent - - By RFK (Gold) Date 2012-07-06 07:16 Edited 2012-07-06 07:18
Currently its 92 degrees in my living room and having this thing running 24/7 is no joy. I may put the kibosh on it. I'll let it run another week. This is just not worth paying the high price of airconditioning, and resume the match after the heat lets up. One week more ought to bring it to 1000 games.
Parent - - By Barnard (Bronze) Date 2012-07-06 09:21
good partial results for Houdini2.0s :smile:
Parent - By RFK (Gold) Date 2012-07-06 15:32 Edited 2012-07-06 15:34
Oh, Blah! This will never end. Give me the best you have, and lets run a manly test and see if Houdart has his head up his ass or not! 100 games 2hr/40 tbs enbled. Best engine wins-period. Arena should allow me to post the games as we go.
Parent - - By keoki010 (Silver) Date 2012-07-06 13:54
Robert that's in line with mine. You have s vs c at 55.9% I have them at 52.9%  My games are 5'+3" Random openings.

H20sP vs H20cP                          :  85 (+ 27,= 36,- 22), 52.9 %
Parent - - By RFK (Gold) Date 2012-07-06 15:20 Edited 2012-07-06 15:26
If there is one test that I don't mind running my aircondition for it would be for a -

100 game 2hr/40 - tbs enabled, ponder off match between Sal's tuned version and Houdini 2.0c. On this 6 core. That would tell the story for me once and for all. Screw this blitz bullshit.

(I guess the truth of the matter is-I don't like blitz testing its endless)
Parent - - By jammy (***) Date 2012-07-06 15:36
I agree with that 100%. I can /could never work out what the hell is the point testing ANY engine only at blitz.
Parent - By RFK (Gold) Date 2012-07-06 15:42
I have " NEVER" believed in it,  and only did this for, Sal! :yell:
Parent - - By Uri Blass (*****) Date 2012-07-06 19:37
I disagree with that 100%

100 games are clearly not enough assuming that you see something like 30-20 and 50 draws.

It is clearly better to have many blitz games at time control 5+3 if you want to learn which version is better.

Note that ccrl shows very high correlation between 40/4 and 40/40 and it is very hard to find 2 programs when A is significantly stronger than B at 40/4 and B is significantly stronger than A at 40/40.
Parent - - By Uri Blass (*****) Date 2012-07-06 19:47
Note that after 500 games we have 167-108 for 2s without draws that seem significant to me
and I think that if after 1000 games we get even only something like 300-200 for 2s and 500 draws we can practically be sure that 2s is significantly better than 2c at 5+3 and stop the test.
Parent - By RFK (Gold) Date 2012-07-06 20:25 Edited 2012-07-06 20:28
Okay! I'll continue to 1000 games or a better. I can do that without putting myself into the poor house. Last year we averaged the high 70's the entire summer-this year we started off in the 90's.

To date-

-----------------------------Rating.dat:-----------------------------

7/6/2012 12:45:56 PM :

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini20s x64                 : 2420   22  22   525    55.6 %   2380   45.7 %
  2 Houdini_20_x64                 : 2380   22  22   525    44.4 %   2420   45.7 %

----------------------------Programs.dat:----------------------------
Individual statistics:

1 Houdini20s x64            : 2420  525 (+172,=240,-113), 55.6 %

Houdini_20_x64                : 525 (+172,=240,-113), 55.6 %

2 Houdini_20_x64            : 2380  525 (+113,=240,-172), 44.4 %

Houdini20s x64                : 525 (+113,=240,-172), 44.4 %

----------------------------General.dat:-----------------------------

Games        :    525 (finished)

White Wins   :    173 (33.0 %)
Black Wins   :    112 (21.3 %)
Draws        :    240 (45.7 %)
Unfinished   :      9

White Perf.  : 55.8 %
Black Perf.  : 44.2 %

ECO A =     66 Games (12.6 %)
ECO B =     12 Games ( 2.3 %)
ECO C =    124 Games (23.6 %)
ECO D =    216 Games (41.1 %)
ECO E =    107 Games (20.4 %)

----------------------------Cluster.dat:-----------------------------
Cluster No. 1:

Houdini_20_x64                 (1)
Houdini20s x64                 (1)

2 programs, 525 games

itoffset = 0.000000
Parent - By Banned for Life (Gold) Date 2012-07-06 20:37
You're a smart guy. Here's a question I've thought about over the years relating to evaluating relative performance of engines:

When you look at a pair of games with the same opening with reversed colors, you have 9 possibilities:

1-2) White or black win both games. This isn't very meaningful and most likely means the opening wasn't balanced.
3) Two draws. Also not very meaning. May or may not mean the opening was overly drawish.
4-7) One engine wins one game and draws the other. Obviously it is more likely the stronger engine got the win.
8-9) One engine won with both colors. This clearly indicates superiority of the winning engine in that position.

Traditionally, all of these cases are lumped together into one number. I don't see this as an optimal method for determination of relative engine strength. I'm pretty sure getting rid of cases 1&2 would improve the results (although they even out, they drag the overall rating toward equality). On the other hand, cases 8 & 9, which are weighted twice as highly as cases 4 through 7 are probably under weighted.

I see these considerations as much more important in an engine match than in a human match, because of the much more consistent play in an engine match.

I'd be interested in hearing your thoughts...
Parent - By RFK (Gold) Date 2012-07-06 15:50
I don't think this is the way to go. I thought I could stand doing this -and I cannot. It is too hot and this is endlessly tedious and lacking in total interest. It is very very boring. Oh God! If I'm going to sweat my balls off watching something it will be a real match.
Parent - - By RFK (Gold) Date 2012-07-11 15:35 Edited 2012-07-11 15:38
The next to last update on a 1000 games.

-----------------------------Rating.dat:-----------------------------

7/11/2012 11:34:46 AM :

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini20s x64                 : 2422   18  18   808    56.2 %   2378   46.3 %
  2 Houdini_20_x64                 : 2378   18  18   808    43.8 %   2422   46.3 %

----------------------------Programs.dat:----------------------------
Individual statistics:

1 Houdini20s x64            : 2422  808 (+267,=374,-167), 56.2 %

Houdini_20_x64                : 808 (+267,=374,-167), 56.2 %

2 Houdini_20_x64            : 2378  808 (+167,=374,-267), 43.8 %

Houdini20s x64                : 808 (+167,=374,-267), 43.8 %

----------------------------General.dat:-----------------------------

Games        :    808 (finished)

White Wins   :    261 (32.3 %)
Black Wins   :    173 (21.4 %)
Draws        :    374 (46.3 %)
Unfinished   :     26

White Perf.  : 55.4 %
Black Perf.  : 44.6 %

ECO A =    103 Games (12.7 %)
ECO B =     19 Games ( 2.4 %)
ECO C =    186 Games (23.0 %)
ECO D =    331 Games (41.0 %)
ECO E =    169 Games (20.9 %)

----------------------------Cluster.dat:-----------------------------
Cluster No. 1:

Houdini_20_x64                 (1)
Houdini20s x64                 (1)

2 programs, 808 games

itoffset = 0.000000
Parent - - By RFK (Gold) Date 2012-07-11 15:44
What's with the "unfinished" games?
Parent - - By keoki010 (Silver) Date 2012-07-11 17:31
Look in the pgn and see if you have time forfeits.  Houdini sometimes doesn't get started in Arena for some strange reason.
Parent - - By mare1975 (**) Date 2012-07-11 18:26
Arena is not good if You dont tune it. For example: If You dont  set to restart it after each game that will happen, also sometimes engine won t perform 100% ( only if not set to restart after each game)

After I tuned Arena, never happen that "unfinished games"... Also; only 2 times Houdini20z  stopped working ( of all engines and games )... Made about 50000 games with Arena 3.0....

If You running ultra fast ( bullet ) games You need to go to "Appearance settings-->Pieces-->and change sliding pieces to " very fast " ( if don t sometimes slows down game and GUI go "ballistic" )

Btw, I think Arena is the best GUI for tournaments...

Regards
Parent - - By keoki010 (Silver) Date 2012-07-12 20:17
No it still happens, I have Arena set to restart each engine after every game and restart the GUI itself after every 20 games. Found somewhat the same problem in Aquarium for Houdini and Komodo. If you leave the engine loaded in memory sometimes it will hang.  Difference is Arena handles it but Aquarium hangs up and freezes!
Parent - - By mare1975 (**) Date 2012-07-12 20:33
Don t  know for Aquarium,but You need to setup GUI  (Arena)  to restart after every game... If not, sometimes wont load hash as You setup...

Or delete Arena and download another one....

Regards...
Parent - - By RFK (Gold) Date 2012-07-13 00:22
I don't think you need to refresh after ever game.
Parent - - By mare1975 (**) Date 2012-07-13 05:26
Well, as I said; If no restart after each game, sometimes wont load hash as it should...
Parent - - By RFK (Gold) Date 2012-07-24 13:21 Edited 2012-07-24 13:24

> Well, as I said; If no restart after each game, sometimes wont load hash as it should...


When running a Round Robin with a number of engines, for certain engines it  seems necessary to restart the GUI  more frequently then after the  recommended 20th game  for the hash to load. I noted this with Critter.  I tried very 4 games and that seems to work fine.

If it hadn't been for your note-I wouldn't have been keeping my eyes open for it. :wink:
Parent - - By mare1975 (**) Date 2012-07-25 13:03
Well, nobody is perfect    :lol:

Thats  why are we all here; to help each other...

Regards...
Parent - By RFK (Gold) Date 2012-07-25 14:58
:lol: Has nothing to do with being perfect! Unless you're an egotist.:wink:
Parent - By RFK (Gold) Date 2012-07-13 00:29 Edited 2012-07-13 00:32
However, I'll do an experiment and look at the next 100 games 900 and see if there are anymore unfinishted games since the last time I checked.

[edit] I'm wondering if those games had something to do with when we had severe thunderstorms and I shut down the computer and restarted after the storm pasted.
Parent - By RFK (Gold) Date 2012-07-11 19:10
Apparently, in " All games" there are 870 games the ones marked "line" show failure to switch sides and were stopped after 15 moves.
Parent - - By RFK (Gold) Date 2012-07-13 03:21
-----------------------------Rating.dat:-----------------------------

7/12/2012 11:19:34 PM :

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini20s x64                 : 3219   17  17   900    55.6 %   3181   46.7 %
  2 Houdini_20_x64                 : 3181   17  17   900    44.4 %   3219   46.7 %

----------------------------Programs.dat:----------------------------
Individual statistics:

1 Houdini20s x64            : 3219  900 (+290,=420,-190), 55.6 %

Houdini_20_x64                : 900 (+290,=420,-190), 55.6 %

2 Houdini_20_x64            : 3181  900 (+190,=420,-290), 44.4 %

Houdini20s x64                : 900 (+190,=420,-290), 44.4 %

----------------------------General.dat:-----------------------------

Games        :    900 (finished)

White Wins   :    288 (32.0 %)
Black Wins   :    192 (21.3 %)
Draws        :    420 (46.7 %)
Unfinished   :     30

White Perf.  : 55.3 %
Black Perf.  : 44.7 %

ECO A =    119 Games (13.2 %)
ECO B =     21 Games ( 2.3 %)
ECO C =    208 Games (23.1 %)
ECO D =    363 Games (40.3 %)
ECO E =    189 Games (21.0 %)

----------------------------Cluster.dat:-----------------------------
Cluster No. 1:

Houdini_20_x64                 (1)
Houdini20s x64                 (1)

2 programs, 900 games

itoffset = 0.000000
Parent - - By Barnard (Bronze) Date 2012-07-13 13:31
good results :smile:
Parent - - By RFK (Gold) Date 2012-07-13 18:17
This might be your average -but it is hard to tell -over time there could be an incremental gain in wins that could bring the average up even a little more.
Parent - By Barnard (Bronze) Date 2012-07-18 02:13
almost identical results than George;that means the elo gain is not influenced by my slow machine
Parent - - By keoki010 (Silver) Date 2012-07-13 13:36
Are you still using SilverSuite for your openings? That 21 games for ECO B doesn't look right.
Parent - By RFK (Gold) Date 2012-07-13 15:49
I've been using Silver largeSuite  from the beginning.
Parent - - By RFK (Gold) Date 2012-07-14 22:15
-----------------------------Rating.dat:-----------------------------

7/14/2012 6:14:10 PM :

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini20s x64                 : 3219   16  16  1001    55.4 %   3181   47.3 %
  2 Houdini_20_x64                 : 3181   16  16  1001    44.6 %   3219   47.3 %

----------------------------Programs.dat:----------------------------
Individual statistics:

1 Houdini20s x64            : 3219  1001 (+318,=473,-210), 55.4 %

Houdini_20_x64                : 1001 (+318,=473,-210), 55.4 %

2 Houdini_20_x64            : 3181  1001 (+210,=473,-318), 44.6 %

Houdini20s x64                : 1001 (+210,=473,-318), 44.6 %

----------------------------General.dat:-----------------------------

Games        :   1001 (finished)

White Wins   :    320 (32.0 %)
Black Wins   :    208 (20.8 %)
Draws        :    473 (47.3 %)
Unfinished   :     31

White Perf.  : 55.6 %
Black Perf.  : 44.4 %

ECO A =    133 Games (13.3 %)
ECO B =     21 Games ( 2.1 %)
ECO C =    237 Games (23.7 %)
ECO D =    402 Games (40.2 %)
ECO E =    208 Games (20.8 %)

----------------------------Cluster.dat:-----------------------------
Cluster No. 1:

Houdini_20_x64                 (1)
Houdini20s x64                 (1)

2 programs, 1001 games

itoffset = -0.000000
Parent - By Barnard (Bronze) Date 2012-07-18 02:12
i think and i know that the s2 version will perform better than the s version :smile:
Up Topic The Rybka Lounge / Computer Chess / Houdini 2.0s vs Houdini 2.0c

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill