If you let me choose the time control I would take the CCRL time control: 40/40. Use 'ponder ON' please. The opponents can be Houdini 2.0c and Komodo 5.
Regarding Komodo 5, I think it's a ''bone hard to crack'' in this time controls, so it would be specially interesting to see also some tests against it.
About the opening suite I let you to choose whatever public opening suite you want. Of course it must be ''public'' because we want to see the games.
Using 40/40, and assuming 80 moves average, the average game would take about 160 minutes. That means we can play about 400 to 450 games.
If we play a match against Houdini 2.0c and Komodo 5 there can be 200 games per opponent. This would require an opening suite with 100 positions.
Alternatively we can play 60/40 to compensate for the relatively slow server (1 MN/sec) and play 120 or 150 games per match (60 or 75 opening positions).
Or keep 40/40 and throw in Stockfish 2.3 to play 3 matches of 120 or 150 games.
I really prefer that you select the opening positions (Silver, Noomen, ... or any combination you want) to avoid any suggestions of bias.
Also, maybe you can add Stockfish 2.3.1 as third opponent. What do you think about playing two matches of 120 games each one at '40/40' time control, one against Houdini 2.0c and the other against Stockfish 2.3.1, but in the next match against Komodo 5 use a slower time control like: 120'+30"?
If it's too slow, we can reduce the time a bit, and play with the previous Fide time control: 90+30, that could be very good too.
Here is my opening suite: http://www.mediafire.com/?5a396plaff6l1t0 - It has 240 positions, but the first 60 corresponds to the Noomen suite (years 2006 & 2008).
I suggest to use 90'+30" for all 3 matches. This will be slightly longer than initially planned, but why not...
So here's the summary of the Houdini 3 long TC test:
- 120 games against Houdini 2.0c, Stockfish 2.3.1, Komodo 5
- 90+30 TC, 512 MB hash, 60 positions from Noomen 2006/2008 test suites played from both sides.
- No table bases, resign after 3 successive scores larger than 5.00, draw after move 120 if both evaluations below 0.10.
- Hardware: AMD Opteron 6274 Server (SSE4) producing approx. 1 MN/sec per core with Houdini.
- The Houdini 3 version that is played can be considered a Houdini 3 "Beta": close to, but not exactly the final release.
- The error margin on the individual 120-game match results will be about ± 45 Elo.
Do you agree?
How many cores will Houdini and Stockfish run on?
To reply to the other questions:
- It's a single-core match, the goal is not to test SMP performance (would be rather difficult with Komodo ), but to get the best chess games within the time limits. That means single-core matches.
- No live broadcast: the matches are run by cutechess-cli on a server, you wouldn't see anything other than a black DOS box with the current match score. Note that anytime 31 simultaneous single-core games are played. After starting the match nothing happens for about 2 or 3 hours. Then slowly game after game will finish, at a rate of 8 to 10 per hour. The full 120-game run should take about 18 hours, the match PGN is gradually written to disk when games finish.
It has 32 Bulldozer modules with 64 cores, but I consider this a 32-core server with AMD hyper-threading and only use 1 core of each Bulldozer module.
This has been my main server for the Houdini 3 development, producing about 100,000 fast games per day.
>Quad AMD Opteron 6274 box running Windows 2008 R2.
>It has 32 Bulldozer modules with 64 cores, but I consider this a 32-core server with AMD hyper-threading and only use 1 core of each Bulldozer module.
>This has been my main server for the Houdini 3 development, producing about 100,000 fast games per day.
Great computer. Certainly it has been an excellent buy for the development of Houdini. BTW, assuming you are using all the cores to run Houdini 3, how many KN/s you get? Also, it's possible to know if Houdini 3 will use 32 cores maximum or this time it will be changed to 64?
> Great computer. Certainly it has been an excellent buy for the development of Houdini. BTW, assuming you are using all the cores to run Houdini 3, how many KN/s you get? Also, it's possible to know if Houdini 3 will use 32 cores maximum or this time it will be changed to 64?
It's great for running many single-core games, but the hardware doesn't scale very well beyond 16 cores. Even compared to my older Opteron 6128 the Bulldozer architecture is a regression.
Houdini 3 will have the same CPU limits as Houdini 2: 6-core for the Standard and 32-core for the Pro.
Will there be an upgrade price from 2.0c standard customers to Houdini 3 Pro
Hopefully around October 10.
Or at least better than an still expect Komodo MP
> Hopefully around October 10.
Any pre-order discounts?
I hope you are doing something concrete to prevent the hacking/pirating of Houdini 3. Chess websites are full of people using pirated versions of Houdini 2.0 c, which seems to be as good as the legal version.
Genuine buyers like me feel like fools and feel cheated when taunted by such people.
Hope you are doing something really effective to prevent the hackers.
> I hope you are doing something concrete to prevent the hacking/pirating of Houdini 3. Chess websites are full of people using pirated versions of Houdini 2.0 c, which seems to be as good as the legal version.
> Genuine buyers like me feel like fools and feel cheated when taunted by such people.
> Hope you are doing something really effective to prevent the hackers.
The only effective solution to piracy is NOT releasing the software (aka Rybka Cluster model).
It's still not too late for me to make this decision for Houdini 3...
The fools are not the genuine buyers but the people that use the pirated versions, they kill computer chess.
If you want chess engines to make progress, you should support engine authors - it's the smart decision.
But I will not follow a cloud/cluster model as was did for Rybka.
I was very disappointed when Vas/Lukas decision decided go into the cluster/cloud model as a priority and didn't follow that.
Also they must have they own "pocket" reason.... but it seems the Rybka purchase/download is a dead project.
It is not for reasons you are pretending or assuming (like "users who dont buy engines but play with pirates"), the real problem for a number one like Vas is that his product is dismantled and then the secrets are stolen by his competitors who all lack the success of Vas. Since the Houdini author is present and discussing I could initiate a debate about the historical past of Houdini. Most of you will remember the match between Houdini and Rybka, the latter not on the designed hardware of course and people were enthusiastic about the strong performance of Houdini. Not realising that this would happen to a number one, if it isnt updated since long.
I would call the naive presentation of such a top clash match (Thoresen) dishonest. (1) Rybka not being updated (2) Rybka against something that was tuned against her (3) Rybka not on its designed hardware (4) not even mentioning the hereditary factors in Houdini, e.g. if a special weak parameter in Rybka is repaired in Houdini, that could already make a difference for better results against each other - to sum this up, we all know that Rybka in its actually best code on the cluster is a different animal, BTW this is why most await so dearly R5, in particular all the other authors who want to participate of the new innocations Vas had found.
The actual reports about H3 are disappointing because on CSS Robert clarified that a focus on an increase of say 40 Elo is vague if the error margin is say +- 25 between H2 and H3. I am a little bit irritated because one time Robert says that 1000 games are better but the actual tests with long TC he played just 120 games between all players.
The problem for Roberts H3 are in the testing without existing super strong HW. H3 has the same problem like a future R5. The different results on little HW are irrelevant for the strength with top HW. If only rational decisions would count, then Houdini must cooperate with cluster Rybka. Then efforts like Komodo or Stockfish would become putty.
not to mention the hereditary factors in Rybka? if a weak parameters in fruit repaired in Rybka that could have made Rybka what it is.
Elaborate Houdini bashing. Sounds like you have a very brown nose and if the butt should fart while your nose is in, then you would have freckles as well.
1) The Task Manager with the CPU load of the 64 cores - only one core per module is used.
2) The Task Manager with 31 Houdini 3 and 31 Houdini 2.0c processes running.
3) The cutechess-cli window with standing after 22 finished games (+12 -1 =9)
> Are u going to post the games ?
Yes, at the end of each 120-game match I'll upload the PGN.
Just to clarify, you didn't use the HT cores because that would give H3 too much of an advantage against engines that didn't take advantage of them?
What's the scaling multiplier from 16 to 32 on your AMD? Is the scaling any different when you use the HT cores? What's the node count multiplier running flat-out versus a single core with no HT?
> Just to clarify, you didn't use the HT cores because that would give H3 too much of an advantage against engines that didn't take advantage of them?
I don't use them because they generate unpredictability. If two cores of a Bulldozer module are used, they run at about 65% of the speed when only a single core is used. That complicates the testing, as there is no guarantee that all threads run at the same speed all the time.
> What's the scaling multiplier from 16 to 32 on your AMD? Is the scaling any different when you use the HT cores? What's the node count multiplier running flat-out versus a single core with no HT?
The scaling from 16 to 32 is poor (I don't recall the exact numbers), and from 32 to 64 is non-existent.
> I don't use them because they generate unpredictability. If two cores of a Bulldozer module are used, they run at about 65% of the speed when only a single core is used. That complicates the testing, as there is no guarantee that all threads run at the same speed all the time.
You must have some way of assigning the individual threads to the modules ?
> You must have some way of assigning the individual threads to the modules ?
I use "start /affinity" in Windows to run cutechess-cli, which automatically gets inherited in the engine processes it spawns.
CPU 0 start /affinity 1
CPU 1 start /affinity 2
CPU 2 start /affinity 4
CPU 3 start /affinity 8
CPU 4 start /affinity 16
CPU 5 start /affinity 32
etc ? On a normal CPU e.g. Phenom II X6
On yours, for 1 thread per module, would it be CPUs 0,2,4,6,8 etc
On the AMD server I run cutechess-cli with "start /affinitity 5555555555555555" to skip every second core.
So for cutechesscli - does one instance run all the matches concurrently ? I must study that tool.
EDIT: I saw another article about a quad, talking about a hex mask, which gives the same values as in my post.
CPU3 CPU2 CPU1 CPU0 Bin Hex
OFF OFF OFF ON = 0001 = 1
OFF OFF ON OFF = 0010 = 2
OFF OFF ON ON = 0011 = 3
OFF ON OFF OFF = 0100 = 4
OFF ON OFF ON = 0101 = 5 confirmed via task manager
OFF ON ON OFF = 0110 = 6
OFF ON ON ON = 0111 = 7
ON OFF OFF OFF = 1000 = 8
but if you think the equal match is deep rybka cluster 200+cores vs. H3 on 12 cores or say 32 cores.. then NO..
Robert Houdart make plausible a cluster version of Houdini 3 x64.Then Fans OF Houdini or shall I say Members/USers Will provide a Cluster Of 200+CORES for Houdini to smash Rybka 4
SIck and tired to point of VOMIT of all this crap of cluster rental blah BLAH blah....
Rybka 4 IS WEAK unless its in a CLUSTER /CLOUD PROGRAM... HAH!!!JOKE ON YOU VAS..
NEWSFLASH....HOUDINI NEW ENGINE TO BE BEATEN... DONT NEED 200+ CORES TO BE BEATEN.... THAT PROVES WHO HAS A DEEPER POCKET BOOK AND GROUPIE FANBASE OF VOLUNTEER'S.
HOUDINI 3 X64 WIL CRUSH RYKA 4 CLUSTER OR NO CLUSTER.. BRING OUT RYBKA 5 WITHOUT THE NEED OF 200+ CORES TO BEAT HOUDINI THEN YOU CAN SELL YOUR ENGINE COMMERCIAL AGAIN..
I RATHER USE FRITZ 13 THAN RYBKA 4 OR 4.1 FOR ANALYSIS......
> I guess you optimized it against Houdini 2. Rybka also always had better results against it’s predecessors.
The Houdini development framework uses 9 engines, including 2 previous Houdini versions (1.5 and 1.03a) but not Houdini 2.
I'm watching the games now against Houdini 2.0c and I really like what I have seen. What a pleasure to see this battle!
76.5-43.5 (+94 Elo ± 42 Elo).
I haven't seen any of the games yet, hopefully everything is fine.
Houdini 3 - Stockfish 2.3.1 match has started - no games finished yet.
I currently use H 1.5 If H3 does half as well as against Komodo 5 I will be buying it without a second thought.
Good improvement at this very high level....
But will it do so good playing the other top players in the field !?
We'll have to live & see....
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill