Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka vs Houdini @TCEC
- - By TheWizardOfOz (*) [ca] Date 2013-10-21 19:54 Edited 2013-10-21 21:04
Some time ago, there was a 6 games match between Houdini 3 and "The cluster" on 64 cores.

2012 Nov/Dec.

Houdini 3 played on a 3.4 GHz real 16 core computer (~25MNodes+ in the starting position).
Cluster Rybka 4.1 played with approx. 110 GB 6pc Tbs and 64 cores.

The result: +4 -1 =1 for Houdini 3
Seemed that Houdini scaled very well on 16 cores.

The first two games were played without any opening, which was really not a good setup for Rybka.
So if we discard those 2 games, the result would have been +2 -1 =1 for Houdini 3

However, on 16 cores, on the TCEC computer, at stage # 1 and 2, both engines seems to be of comparable strength so far.

Any explanation to this disparity of results ?

On TCEC, they only use 5 man tablebase.
Maybe using 6-man tablebase with the cluster was not a good setup for a computer match ?
Parent - By Bloodbane (**) [fi] Date 2013-10-22 03:55
Luck. I only four games(or six) anything can happen.
Parent - - By Peter Grayson (****) [gb] Date 2013-10-22 08:59
Here's an example of the Houdini 3 versus Deep Rybka 4.1 Noomen 2012 match played out on two equal PC's through Autoplayer. The are several strings of draws with one sequence of six , two of five etc. That is why for evaluation testing a broad range of opening lines should be used that although balanced should give opportunities for each side. The Noomen 2012 set is a good example of contemporary lines that fulfills that requirement.

PeterG
Parent - - By NATIONAL12 (Gold) [gb] Date 2013-10-23 00:28
What bothers me in nTEC is that although  Martin is using reversed colours in the match in stage 3 onwards,some openings even by top GM's are not balanced we will see engines because of position either getting wins they don't deserve or draws similar because position allows this with reversed colours.
Also I have never been convinced by test suites as no engines would ever get to these positions and imo are pure tactics.
Just my take on things.
Parent - - By turbojuice1122 (Gold) [us] Date 2013-10-23 02:58
I agree about test suites--they don't effectively show how the engines would play in games.  However, they are useful in comparing engines for analysis purposes.
Parent - By Peter Grayson (****) [gb] Date 2013-10-23 10:23

> I agree about test suites--they don't effectively show how the engines would play in games.


Where the engine gets to in a game is usually guided by a book or test suite and test suites are very useful for testing new moves in openings to give a fixed number of games with however many variations are being employed.  In this way the resultant games can be used for tuning certain book lines. I'm not keen on using an engine without a book because there is no benefit where an opening line has proven track record.

My preference for engine testing is that opening lines should be contemporary and played as both sides. As far a I can tell, engines gain time on the time control using test suites just as they would do if using a book so no penalty there and takes out any randomness of book move selection.

PeterG
Parent - - By The Wizard (***) Date 2013-10-24 07:53
Hi,
    Many moons ago in a chat with John Nunn about chess engines he thought to be fair that two games from the same position should be played obviously with each engine getting a game with each colour, now it is up to each individual to chose the opening line\lines ... I don`t know if JN has changed his opinion but it does stop an engine getting a bad position out of an opening book which it has no control over.

As an example and it is really only for my own interest I am running some tournaments from the positions below. Whatever engines I use in the tournaments they get to play each side of the position 4 times

1. e4 d5 2. exd5 Qxd5 3. Nc3 Qa5 4. d4 Nf6 5. Nf3 c6 6. Ne5 Be6 *

rn2kb1r/pp2pppp/2p1bn2/q3N3/3P4/2N5/PPP2PPP/R1BQKB1R w KQkq - 0 7


1. e4 Nf6 2. e5 Nd5 3. d4 d6 4. c4 Nb6 5. exd6 cxd6 6. a4 *

rnbqkb1r/pp2pppp/1n1p4/8/P1PP4/8/1P3PPP/RNBQKBNR b KQkq - 0 6


Regards
Parent - - By turbojuice1122 (Gold) [us] Date 2013-10-24 11:00

> they get to play each side of the position 4 times


I assume that here, you're taking into account multiprocessing randomness to see if there is a systematic tendency for a position to go one way or the other?
Parent - By The Wizard (***) Date 2013-10-25 06:47

>I assume that here, you're taking into account multiprocessing randomness to see if there is a systematic tendency for a position to go one way or the other?


Hi Turbo,
              Yes .... I am always curious lets say in a 4 engine tournament which engine chooses what move, do they play an alternative next time round and at what point do they deviate.

It helps reach certain conclusions about positions that interest me these days and then you can run another tournament further down the opening line from a new start position.

As an example I came across http://www.youtube.com/watch?v=yPJeMrhL19w and around 8 Minutes 30 seconds into it you reach

r1b4r/1pk1ppbp/p5p1/2P5/5P2/1R3N2/P1PB2PP/5RK1 b - - 0 16


The author then mentions that 16...a5 is not good because of 17.Rb5 and the black a pawn is simply weak ...... Now what is funny is that every engine I have tested wants to play 16...a5 :lol: with the second choice tending to be the game move 16...Be6 ........... The games I have found are all black wins in fact after 16...a5 17.Rb5

r1b4r/1pk1ppbp/6p1/p1P5/5P2/1R3N2/P1PB2PP/5RK1 w - - 0 17


Now after running a tournament Rfb1, Re1 and Bc3 have all been played but not Rb5

Everyone has there ways of doing these things but this works for what I want

Regards
Tony
Parent - By Peter Grayson (****) [gb] Date 2013-10-23 10:09

> Also I have never been convinced by test suites as no engines would ever get to these positions and imo are pure tactics.
> Just my take on things.


I find test suites are very useful for comparing different engines' performance with thematics e.g. open or closed positions, fianchetto games such as the Sicilian Dragon or some of the Reti type lines but of most benefit when testing new moves in openings. Sometimes the human eye can spot a potential good move missed by the engines and then running a number of games in a time control rather than ply-depth that gives a type of Montecarlo run on the ensuing lines that give useful potential performance information.

PeterG
Parent - - By Quapsel (****) [de] Date 2013-10-23 05:11

> Seemed that Houdini scaled very well on 16 cores.


Seemed that Houdini ist very strong in general!
CEGT-one-core gives H3 impressing 100 ELO more than R4.1.

I believe to get really more strength when using 32, 64 or 128 cores instead of 16 cores is a very hard Job! Really successful in case of Rybka? Or more or less a Marketing idea?

Quap

PS:
Have you seen http://www.chesscluster.com/index.html?
Parent - - By Mark (****) [us] Date 2013-10-23 10:02

>> PS:
> Have you seen http://www.chesscluster.com/index.html?


Should be interesting!  Based on Stockfish...
Parent - - By Labyrinth (*****) [us] Date 2013-10-23 11:24
It would be nice if they had some benchmarks!
Parent - By Mark (****) [us] Date 2013-10-23 12:22
It sure would!  Seems like it's still pretty early in the development process, though.
Parent - - By Carl Bicknell (*****) [gb] Date 2013-10-24 08:29
woooah, that's big news. Is it genuine though?
Parent - By CumnorChessClub (***) [gb] Date 2013-10-24 10:32
http://www.chesscluster.com/index.html appeared on the web in july/August -2011 and the only thing I can see that's changed is the engine that it is based on.
Parent - - By Dr.X (Gold) Date 2013-11-12 22:58

> Based on Stockfish...


"Based on " is a rather nebulous way to reference the chess engine behind the cluster!
Parent - - By Eelco de Groot (***) Date 2013-11-13 16:37
The engine is called Magneto, based on Stockfish 3. The domain seems to be registered by a 30 year old Polish chessplayer, MÄ…ka Marcin. Here are some of his chessgames: http://www.chessgames.com/perl/chessplayer?pid=57907 Do we know anybody else living in Poland and developing chess clusters but who has disappeared from the radar so to speak, for the past two years? It may all be a coincidence of course, the only real thing that would connect them is Poland.

Eelco
Parent - - By Dr.X (Gold) Date 2013-11-13 21:24 Edited 2013-11-13 21:36
Interesting thought. Even the format leaves one to wonder. When someone likes a tasteful format it is hard to give it up. At the same time-it would be  bad business to copy someones format that you knew who left a questionable impression.

[edit] There is enough difference -but at the same time there is a form of similarity that just cannot go unnoticed. The few colors used  are dramatic enough to make both standout.
Parent - By Eelco de Groot (***) Date 2013-11-13 22:24
I had not really seen the Rybka Cluster site before, from your picture there does seem a resemblance! But that could be some copying from Cluster Chess. Maybe Vas is not at all involved, or only gave some suggestion to a fellow chess player, that it could be done. He would not be against the competition, I think he would only welcome it. Chessplayers can be quite conservative and if there are more sites like this, it would help the format. Professionals have different wishes and needs, this is not pointed at simply computer chess enthusiasts I think. Anand as another example, is working closely with the HIARCS team and it certainly seems to help him in his preparation now :cool: There it will be of importance that Harvey really is an openings specialist, that gives a lot of added value. Vas would not be interested in that even though he may be the better chessplayer. Still, Stockfish 3 is not quite State of the Art anymore so from that, it is not sure the site has been updated a lot. It has a "copyright 2013" notice at the bottom but not much else that is a sign the launch is imminent. 

Eelco
Parent - - By Dr.X (Gold) Date 2013-11-13 21:39
And this shoots both ways-

Anonymous

You don't have to disclose your identity to us.

Confidential

Data being generated through the analysis process is the client's sole property. We don't use it in any other way or disclose it to any third parties. You can keep your analysis data on our servers to have a secure backup or you can delete it from our servers and it shall be gone forever.
Parent - - By keoki010 (Silver) [us] Date 2013-11-13 23:56

> Data being generated through the analysis process is the client's sole property. We don't use it in any other way or disclose it to any third parties. You can keep your analysis data on our servers to have a secure backup or you can delete it from our servers and it shall be gone forever.


Deleting something from a server does not mean it is gone forever. Some of our nations leaders found that out! :twisted:
Parent - By Dr.X (Gold) Date 2013-11-14 00:10
As well as "...or disclose it to any third parties." is wishful thinking and is a almost like waving a red challenging flag  to some 13 year old nascent enterprising hacker (s) !
Parent - By TheWizardOfOz (*) [ca] Date 2013-10-23 18:27
You can rent up to 2048 cores...:eek:
Parent - - By TheWizardOfOz (*) [ca] Date 2013-10-23 18:31
Some updates from Houdart on the TCEC Chatwing during the game Houdini-Junior.

He says that the special dev version used in stage # 3 does not use contempt (just an experiment he said).
At later stages, he might change this.

He also said that @2 minutes time controls, he saw a 40-50 ELO improvement with this dev version.

Finally, if everything is fine, Houdini 4 should be ready in November.
Parent - - By jammy (***) [gb] Date 2013-11-01 12:17 Edited 2013-11-01 12:20
Well now Dec it seems, and far from cheap!

http://www.topschach.de/houdini-standard-p-2747.html

But I would make a guess that it will also be shipped out with a new Aquarium too!
Parent - By Peter Grayson (****) [gb] Date 2013-11-12 21:45

> Well now Dec it seems, and far from cheap!


That's for the Chessbase edition that costs extra for the CB frills as well as the uci engine. If marketed as per H3 release, the uci engine alone would be released first and at less cost direct from Cruxis.

PeterG
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka vs Houdini @TCEC

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill