Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / 40 core rybka 3
- - By Peter Hegger (*) [ca] Date 2008-10-03 04:23
Now here's the machine that should be used versus a super GM in a final death or glory, last chance for humanity, 8 game, 40/120 match.
In my dream match there would be no handicaps of anykind.
I see a real need to have an event like this since the handicap matches have skewed people's opinions of human chances versus computers.
I've read comments such as "see, the human is still stronger than the machine" after Milov won with his huge head starts.
I'm afraid that the sun won't set on the era of human chess supremacy until such a match is played and the human gets beaten badly.
Of course the chances of such a match are slim to nil, but one can always hope.

Peter

P.S. I'd gladly pay to watch it. Maybe if enough tickets were sold the whole thing might become semi-feasible.

Parent - - By Silvian (***) [ro] Date 2008-10-03 07:27 Edited 2008-10-03 07:52

> Now here's the machine that should be used versus a super GM


No, no, no....... definitely NO ! Here is this computer:
http://www.sdsc.edu/
(say " thank you !" please ) .

> should be used versus a super GM in a final death or glory,


I'm correct if I understand that the losing GM will be executed ? (lethal injection.........?).

> In my dream match there would be no handicaps of anykind.


I have the same nightmares !

> I see a real need to have an event like this


No, for me LHC/Geneve is the bigest event ! ( a furore normanorum libera nos ,Domine.........).But an uncalled for one !

> the human is still stronger than the machine


True ! See this post:
http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=7690
You can read here how the dog of Mr. DonaldLL chewed his machine !

> after Milov won


What ? A handicap match ? And only eight games ?

> I'm afraid that the sun won't set


True ! Possible to become even a supernova.

> I'd gladly pay to watch it.


Good ! What about to be the single beholder.For 5,000,000 $ of course !

> Maybe if enough tickets were sold


No ! North America is in recess.

All the best,
Silvian :)
Parent - - By Linus (***) [at] Date 2008-10-03 09:00
Now I know what I want for christmas:

RockStar Cluster, 258 Intel Xeon CPUs, 256 GB RAM, 1.445 Teraflops
Parent - By Dragon Mist (****) [hr] Date 2008-10-03 09:08
And full set of 3-4-5-6 Nalimov TBs! And a brand new no-TB-MP-bug Rybka!
Parent - By Silvian (***) [ro] Date 2008-10-03 11:09
I have already one ( but only in a photo  :) ) !
My brother-in North America - works with a such machine !

Poor Sivian :(:(:(:(:(...................................................... !
Parent - By Linus (***) [at] Date 2008-10-03 09:01

> Now here's the machine that should be used versus a super GM in a final death or glory, last chance for humanity, 8 game, 40/120 match.


In a no-handicap match a simple octal will do.
Parent - - By turbojuice1122 (Gold) [us] Date 2008-10-03 23:33
I would be much more in favor of a stable Skulltrail, i.e. something that is tried and true instead of something that is speculative.  Vas can afford to be speculative in this tournament because there is now obviously no doubt in anyone's minds that Rybka is by far the strongest chess entity ever, so the next step is experimentation for the future.
Parent - - By Ray (****) Date 2008-10-04 08:45

> there is now obviously no doubt in anyone's minds that Rybka is by far the strongest chess entity ever


Some of the Hiarcs fans would perhaps disagree

I would have preferred Rybka to play on a single Skulltrail 8-core. You know what will happen now - and you already see the comments - that this was a hardware victory not a Rybka victory.  I don't agree with that, but 8-cores would have been enough
Parent - - By M ANSARI (*****) [kw] Date 2008-10-04 09:05
The cluster is to show what the future will bring.  Cluster is nothing more than Rybka with more cores to play with.  I can't wait to see what the cluster setup is exactly, and how it works.  I really doubt that the first generation cluster would have a dramatic performance boost to an overclocked 8 core Skulltrail, but the experience with what to do with more cores will be a big advantage for Rybka 4.  I am sure for Vas this is just an experiment and the most likely cluster setup would be a passive cluster that will not hurt performance and if things go wrong it would simply revert to a simple 8 core Skulltrail.  The additional cores are probably just "helpers" and are only activated if certain triggers are activated.  Hopefully Vas will describe all this in more detail at the end of the tournament.
Parent - By Ray (****) Date 2008-10-04 09:15
Yes, agreed
Parent - - By jaboo2 (**) Date 2008-10-05 19:40
But what was this 40 core cluster? Was it a 5 x 8-core dual socket computer? But how were the motherboards linked? Was it with a fast computer bus or how exactly?
Parent - - By Ray (****) Date 2008-10-05 20:28
Yes - some of the Hiarcs fans are making a big deal of how it took Rybka's 40 cores to beat Hiarcs on a mere 8, e.g. I even saw a comment that WCCC08 was a moral victory for Hiarcs... Of course it is comparing apples to oranges - 8 cores on a single motherboard with Hiarcs, 40 cores on 5 motherboards spread across 5 machines - nowhere near as efficient as a hypothetical 40 cores on a single motherboard, even if that was possible. What exactly it is the equivalent of I'm not sure....
Parent - By jaboo2 (**) Date 2008-10-05 20:40
But how exactly were the 5 motherboards linked to each other?
Parent - - By M ANSARI (*****) [kw] Date 2008-10-05 20:43 Edited 2008-10-05 20:47
I think only Vas knows how exactly but I have some ideas I think.  You have 5 equally strong 8 core systems, first you set one of them as a Master and the other 4 as slaves.  The Master plays as MV mode with 4 variations and feeds each of the slaves one of the variations.  Immediately you will get a much broader and deeper search tree much much quicker.  Another way is to have a Master system run and the others running passively in the background with different engine setups.  Engines will be tuned differently to cover different aspects of chess playing ability.  For example one slave can be tuned for tactics and another for endgames.  A certain "trigger" evaluation difference can be activated when one of the passive slaves gets a trigger hit.  In this case Rybka would still be tactically aware without hurting other strengths.  Also Rybka master could play without EGTB's and thus play stronger without having the weakness of not having EGTB's since one of the slaves will have that task.

Connecting the cluster can easily be done by ethernet connections or fiber optic connections if cost is not a problem.  You could actually create a UCI engine that could play from CB interface and have several different engines being loaded via Chessnet software.  Chessnet allows you to name seperate UCI engines and allocate an IP address to each engine.  At least this is how I was trying to setup a Rybka cluster some time back with Rybka 2.3.2a.  Ofcourse Vas could be doing something completely different, but I like my method because it is quite easy to implement and there is no risk of worse play since the slaves are passive and the master system always has to "agree" to a trigger move.  Also there is no risk of hash bugs or latency hits by trying to share memory across the platform and your system can grow by as many slaves as you want.  The trickle down advantage of this is that this will scale automatically when hardware gets more and more cores.  This is why I think Vas is doing a very wise decision by going the cluster route.  It is not simply a geek experiment, there are many things that will be learned and implemented that will be put to good use in future Rybka versions.
Parent - - By jaboo2 (**) Date 2008-10-05 21:25 Edited 2008-10-05 21:34
What is a trigger move? What is a trigger? Do you mean 'trigger' as in event-driven programming?

http://en.wikipedia.org/wiki/Event-driven_programming
Parent - - By M ANSARI (*****) [kw] Date 2008-10-06 06:35
A trigger would be a pre-decided evaluation difference that would change the passive nature of the slave to an active one.  In this case let us say that the trigger would be a +.50 centipawns in evaluation.  So if one of the slaves sees a +.50 better evaluation than the Master, that would set off a flag or trigger which would allow it to feed that line to the Master to check it out.  If the master agrees then it will adopt that move.

By the way the cluster system mentioned is purely hypothetical and is how I would have gone about it.  That does not mean that Vas has adopted that method.  But I spent quite a long time on this about a year ago and am simply sharing what ideas I had.
Parent - By Lukas Cimiotti (Bronze) [de] Date 2008-10-06 09:03
One little hint: Vas doesn't want the slaves to be lazy - they have to work all the time ;)

Regards,
Lukas
Parent - By Dragon Mist (****) [hr] Date 2008-10-06 12:40

> The Master plays as MV mode with 4 variations and feeds each of the slaves one of the variations.  Immediately you will get a much broader and deeper search tree much much quicker. 


My guess is this is very close to what really happened. Judging by the output on the engine screen Lucas provided recently.
Parent - By Vasik Rajlich (Silver) [pl] Date 2008-10-12 05:43
Some of the details are different but you're right about some things:

1) The main reason for doing this was to understand the problem better. Multi-processor search is a huge topic these days. Had there been no WCCC, I would probably have waited until after the bugfix version & mobile ports, but it was good to do this earlier rather than later.

2) I do want to take advantage of search stochastics.

3) Slaves only talk to each other via the master. The network should have the property that dead slaves do as little to cripple other slaves as possible. Unfortunately, there are limits to this and the current cluster is after all a bit sensitive to dead slaves.

Vas
Parent - - By Lukas Cimiotti (Bronze) [de] Date 2008-10-06 08:49
I've already posted some things in the huge WCCC Beijing thread. The cluster consists of 5 seperate computers:
There is one Skulltrail 3.8 GHz
another Skulltrail 4.2 GHz
one 2x X5460 @ 3.4 GHz
one 2x X5365 3 GHz
one 2x E5430 @ 3 GHz
all computers have 8 GB of memory each (4x2 GB FB-DIMM 800 MHz - the X5365 has 4x2 GB FB-DIMM 667 MHz)
all computers were (of course) built by myself
each of them has a dual CPU socket 771 mainboard (2x Intel D5400XS, ASUS DSEB-DG, ASUS Z7S WS, Intel S500XVN)
all have fast harddisks for tablebases (mostly solid state disks)
operating system is Windows XP x64
they are connected by normal 1 Gbit/s ethernet using a normal 8-port Gbit switch by SMC

if there are any more questions - feel free to ask

Regards,
Lukas
Parent - - By Linus (***) [at] Date 2008-10-06 11:24

> if there are any more questions - feel free to ask


Do you have a photo, you could post here? I would like to see this monster. :)
Parent - - By Dragon Mist (****) [hr] Date 2008-10-06 12:35 Edited 2008-10-06 12:39
Yes, here's the photo :-)
Parent - By Lukas Cimiotti (Bronze) [de] Date 2008-10-07 06:08
Well, this is Vas with his computers.
Parent - By Dragon Mist (****) [hr] Date 2008-10-06 12:39
Do you need additional software (besides cluster version Rybka) to make it work?
Parent - - By Bouddha (****) [ch] Date 2008-10-06 15:23
What is your nick on playchess room ?

will you be using this system on playchess ?

regards
Parent - By Lukas Cimiotti (Bronze) [de] Date 2008-10-06 15:41
Rechenschieber, Victor_Kullberg and Abdul H

I have already used it on Playchess. But atm testing and improoving the setup is more important.

Regards,
Lukas
Parent - - By sainzlei (*) [tw] Date 2008-10-06 20:58
Hello Lukas ,

How many speed nodes on starting position on this monster ?

Regards.,
Sainzlei @ Republic of Taiwan
Parent - - By Dragon Mist (****) [hr] Date 2008-10-06 21:11
I think Lukas gave that number, it's around 1,200 kN/s?!
Parent - - By NATIONAL12 (Gold) [gb] Date 2008-10-06 21:37
i dont think it was Lukas that gave that number,but someone who was watching mentioned over 1k.
Parent - - By Dragon Mist (****) [hr] Date 2008-10-06 22:35
Yes it was Lukas, it's in the thread "Deep Rybka worse than Fritz 11", only the number is higher, 1,600 kN/s.

(I checked FAQ and help, and am unable to figure out how to put a link to a post, not to mention putting my own text instead as Vytron does "#$% :-) )
Parent - By Arsha (**) Date 2008-10-06 23:21
Parent - - By Vempele (Silver) [fi] Date 2008-10-07 05:20 Edited 2008-10-07 05:38

>(I checked FAQ and help, and am unable to figure out how to put a link to a post, not to mention putting my own text instead as Vytron does "#$% :-)


This is how it's done (I added an invisible empty [b][/b] tag to make it a non-link):
[url=http://www.your.url.here]Link text[/url]
Parent - - By Dragon Mist (****) [hr] Date 2008-10-07 06:50

> This is how it's done (I added an invisible empty [b][/b] tag to make it a non-link):
> [url=http://www.your.url.here]Link text[/url]


Yes, but how do you link to a particular post within a thread? Address doesn't change when you scroll down the screen, so if I just copy a link from the address bar and paste it, it would point to first post in the thread?
Parent - By Uly (Gold) [mx] Date 2008-10-07 07:54
Find the post ID by looking at the number that appears on the address when you reply to that post and add:

http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=

before it such as:

http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=112486
Parent - - By Lukas Cimiotti (Bronze) [de] Date 2008-10-07 06:13
I posted it somewhere here - it was around 1660 kn/s.
But kn/s do not mean much. It's real playing strength that matters.

Regards,
Lukas
Parent - - By nine castles (**) [us] Date 2008-10-07 06:22
You do understand that all things being equal faster search leads to better play, right? If we accept the claim that each doubling of the cores adds 50 elo, then vas' +100 elo estimate might be conservative.
Parent - - By Lukas Cimiotti (Bronze) [de] Date 2008-10-07 09:42
You do understand that all things being equal faster search leads to better play, right?
Yes, I do. But you don't understand what I wrote.
Fact is: not every node calculated by every core is counted, but there is an algorithm to calculate them. The current algorithm of the cluster version is very coarse. The slaves' strength is not considered. Using my hardware I can make a setup where 8 weaker slaves are present. The node count goes up to ~3000 kn/s, but playing strength drops by roughly 60 Elo.

If we accept the claim that each doubling of the cores adds 50 elo...
I don't accept that claim. In fact the first doubling of cores adds roughly 50 Elo points (or even a bit more), all further doublings add less.

Regards,
Lukas
Parent - By nine castles (**) [us] Date 2008-10-07 15:02
Ah you're right, the threading in this huge post confused me, you are talking about cluster computing. Right-o
Parent - By Carl Bicknell (*****) [gb] Date 2008-10-07 13:15
he knows what he's talking about
Up Topic Rybka Support & Discussion / Rybka Discussion / 40 core rybka 3

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill