Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / GM + chess engine vs Rybka
- - By drkirkby (*) Date 2007-07-13 12:28
Any thoughts of the result if a GM played with the aid of an engine vs Rybka? Let him/her use any software tools (Fritz, Toga, engame tablebases etc) he/she wants - except Rybka of course. Both engines use the same hardware.

The GM + other engine has the same time controls. The GM then decides whether to make the move suggested by the engine, or one he thinks best. Since Rybka is stronger than any other engine, it is clear if he just relies on the engine, he/she will loose.

It would probably be quite cheap to set this up with a lesser GM - not Kramnik of course.

I personally think the result would be much more intersting than that of games at pawn odds.

It would be good publicity for Rybka if she wins, yet it could not fairly attract negative publicity if she looses.
Parent - - By turbojuice1122 (Gold) Date 2007-07-13 12:47
Rybka would lose this very badly--In a 20-game match, I would say that the GM + engine would score at least 15 points.  This is because the horizon effect comes into play horribly for Rybka--the GM (assuming that the GM isn't a patzer at using chess engines for analysis) would be able to use Fritz to move forward in important areas of the analysis, especially with potential sacrifices, and see how things are standing at the end of a few important long lines.  Rybka wouldn't stand a chance.  A much more even match would be to have Rybka on very powerful hardware against a GM + engine on single processor.
Parent - - By Alkelele (****) Date 2007-07-13 13:24
According to the logic you present here, GMs + Rybka would totally wipe the floor with a pure Rybka. But this is not what we see in the Freestyle tournaments. Furthermore, I believe (but cannot say for sure) that some GMs played with non-Rybka engines as help in the 2nd and 3rd Freestyle and got severely trashed as a consequence of this.

Besides this evidence, there are several ways to counter your argument -- here just one example of a phenomenon that your argument doesn't consider:

Fritz will disregard some strategically good continuation that starts with some tactics, because Fritz misevaluates some positions resulting from the intermediary tactics. The GM will disregard the good continuation, because he misses the intermediary tactical options. While Rybka both realizes the tactics and evaluates the resulting positions correctly. Result: Rybka 1 - GM + Fritz 0. This is not fantasy, I have seen many positions where a thing like this seems very likely.

Basically, the old myth is that "GMs just need engines for tactics, their own strategical judgement is far superior to that of any engine anyway". This is really just a myth. We can split the claim up:

1) The strategical judgement of a GM is superior to that of any engine.
2) GMs just need engines for tactics.

The flawed reasoning is that 2 follows from 1. It doesn't take into account things like

a) Speed.
b) An engine's tactical blind spots caused by bad evaluation.
c) Some positions need a lot of concrete calculation with evaluation "just" being of a quality above a certain treshold. There is no logical guarantee that a GM + a bad evaluation engine will be able to produce such a mass of reasonably good analysis.
d) ...

But further experiments would be interesting, no doubt... Besides those we can already hint at from the 2nd and 3rd Freestyle :-)
Parent - By Nelson Hernandez (Gold) Date 2007-07-13 15:03 Edited 2007-07-13 15:06
There are a number of intangibles too, which all too often are fairly decisive.

-  overall centaur ability, which as you know is a combination of several factors (avoidance of mouse-slips, time management, search method)
-  opening preparation, and by this I mean the most efficient way to develop a flexible and varied repertoire
-  temperament, something that is very often taken for granted but is really critical when under pressure
-  hardware, and by this I don't just mean pure speed by how you array your hardware assets in the most helpful and efficient way
-  opponent research and grand strategy, which is something that transcends opening prep

Finally there has to be the simple acknowledgement by notoriously egotistical GMs that computer chess requires a different set of cognitive skills and is therefore almost a different game.  (Otherwise how do you explain the Cato team's notable success despite abysmal OTB chess skills?)  The fact is that OTB skills do not translate particularly well into the computer chess realm in the heat of competition.  The only exception I can think of is in endgames, where a GMs positional analysis can often (but not always) exceed a computer's. 

Of course, all these categorical statements can now be made thanks to Rybka.  Before it came along there were all sorts of things GMs could legitimately say to preserve their dignity and self-worth and denigrate their mechanical rivals.  But now things have clearly shifted into a new realm and it is time to face the stark reality that we are at a point where, during competition, a GM's skill-set offers relatively little incremental value compared to the value the machine is offering.
Parent - - By turbojuice1122 (Gold) Date 2007-07-13 15:16
But in the original proposal, things were kept equal--we assume that Rybka doesn't have some really awesome opening book with it such as the type that were often used in the Freestyle tournaments, and we assumed that the hardware was equal.  In the case where IM/GM + Rybka didn't beat pure Rybka in the Freestyle tourney, we were generally dealing with a Rybka on much superior hardware.  GM + (Rybka on a quad) will demolish pure Rybka on a quad.  I know because when I had more free time, I playing as a centaur would almost always beat my opponent if the opponent was pure engine with very similar hardware as myself.  If the opponent had noticeably stronger hardware, things were a bit more equal.  The reality of GM + Fritz against pure Rybka will lie somewhere in between, but I remain quite confident in the GM + Fritz if the hardware is the same and Rybka doesn't have a really awesome opening book.
Parent - - By Nelson Hernandez (Gold) Date 2007-07-13 17:24
Pray tell, who has a "really awesome" opening book?  Are you talking sheer size, best-move identification, customization/opening preparation, or something else?  Book wins happen from time to time but they are usually the result of someone having an awesomely BAD book. 
Parent - By turbojuice1122 (Gold) Date 2007-07-13 23:12
I will leave the discussion of who has a "really awesome" book up to the philosophers to debate...
Parent - - By Uri Blass (*****) Date 2007-07-13 21:53
I think that you simply overestimate GM
I do not expect even GM+rybka to score more than 14.5 out of 20 against rybka with no help.

Remember that the rybka team with vasik could not win with white in the last round of the last free style and this team could have bigger advantage because they had the programmer who know the weaknesses of rybka better than other players.

It is only one game but experience from the free style shows that rybka with no help scores clearly more than 25% against combination of rybka and humans.

Uri
Parent - - By turbojuice1122 (Gold) Date 2007-07-13 23:11
These Rybkas with "no help" that do well against Vas+Rybka are typically Rybkas belonging to very experienced Freestyle players, which is quite different from an "out of the box" Rybka that I'm basically assuming for such a match against a GM.  If we're talking about something else with, say, Jeroen's private book or the books of Nelson Hernandez et al, then I would say that a match against a GM who is allowed to use Fritz on equal hardware is a toss-up.
Parent - By Uri Blass (*****) Date 2007-07-13 23:22
I see no reason to think that the match is against rybka with the public book.

If the GM is going to use engines to help him then it seems obvious that Vasik will also use the best book that he can use in order to help rybka.

Uri
Parent - By Victor Zakharov (*****) Date 2007-07-13 13:59
I think this idea can work if after his move GM sees Firtz evaluation and the main line and remake the move.

So blunders will be eliminated. I beleive that in the previous match with such tactics Ehlvest could save two half-points.
But who knows may be engine ideas can mislead GM sometimes.
Parent - - By lkaufman (*****) Date 2007-07-13 14:05
     My feeling is that if the GM has free choice of software, he would be a slight favorite. If we limit his choice of software (or the hardware on which it runs), there will obviously be some limitation that would produce a fair match. My guess is that the hardware/software must be about a class below Rybka, assuming a GM rated around 2600, for the match to be fair. This suggests that a Rybka-Junior match held this way as a handicap match might be interesting if the one in Mexico doesn't take place, or if it does take place and Rybka wins by the expected crushing score.
     This type of match might produce interesting chess, but it is primarily an engine-engine match, with some human involvement. Junior (for example) would be roughly 300 points stronger than a 2600 GM and would clearly be the dominant "partner" in the pairing. I don't see it as a substitute for engine vs. human matches, rather as one way to have an engine vs. engine match in which the result is not obvious in advance. As you rightly point out, Rybka has nothing to lose from such a match.
Parent - By drkirkby (*) Date 2007-07-13 15:37
My own oppinion is that if the Rybka had better hardware, then the results would be of little interest or value.

But I think it would be an interesting experiment on the same hardware.

I guess (and I'm no expert on computer chess at all), is that as hardware improves more and more, the benifit of having the GM present would be less and less and the superior program would win.
Parent - By Uri Blass (*****) Date 2007-07-13 22:13
I think that if the target is human-machine matches it may be interesting also to give humans the right to consult.

I wonder how much better can be a team of humans that can consult during the game relative to one human.

Can team of N human with equal strength earn more rating from consulting relative to computer with N processors relative to computer with 1 processor.

Of course things may be dependent on time control and in blitz time control the right to consult probably does not change much for humans(only small help)
Note that I assume that humans do not take stupid decisions to consult in case that they can expect it to be counter productive and that they trained at home in consulting in order to avoid counter productive decisions.

Uri
Parent - - By kstevens (**) Date 2007-07-13 14:16
I just completed a three game match against the latest version of Rybka using my program Fritz 8. I won two of them and the third game was a draw. I was able to use my brain as well as my program as decided before the match. I even played Black twice and White once. This is a result from a player that is not a GM and not even a Master. I really can’t believe Rybka would stand the slightest chance against a GM using a computer program.
Parent - By Hetman (*****) Date 2007-07-13 18:59
Hi,
I understand that you played the games on the one machine.
You are probably to shy but if you are candidate to master it seems to be enough.
I played engine match equal endgame Rybka 2.3.2 against H9 on 1 proc machine and I  would say that Rybka does not have advantage.:-(.The H9 has searched chances to win.

Regards
Hetman
Parent - - By lkaufman (*****) Date 2007-07-13 22:13
     One factor is the hardware and operating system. If you played on 32 bit operating system, Rybka's superiority is somewhat less than on 64 bit, as Rybka benefits greatly from this, but Fritz does not. Also, if you played on single processor machine, there is more chance that the human player's input might be helpful. If both programs play on a quad, for example, it is much less likely that a player below GM level can help the play so much.
     Probably an even bigger factor is the opening. If you are playing against an opening book that you can see, this is a huge advantage. You can choose variations that will suit Fritz for example, whereas Rybka is just choosing at random. What were your rules for the opening choices (what book, what settings, could you look at the Rybka book in choosing your lines, etc.)? To really test this idea of human plus engine vs. Rybka properly, I would advocate using randomized openings with each side playing white once, like in CCRL/CEGT testing.
Parent - By drkirkby (*) Date 2007-07-14 00:49
I  would suggest that the hardware should be modern & state of the art. That means 64-bit and multiple CPUs with multiple cores and sufficient RAM that neither program's performance is crippled by lack of memory. I have not been following hardware developments lately, so I don't know what is available at semi reasonable costs. There may need to be some limit to keep the costs reasonable - perhaps no more than 8 CPUs.

I suggest if one party want to use XP and the other wanted to use Vista, they should be allowed to do so. Basically both parties would have a free hand to make their software work as well as it possibly can.

I think crippling a program with low spec hardware, or restricting the use of an opening book would not be very useful. It would be most interesting to know what is possible today when the software is working as well as it can be practically made.

I would suggest the use of any opening book the user wants to use.

I don't think it would be sensible to ask the GM to list the openings he would play in advance - he should be free to use whatever he sees fit. Therefore there seems to be no good reason that either bit of software should be required to make the opening book public.  The same hardware would be best, but there may be an argument for one side to use Intel and the other AMD, if they thought they would do better with one or the other.
Parent - By Gaмßito (****) Date 2007-07-13 22:19
Yes, I agree with you here. 
 
If the GM has experience against machines, and he knows well, how to use the program and how to play against Rybka, then he will have more chances to beat Rybka. Of course that will not be an easy task but,  without many doubts, the human has a strong advantage here. 
 
In fact,  to have a strong engine to check step to step all the moves and avoid some difficult tactical blunders, it is very very advantageous. If it is enough advantageous for myself, even much more for a GM.  
 
Regards, 
Gambito.
Parent - By irvstein1 (***) Date 2008-06-11 12:39
you are right plain and simple
Parent - - By Vasik Rajlich (Silver) Date 2007-07-14 07:18
First, I almost completely agree with the comments of 'Alkelele' below. Another data point we have gotten from the freestyle tournaments is that top centaurs seem to play somewhere in the neighborhood of 150 Elo above their computer assistance. It's logical to think that a top centaur player could compensate for a 150-Elo weaker engine. The question would be if the GM who participated in this match was himself a top-centaur player (which is far from guaranteed).

The idea of holding this type of match is interesting, but it's a bit arbitrary. I also don't want to 'pick' on other engines.

However, it would be interesting to hold some sort of centaur-play competions. It makes sense philosophically - centaur play is actually exactly what Rybka is for. One idea I've been bouncing around is to have a freestyle Scheveningen-style team match between a team of GMs (chosen by FIDE Elo) and 'top centaurs' (chosen by Pal/CSS freestyle results and maybe correspondence chess success). I would favor an equal-hardware restriction. Both sides have made it quite clear to me numerous times that they are quite sure that they would win, so there would be a lot of ego at stake, which is always a good thing :)

Anyway, there is a lot of stuff coming up so if we do this, it will be in October or later.

Vas
Parent - By Hetman (*****) Date 2007-07-14 13:47
Hi,

top centaur  vs GM+ programm great idea.
It would be interesting to watch such a competition.
Regards
Hetman
Parent - - By Uri Blass (*****) Date 2007-07-14 19:51
I wonder what is the proof that centaurs get 150 elo advantage relative to engine.

The main problem of the free style is that there was no equal hardware and we do not know the hardware that was used by the different teams.

It is logical to think that centaurs used more computer time for the simple fact that a team can use many computers
Only equal hardware competition can tell us the rating advantage that centaurs get against computers.

Uri
Parent - By Hetman (*****) Date 2007-07-14 20:46
Hi,

it is way to nowhere. There is no proof for and there is no proof against.;-). The experiment shall answer.

The aeroplane has the pilot and autopilot. Most time of the flight is controlled by autopilot(computer programm)
but starting and landing is driven by pilot. Here is clear.
Humans would like to think that they add some quality to the engines calculations. :-).
If not then it will be software and hardware race,only.

Regards
Hetman
Parent - - By Alkelele (****) Date 2007-07-14 23:22

> The main problem of the free style is that there was no equal hardware and we do not know the hardware that was used by the different teams.


Yes we do... You like to put forward the example of the Rajlich team not beating a pure Rybka. Please allow me to put forward the example of me beating a 32-core pure Rybka using only 6 cores.

> It is logical to think that centaurs used more computer time for the simple fact that a team can use many computers. Only equal hardware competition can tell us the rating advantage that centaurs get against computers.


Being able to use output from several engine entities is a genuine centaur advantage. If there is no engine/machine that is able to take advantage of several computers at once, while centaurs can, then that means that centaurs are superior to pure automatic entities in this particular aspect.

In general, you put forward tons of statements like "we can't know this and that for sure". You are right, but we can put forward qualified assessments, and those assessments tell us that centaurs can add about 150 Elo to a pure engine. This is simply what the evidence so far tells us. If you have another theory than "+150 Elo", then please put forward the evidence, instead of taking us on yet another kindergarden epistemological ride.
Parent - - By Uri Blass (*****) Date 2007-07-15 19:19
You say:

" If there is no engine/machine that is able to take advantage of several computers at once, while centaurs can, then that means that centaurs are superior to pure automatic entities in this particular aspect."

My response:
1)It is not correct that no machine is able to take advantage of several computers at once and gridchess in WCCC took advantage of many computers.

I suspect that it is possible to do it also for rybka if people are interested and pay vas for it.

I suspect that speed improvement of sqrt(n) when you use n computers may be possible and if it is correct it means that
1024 quads may be equivalent to something that is 32 times faster than a single quad.

It may mean advantage of 250 elo relative to a single quad assuming 50 elo for doubling the speed.

2)You cannot know the relative advantage of centaurs when the hardware of the participants is not known.
Relative advantage of centaurs with equal hardware may be dependent on the hardware that is used so claiming that centaurs have X elo advantage when you mention only time control and do not mention the hardware is meaningless.

Uri
Parent - - By Alkelele (****) Date 2007-07-15 21:57 Edited 2007-07-15 21:59

> 2)You cannot know the relative advantage of centaurs when the hardware of the participants is not known.


My point still is that people who play in these tournaments DO HAVE some knowledge about what the other participants are/were using. So we DO have significant data points, even though they are not public and you, as a consequence, cannot confirm them.

> Relative advantage of centaurs with equal hardware may be dependent on the hardware that is used so claiming that centaurs have X elo advantage when you mention only time control and do not mention the hardware is meaningless.


No, it is NOT meaningless, since we naturally always have the small print "as things currently seem to be", even though we don't add it to each and every post.

I actually don't understand your general point. We have some people who have thorough first hand experience with these things, and they estimate that centaurs can optimally add about 150 Elo points in Freestyle tournaments, everything else being roughly equal. There is, IMHO, no relevancy in pointing out that these estimates are not scientifically based on thousands on carefully monitored tests under very specific test conditions -- all of us already KNOW that. Experienced people put forward these more or less personal and reasonable estimates EXACTLY because they don't have any scientific tests to point to.

All Vas or others are really saying is:

The best centaurs are currently about 150 Elo better than pure engines.

Please note that, in this context, a doubling of cores is quite insignificant, in terms of elo. Even if we "penalized" the centaur for having double as many cores (because he used two computers), he would still have a 100 Elo "objective" advantage by these estimates...
Parent - - By Vasik Rajlich (Silver) Date 2007-07-16 11:10
In fact, the data we have is already not too shabby. Our team has played probably 25 games vs standalone Rybkas in these tournaments, and each of these Rybkas reported its hardware. (This is a feature of Playchess.)

Furthermore, when someone like Dagh or Jiri tells me what hardware they used, I personally have zero doubts about the accuracy of this, although a neutral observer may not.

The interesting question for me is if with more training we could push this number up to around 200 Elo. This might be feasible. Currently, all the serious centaur teams in these tournaments have some ugly blemishes on their record. For us, it's our loss vs OpenFormula, which I am very confident would not happen again. In general, I'd say that the gap between centaurs and engines has increased slightly over the past year.

Vas
Parent - - By irvstein1 (***) Date 2008-06-11 12:34
is there a limit as to hardware used ?? and if not how would rybka do on a machine that could think 500,000,000,000,000 nps say 500tb hash tables . would a beast like that win the freestyle without human help? thats alot faster than hydra ,im assumeing hydra did not have a human helping it ???
Parent - By BB (****) Date 2008-06-11 12:40 Edited 2008-06-11 12:42

>thats alot faster than hydra ,im assumeing hydra did not have a human helping it ???


The Hydra used in the Adams match (and other GM battles) was computer only. The sponsor of PAL Chess and Hydra has also used it/her (presumably as a centaur) under the name Zor-Champ in some Freestyle tournaments [and succesfully too, such as winning the 2nd Finals].
Parent - - By missingkasparov (*) Date 2008-06-10 07:57
Hello I have a question does anyone know the difference between rybka 2.1c rybka 2.3.2a?
Parent - - By dareapa (**) Date 2008-06-10 08:58
One is stronger than the other.
Parent - - By Vempele (Silver) Date 2008-06-10 16:48
One is much older.
Parent - - By Banned for Life (Gold) Date 2008-06-10 17:02 Edited 2008-06-10 17:06
One has magical powers, especially when run on mythical hardware.
Parent - - By Vempele (Silver) Date 2008-06-10 17:15
No, that's 2.1d3.
Parent - By Banned for Life (Gold) Date 2008-06-10 17:20
Yes, you're right...
Parent - By irvstein1 (***) Date 2008-06-11 12:25
everything but hardware is fair , because the gm will be very very strong ,going foreward and backward seeing what the resulting position will be . so i would only give him a 386 25 mhz with chessmaster 3000 and say now were even at least you should not make severe tactical blunders and he can still go foreward and backward and see what the future looks like. my money even like this will be on the gm though if it is slow time controls . because the gm will basically get to study each position , it would be like im at a tourn and im playing this guy ,our board is set up and we are playing. now to the left of us he has another board set up with our position there .he is constantly going to that board and looking at diff variations and he has a buddy there helping him granted his buddy aint that good but im getting mad by now they keep talking about the game etc . thats why the gm will win.
Up Topic Rybka Support & Discussion / Rybka Discussion / GM + chess engine vs Rybka

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill