Not logged inRybka Chess Community Forum
Previous Next Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka vs Benjamin - conclusions (6536 hits)
- - By Vasik Rajlich (Silver) [hu] Date 2007-08-10 12:24
Rybka has completed her match against GM Joel Benjamin at 'full pawn' odds, and won by the narrowest possible margin - 4.5 to 3.5. I'd like to thank Larry Kaufman for his organization of the match, GM Benjamin for his willingness to play (and almost pull off the surprise in the process), and our sponsors for making it all possible.

Benjamin & Ehlvest

Rybka has now played both GM Jaan Ehlvest and GM Joel Benjamin under fairly similar conditions. GM Ehlvest came into his match as a gunslinger. When he had better positions, he went full bore ahead, throwing the kitchen sink and everything else at Rybka in order to crack her defenses, her Elo and general abilities be damned.

GM Benjamin approached his match as a student of computer chess. He had an accurately assembled game plan, which revolved around avoiding unclear tactical positions at all costs and bailing out at the first sign that his position was on a slippery slope. He was also perfectly happy to use various anti-computer tricks to improve his chances. Just to pick one example, in the following position:

1k1r4/pbpqr1p1/1p3p2/2Pp4/2nP3P/2RQPNP1/P4PBK/2R5 w - - 0 30


Benjamin played 30. Bh3 Qd7 and then 31. Bg2. This would be a nonsensical move against a human, but Joel was quite confident that Rybka would repeat with 31. .. Qd7. This allowed him to gain a minute on the clock, another couple of minutes of thinking on Rybka's time, and also to get information about Rybka's evaluation of the position.

General Impression

If we substitute the exhibition game for the unfortunate game 1, then we can observe that in six of the eight games, GM Benjamin was in total control from beginning to end. In a few of these games, he slightly increased his advantage (games 7 & 8). In other games, his advantage slighlty decreased (game 3). In none of these games was there ever a moment where Rybka had started to genuinely untangle herself and actively threatened to win - Rybka simply defended from beginning to end.

There were two games which did get away from GM Benjamin.

The first one is I bet quite annoying for him. In game 4, from yet another clearly worse position, Rybka sacrificed a pawn with 21. .. c5. There were several ways for white to keep a nice, comfortable, risk-free advantage. I can only guess why Benjamin chose this moment to deviate from his match strategy - perhaps, he smelled blood. Accepting the sacrifice gave him two extra pawns, one of them a passed pawn on e6. However, the position opened up, and to make matters worse, the sacrifice was actually quite sound. No human could survive what happened next.

The second game which got away was game 5, where Benjamin played passively. Probably, this was just a slightly bad game, maybe a 2400-level or 2500-level game rather than a 2600-level game, the kind which is inevitable for a human from time to time. Rybka slowly turned the tourniquet and once black was bound up, a loss was inevitable.

At any rate, probably, the match result was quite natural and logical. In general, with good preparation and a sound and studied approach, the human has no basic problems in these games. However, he is a human, and can probably expect one generally bad game and maybe another irrational moment somewhere. It's worth noting that GM Benjamin did manage to avoid making any crude blunders.

Human Learning

The game results alone do not indicate that Benjamin's performance improved as the match progressed, but games 7 and 8 were definitely his best. He did his best job in those two games in improving his starting positions and generating the best winning chances. I doubt that learning was really a factor here, but it's possible that he did get a sort of "second wind" on the last day. On paper, his best chances to end up with an even score disappeared after game 6, and it was after this point that he played his best chess.

Takeaways for Rybka development

Aside from game 1, which doesn't really help us, there was no really absurd behavior from Rybka in this match. 12. .. f5 in game 4 is discussible. Ditto for 37. .. g4 in game 8. Her general play in games 7 and 8 could probably be improved at various points.

Another idea is to do something about the repetition trick that Benjamin used over and over to gain time on the clock and to conclude games when his own clock ran a bit low. However, avoiding repetitions in bad positions is a cure which might end up being worse than the disease.

What is Next

This will be for Larry and our sponsors to decide. One idea under current discussion is draw odds - Larry has been speaking with Steve Blincoe, who is interested in sponsoring such a match. Please stay tuned.

Vas
Parent - - By Ankan (*) [in] Date 2007-08-10 16:31 Edited 2007-08-10 16:36
Thanks Vas and Larry for organizing the match... although it was a bit boring as most games were drawn and Benjamin didn't win any game (after the contempt bug was fixed). Please keep longer time controls in the next match. I still feel a grandmaster can do better if given more time. In almost all games in this match Benjamin was running out of time and had to settle for draws. One game a day would be nice.
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-08-11 08:23
Yes, the full classical time control would make some sense. IIUC, Larry and Joel discussed this and it was Joel who rejected it.

Vas
Parent - - By lkaufman (*****) Date 2007-08-11 15:57
Yes, Joel did reject it, in part because he did not want to appear to be asking for such a huge increase over the Ehlvest pawn handicap time limit. However, now that he has played and lost this match, we have agreed that any future matches should be played at a time limit that would be accepted by FIDE for a title norm tournament. This means either increasing the base time limit from 60 to 90 minutes, or increasing the increment from 30" to one minute. Both Joel and I are inclined towards the second choice, as it guarantees that time pressure will play a limited role (as the player will always have no less than 1 minute per move to average in his remaining moves).
Parent - - By ernest (****) [fr] Date 2007-08-11 21:44
Hi Larry,

Maybe a little OT :-)

but did you get Joel to comment on the compared strengths of Deeper Blue (10 years ago) and today's best programs on state of the art hardware?
Parent - - By lkaufman (*****) Date 2007-08-11 21:53
In effect he did so, as he estimates that Deep Blue played around 2750 level then, and he now is a firm believer that Rybka does indeed deserve a rating of 3000 or more.
Parent - - By JhorAVi (***) [ph] Date 2007-08-12 05:13
I believe that Benjamin will win convincingly beat Deep Blue if it offers him pawn odds. Do you think Joel will a agree to that Larry? :)
Parent - By lkaufman (*****) Date 2007-08-12 14:17
Yes, I agree. Joel said he felt that he would easily win a match with Kasparov (at his best) at pawn odds like our match, and he considers Deep Blue to be about the same level with Kasparov.
Parent - - By Hetman (*****) Date 2007-08-10 17:05
Hello,

it is very good, unbiased summary of the match.

I think that the GM problem with self learning has been caused by the lack of time for analysis. 2 games per day it is exhausting and spends all human time, he has to relax computer does not. It was in favour of Rybka because her team has had time for analysis and improvement. It will be good to solve it in future by giving the second to GM or forbidding Rybka from internal changes during competition.
I am fighting :-) for the equal conditions for the human and programm.

Not only the over desk conditions are important, if we have limited the electricity consumed by computer it will simulate human situation.
Rgds
Hetman
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-08-11 08:25
This is also true. Note that we did not change Rybka between games, but anyway some time for the human to digest what is happening might also help.

Vas
Parent - - By Hetman (*****) Date 2007-08-11 16:02
I have understood that you have fixed sth after the 1st game. It was not the change  ?

The other thing I have to point out. I think that Rybka matches are real ones. That ones organised with cb and all drawn look as prepared ;-). Like the  human got there more money for the draw then for the win.
Regards
Hetman
Parent - - By lkaufman (*****) Date 2007-08-11 22:41
     It's true that we corrected the fatal bug discovered during the first game. The only other choices would have been to play the remaining games without contempt factor (which you would still call a change) or to forfeit the match, as the bug was fatal (Rybka would have simply blundered away her pieces). Bearing in mind that there was no rule against making changes between games, I'm sure you'll agree we made the right choice.
Parent - - By FWCC (***) [us] Date 2007-08-12 01:46
Hello Larry,how did Joel feel during the match?Was he in good spirits?Did he complain about fatigue?I think these matches should be reduced to 1 game per day but if the GM agrees to more then so be it.Was Joel optimistic about a match victory for himself?
Parent - By lkaufman (*****) Date 2007-08-12 02:59
Joel was in "good spirits", but did complain about fatigue at times. Curiously, he said he felt best just before his worst game (e2 handicap)! He was moderately optimistic before the match, but gradually realized that Rybka was even stronger than he expected, especially on my quad. He now feels that any future matches should be at a slower time control, probably 1 hour + 1 minute (rather than 30"). Of course he would prefer to play only one game a day, but as this would entail more time off from his chess teaching work it would raise the coast considerably. All American players are quite used to playing two rounds a day, so although it is not optimum, it is certainly normal for them.
Parent - - By Hetman (*****) Date 2007-08-12 05:52
I am not judging.

I think only that if Rybka has external support during the match the GM shall have either but it is at  his expensess.

I remember from the chess history the revange match Smyslov-Botwinnik. Smyslov has been ill and wanted to postpone the match but it did not happen.:-(. Not everything could be written in rules.

Rgds
Hetman
Parent - - By lkaufman (*****) Date 2007-08-12 14:24
Joel was free to study or play Rybka (on his laptop) between rounds, and to modify his planned openings based on what he learned during the match. He was welcome to discuss strategy between games with any GM friends he has. I don't see anything unfair about the rules.
Parent - - By Hetman (*****) Date 2007-08-12 17:26
I do not have anything bad on my mind but... how would you define the match human GM vs Rybka not to be the match Human vs Rybka + human ?

I wonder if someone is changing sth in the programm or in its parameters during the match he is involved in the match directly.

Rgds
Hetman
Parent - - By lkaufman (*****) Date 2007-08-12 20:15
I would say we started the match over (call it a second match if you like) with the exhibition game, but during the eight game match starting with that exhibition game the program and settings were not changed.
Parent - - By Hetman (*****) Date 2007-08-13 16:48
If GM has agreed and it was in the rules does not matter concernig rules

My point is to finish with the fiction that there are matches 'human vs machine' when they are competitions human vs  machine + human.
I do not mean that particular match.

Humans are designing automats to have the support and help and then thouse automats start fighting against them.
It is very sad - when human gets the chance to win the other human starts to help the machine.;-)
It is strange that humans want to prove that automats are better then humans :-(. Are we on the way to eliminate humans as the unneccesary luggage ?
I treat the programms as the tools to help me not to fight me :-).

Rgds
Hetman
Parent - - By lkaufman (*****) Date 2007-08-13 23:28
For me the goal is not to prove that engines are stronger in chess than humans -- that is already quite clear. My goal is to prove that they can play MUCH stronger than humans (thus pawn handicap), because this will prove that chess is not dead, and that humans can hope to play much better than they currently do. I see these handicap matches as inspiring, not discouraging. 
Parent - - By Hetman (*****) Date 2007-08-14 17:06
The humans can play better of course and chess are not dead, even without computers.

I do not see the positive correlation between the strentgh of the computer programms and strength of the chessplayers.
Invention of the car has not caused that people are runnig  faster. There is the opposite effect they are runnig slower and there are many illnesses caused by the lack of moving. The people are moving faster using the cars . I would expect the similar effect in the chess - better play using computer and worse without them, when the human will go out of the horizon of the computer analysis.
Rgds
Hetmans
Parent - - By Uri Blass (****) [il] Date 2007-08-14 21:26
I disagree.

There is a difference.

Cars cannot teach humans to run faster when computers can teach humans to play better.
I believe that the right software should be enough to teach GM's to play significantly better than what they play today
and maybe beating them with pawn odds will simply be impossible.

Uri
Parent - By XmikeX (**) [us] Date 2007-08-15 03:11
As echoed earlier, if software consistently demonstrates its superiority across the board and if this superiority is comprehensible to the human brain, then it would be foolish not to learn from software.

However, can humans be taught if they do not wish to learn?

On this, we could quote that famous anti-Rybka comment from young GM Nakamura.

We could.... but we now know the Rybka team works better than a single brain, if in denial, 8 days of the week.

XmX
Parent - By Hetman (*****) Date 2007-08-15 06:29
To Uri.
I agree with that what you disagree ;-) but partly.

You have right concerning the GM's which are professional players but for the remaining majority I do have right.
The amateurs will use the computer programms and  will loose tactical skills. You can observe that with CC players.
Most of them are powerless in the on board chess. They are corr IMs and are losing to 1800 players in the on board games.

Rgds
Hetman
Parent - By saxon (**) [hr] Date 2007-08-16 11:49
I agree with Uri Blass.
No matter how computer programs are good and run on no matter how powerful hardwares,
strong GMs should be intelligent enough to always force draws in pawn odds
games vs. computers.
In that sense GM Joel is more intelligent than GM Jaan.
Without that fatigue,in  one of the games (I think he simply forgot for the moment what he is playing against :)),
GM Joel would draw the match.
By playing vs. computer intelligence I don't mean "anticomputer play",but proper evaluation
of GM what machine can do and what not,what machine understands and what not.
I bet than V. Rajlich ,were he 2800 elo chess player would be capable  of drawing Rybka (in normal match)!
GM Jaan ,no matter how respectfull chess player he is,clearly does not understand most of these things,
GM Joel does understand.
Good example of understanding is ,of course, Kasparov.
His last official computer game vs computer program Junior ,clearly demonstrates that kind of intelligence in humans.
Baad example ,of course,is Adams.
He started his match vs. Hydra by loss,than drawn game gave him perhaps confidence that maybe he could
catch up by one victory current result.
What a sucker he was we all know today!The more he attacked more he was punished.
Instead of loosing the match decently,we all know what was the final result of it.
    
Parent - - By albitex (***) [it] Date 2007-08-12 02:37
I would like to underline the fact, that Mission Control, despite has participated in the
Freestyle with a supercomputer, has not won but sixth has arrived.
I have also thought on the fact that Mission has been beaten,
for example from you Vas, to the 26° movement.
Even if your team and you used the computers, I am sure that the winning movement
it is not gone out of a PC.
I am convinced that a teacher that plays with the help of a PC, (profit particularly
to avoid mistakes and in the ends, thanks to the tablebases) cannot be beaten from any computers that play without the human intervention.
To the respect I would like to submit you an example. In my game as centaur, attached to the present one, to the 27° movement Rybka proposes me 27... f6. Even if the game would be ended draw equally, I think that f6 for the black is a  horrible movement.
I would like to underline the fact, that all the engines with which I have analyzed the game propose me 27... f6.
In short, with this I want to say that according to me, the engines still have some evident limits. Even if with Rybka we have made some notable progress.
Hi
Attachment: Centaur.pgn (6.3k)
Parent - By albitex (***) [it] Date 2007-08-12 17:20 Edited 2007-08-12 17:37
continuation preceding message:
My adversary, that used Rybka in formality engine, has allowed me to do
the manoeuvre g6,Bf8,Bg7, without disturbing me or to create me a some problem.
A human adversary, also not a teacher, would surely have done a
some against game.
note: to the 13° movement, Rybka proposes me to sacrifice the horse for two pawns -
13... d4 14.Na4-Nxb4 15. axNb4-BxNa4 16.RxBa4-dxe3 17.fxe3-Bxb4 -
I am not very convinced of the effectiveness of this sacrifice. (at the end of the variation, after having given the horse, Rybka appraised the position draw (+0,22). For what motive I should give a horse to get an draw position, that
can I have without sacrificing nothing? ).
Other times Rybka has sacrificed the quality, finding then itself in disadvantage in final.  The sacrifices of quality
they are useful if they bring advantage to an real, or they help to defend the position being simplified. But in this case I don't see  sincerely the necessity .
Previous Next Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka vs Benjamin - conclusions (6536 hits)

Powered by mwForum 2.22.1 © 1999-2010 Markus Wichitill