Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Rook Endgame Eval Bug was not fixed!
- - By dragon49 (****) [us] Date 2010-06-01 03:02
This is PATHETIC!!

My Deep Rybka 4 SSE42 x64 just drew a game against a much weaker engine.  This can happen once in a while and it is not what I am complaining about.  This version was up a (doubled!) pawn, in a Rook and Pawn endgame, however, the rook was in front of the pawn, making the position a dead draw.  In spite of this, Rybka gave a winning score for a long time.  In the diagrammed position, Rybka shows +2.43.

C'mon Vas, how hard is it to fix this eval bug that you are aware of?!

R7/5kpp/P7/r4p1P/5P2/5K2/5P2/8 b - - 0 45

Attachment: bug.pgn (9k)
Parent - By chessorpoker (**) [us] Date 2010-06-01 04:08
better to trust zappa mexico or fritz 5.32 in these positions
Parent - - By daylen (**) Date 2010-06-01 04:41
Stockfish also has this "bug."

It's not like, a piece of cake fixing bugs. I don't think Vas wants to add exceptions to every single complaint he gets on the forum. Rather, you want to discover a pattern in the position to address the entire issue, instead of just fixing up a specific bug. So I don't know, maybe Vas will add doubled-pawn-in-endgame-and-bad-rook-placing when he finds a good way to do it.
Parent - - By Ricky (***) [us] Date 2010-06-01 16:49
Clone?
Parent - - By daylen (**) Date 2010-06-02 23:25
If you believe that Stockfish is a clone, you are very ill-advised. Translation: Stockfish is not a clone.
Parent - - By Ricky (***) [us] Date 2010-06-03 03:59
Did I said that Stockfish is a clone? No. Nervertheless, you said that it have the same bug?  What is the trunk of the bug? So, two diferent programs have the same bug, as per you msg. Please explain, it. Because if not you are very ill-advised, well, I don't think so. So please explain it.

Oops Stockfish is based on Glaurung, so, it have some of the code from it, right? so, some of the code is cloned.

The term clone is derived from κλῶνος, the Greek word for "trunk, branch" <- I just cloned the line from wikipedia.

Best regards,

Ricky
Parent - By daylen (**) Date 2010-06-03 05:30
Sorry for the misunderstanding on my part. I thought you were one of those who think Stockfish is a clone of Rybka. :)

Stockfish scores the position as +2.x.

If you want to know the back story, the open-source Glaurung engine developed by Tord was kind of "put on hold" when Tord went to pursue other interests. Marco came along and made various improvements to Glaurung and named it Stockfish, also released under GPL. In that sense it isn't really a "clone," which is a word with negative connotations, but rather a "fork," or divulgence from the path.
Parent - - By Uly (Gold) [mx] Date 2010-06-01 05:17
Does decreasing the "Rook Endgame Scaling" Parameter help in this position?
Parent - By dragon49 (****) [us] Date 2010-06-01 12:46
I don't know - I will play around with the setting tonight.
Parent - By GCP (***) [be] Date 2010-06-01 05:21

>C'mon Vas, how hard is it to fix this eval bug that you are aware of?!


Generally, very hard. Quite often, fixing these kind of things makes the ELO go down, because there are about 10 non-obvious extra conditions that have to be true before it should be activated, and you don't think of them.

It's easy to point to a single position where an engine goes wrong, but to "fix" a position and actually improve the engine you have to cover all other possible positions too.
Parent - - By M ANSARI (*****) [kw] Date 2010-06-01 05:47
This is not a "bug" but rather a weakness still in Rybka and Vas is the first to acknowledge it.  I think he added some rook endgame heuristics but still Rybka will steer some games into a drawn rook endgame a pawn up thinking it is winning.  Adding some extra 6 piece EGTB's that involve rooks and pawns helps some, but it is still an inherent weakness.  These things take time to add without hurting the engine.  I remember something similar with R3 under evaluating a bishop pair, but that has been dramatically improved in R4 to a degree I would not have thought possible.
Parent - By dragon49 (****) [us] Date 2010-06-01 12:48
The Rybka 4 release must have made a lot of people very happy, as it is good and refreshing to see users defending Vas.  Perhaps my, "This is PATHETIC" comment pressed some buttons.

Later
Parent - By NATIONAL12 (Gold) [gb] Date 2010-06-02 23:48
no disrespect intended but sometimes you must use your own brain.because engines can be stupid at times i now play only corr games.:)
Parent - - By Vasik Rajlich (Silver) [pl] Date 2010-06-04 17:38
Actually, Rybka 4 has a heuristic specifically for this case. Passed pawn, not defended by own pawn, own rook in front, opposing rook in back. You can thank the Beta testers for this one, they wouldn't let me sleep until I added it. :)

Vas
Parent - - By dragon49 (****) [us] Date 2010-06-04 17:44
Is the heuristic working correctly, with the +2.43 Eval in the diagrammed position that I posted?
Parent - - By Vasik Rajlich (Silver) [pl] Date 2010-06-04 17:47
Yes. :)

Vas
Parent - - By George Tsavdaris (****) Date 2010-06-04 18:17
The heuristic does what? Makes Rybka play the best defense with the weaker side(the one with the black here) and assure a draw?
Parent - By Vasik Rajlich (Silver) [pl] Date 2010-06-05 11:40
Decreases the scores for such positions (and therefore tries to avoid them when ahead and seek them when behind).

Vas
Parent - - By dragon49 (****) [us] Date 2010-06-04 18:19
You must be politely saying that I should be less of a fish, analyze the game without Rybka and figure out how white is winning!
Parent - By Vasik Rajlich (Silver) [pl] Date 2010-06-05 11:40
I just meant that the score would have been even higher without the heuristic.

Vas
Parent - - By InspectorGadget (****) [za] Date 2010-06-04 19:12

> You can thank the Beta testers for this one, they wouldn't let me sleep until I added it. :-)


I hope the next beta testers won't let you sleep until you include bishop under-promotion. Without it, Rybka does not fully understand chess :)
Parent - - By M ANSARI (*****) [kw] Date 2010-06-04 21:56
When bishop under promotion starts costing Rybka 1/2 points then you will most definetly hear some major raucous.  As it is BU only affects maybe 1 in 10 million games (if that much) and except for some chess endgame studies it is basically useless.  It would be nice to have in an analysis module where it would not affect playing strength ... but really there are a million things that are many times more compelling to fix.
Parent - - By Dragon Mist (****) [hr] Date 2010-06-04 21:58
I agree. BU irrelevant.
Parent - - By InspectorGadget (****) [za] Date 2010-06-04 22:13

> I agree. BU irrelevant.


I once played a game which needed BU to avoid stalemate (Queen or Rook we stalemating) and it was delivering a quick mate. I was very excited to see that. My opponent resigned immediately when he saw me under promote to a bishop :)
Parent - - By Dragon Mist (****) [hr] Date 2010-06-04 22:16
You actually played a game where BU was important??? :-)
Parent - By InspectorGadget (****) [za] Date 2010-06-04 22:34

> You actually played a game where BU was important??? :-)


Bingo! You'll never know when you gonna need it :)
Parent - - By Lukas Cimiotti (Bronze) [de] Date 2010-06-05 07:18
This would probably a once in a lifetime event ;)
I also got one remarkable game: my computer announced a mate in 91 - it ended in a won knnkp position :)
Parent - - By XNetman (**) [gb] Date 2010-06-05 08:11
Given that Rybka as an entity plays many more times the number of games a human does then it ceases to be a once in a lifetime event.

Rybka happily underpromots to a knight it seems when necessary, it should be able to do the same with a bishop.
Parent - By Moz (****) Date 2010-07-19 22:53

>> Rybka happily underpromots to a knight it seems when necessary, it should be able to do the same with a bishop.


There's a huge difference!  The knight can do things the queen cannot which makes it uniquely suitable for promotion in certain types of positions.  Positions where a queen promotion leads to a worse position than a bishop promotion are exceedingly rare.
Parent - By isilverman (***) Date 2010-07-20 00:26
Speaking of once in a lifetime events, 33 years ago I played a game against Bill Schluger in New Jersey. He was in his 80's at the time and is long dead.

I remember him because in this offhand game, I was White, and, incredibly, and without any irony, the game went 1 e4 g5 2 d4 f6 and you may be able to guess the next move without using Rybka.

Never thought it would ever happen, but it did.

A propos of I'm not sure what...
Parent - By AndrewWalker (**) [au] Date 2010-06-05 00:35
All this talk about BU being irrelevant is nonsense. Just have a separate version for it,
or a UCI switch, and be done with it!

Andrew
Parent - - By tomgdrums (****) Date 2010-07-19 22:08
Hi M ANSARI,

For analysis what do you think the best setting for Rook Endgame Scaling is?

I have the complete 3 and 4 man tablebases on my computer.

Thanks!
Parent - - By M ANSARI (*****) [kw] Date 2010-07-20 04:54
I think this is a very complicated issue, and to be honest I would leave it at default for play and maybe play with it for analysis.  It is no secret that Rybka 4 will go into a drawn rook endgame a pawn up with a very high evaluation.  In most cases no damage is done as the game would have been drawn anyway, but in some cases (which I have tested) R4 will give up a huge winning advantage and simplify into this drawn rook endgame and end up losing half a point.  In those games during analysis, I did play around with rook endgame scaling and managed to avoid the simplification of rooks and it ended up winning the game.  I think I used a rook scaling of average 60 to avoid the simplification.  I would definitely not use it in game play as it probably does more harm than good.

The more I have seen R4 games the more I have toned down my criticism of its endgame play, and I think I now understand why Vas has left it as it is.  R4 wins some pretty amazing games by simply outplaying its opponent in rook endgames.  The overly high evaluation makes R4 be quite aggressive and dynamic, and in many cases it will squeeze out a win.   It is most likely the best compromise for R4 at the moment.  Of course it is hugely frustrating when R4 avoids a winning endgame to simplify into an obvious rook endgame, but if you look at an endgame book you will see that 3/4 of it is only about rook endgames.  That is not unusual since 10% to 15% of games end up in rook endgames, so it is significant.  I am not sure how much 3/4 of a book of heuristics is in chess code, but most likely it is a major project.

By the way, rook endgame play by R4 can be greatly enhanced by having the relevant 6 man EGTB's.  I am sure you can find info on that if you do a search.  Am not sure how many GB the relevant rook EGTB's are, but they should be reasonably manageable.
Parent - By tomgdrums (****) Date 2010-07-20 05:10
Thanks for the info!!

I definitely won't mess with Rook Scaling for game play. (that is a small part of how I use Rybka anyway)

But I will definitely use your setting of 60 for Rook Scaling when doing analysis (which is the biggest use I have for Rybka)

I have also been thinking of using Rybka 4 as my engine for practicing endgames (pgns etc.).  I have been using Zappa Mexico II but every once in awhile I have noticed that Rybka might play harder for the draw in a losing situation so that would  be a better test for me!!  Thanks again for the help!
Parent - - By Vasik Rajlich (Silver) [pl] Date 2010-06-05 11:39

> I hope the next beta testers won't let you sleep until you include bishop under-promotion. Without it, Rybka does not fully understand chess 


With it too. :)

Vas
Parent - By InspectorGadget (****) [za] Date 2010-06-05 13:08

> With it too. :-)
>
> Vas


Yeah sure :-D
Up Topic Rybka Support & Discussion / Rybka Discussion / Rook Endgame Eval Bug was not fixed!

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill