Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Unique Rybka evaluation reproduced in Zappa?
- - By Peter Grayson (****) Date 2013-09-18 00:59
When Rybka 1.0beta was released, some observers noted a unique low evaluation in the starting position if White's knight or bishop was removed. At that time other engines gave a negative evaluation for white in the range -2.5 to -3.5. Rybka 1.0beta gave an unusually low value in black's favour at around -1.8.

This unusual but unique behaviour was seen up to and including Rybka 2.3.2a but by Rybka 3 had changed closer to the more conventional -3.0 for a knight or bishop deficit that is the equivalent of 3 pawns for a knight.

Just two months before Rybka's release, Zappa 1.1 dating August 2005 was giving an evaluation towards the high end at -3.94. However by the time of release of Zappa Mexico and subsequent Zappa Mexico II both of these Zappa engines were also giving <2.0 evaluations if White's knight or bishop were removed from the start position.

Given that Zappa actually defeated Rybka in the "Clash of the Titans in 2007" that was probably close to Rybka 2.3.2a and the Zappa Mexico engine, then given Rybka's unique evaluation seemed to be reproduced in Zappa it again raises the question of why wasn't Zappa used as the benchmark for similarity of engines in the ICGA tests instead of those low end open source engines?

White knight removed
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/R1BQKBNR w KQkq - 0 1


White Bishop removed
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RN1QKBNR w KQkq - 0 1


By the time of Rybka 1.2f, removing White's rook also produced an unusually low evaluation of just -2.2 when other engines were using the equivalence of 5 pawns for a rook. Zappa 1.1 was giving closer to 6 pawns. However, once again, by the time of Zappa Mexico and Zappa Mexico II, Zappa was giving a similarly low Rook value as Rybka at -2.8.

White's rook removed
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/1NBQKBNR w Kkq - 0 1


Originally Rybka and later Zappa are the only engines I can find that exhibited this unique low evaluation in these circumstances. It did cross my mind and my sense of humour ... what if Rybka actually lost to a version of itself ? Now that would be what I would call pulling a fast one.

Perhaps there was more exchange of information between authors than we knew about and hence similarity but it would be nice to know if Zappa's post Rybka rapid strength increase was due to more than natural progression.

For the purposes of these tests I used single thread and 64 Mb hash that would have been a likely configuration in 2007.

PeterG
Parent - - By Adam Hair (**) Date 2013-09-18 01:29
The comparison was made with the more prominent open source engines from that time, presumably because those engines would have been the only ones which code could have been copied from. It would have been better if source code from some of the top engines from that period could have been used in the comparison, but that did not occur.
Parent - By turbojuice1122 (Gold) Date 2013-09-18 03:28
The point being that there was no control group.
Parent - By user923005 (****) Date 2013-09-18 06:53
The comparison was made with, much weaker open source engines of the time.  All of them were well over 100 Elo below Rybka.  Crafty, for instance, would have been a better choice.  Glaurung another.  However, those engines would probably have had the advanced evaluation features, making their comparison a lot less powerful.

I complained about it at the time as an utter absurdity, which it was.  And those engines were 500 Elo below the Rybka version that actually played in an ICGA contest.
Which is the only version for which the investigation was applicable.

IMO-YMMV
Parent - - By Banned for Life (Gold) Date 2013-09-18 04:08
Perhaps there was more exchange of information between authors than we knew about

Anthony went through the source code for Strelka 2 in January 2008 and discussed what he found with Vas on this forum. You can search on his nick, Anthony C, to see the discussion. Of course this was three months after the match in mexico...

He tried some of the concepts in ZM II. One was extreme futility pruning, which didn't work well enough to be set as the default.
Parent - - By Peter Grayson (****) Date 2013-09-18 09:32

> Anthony went through the source code for Strelka 2 in January 2008 and discussed what he found with Vas on this forum. You can search on his nick, Anthony C, to see the discussion. Of course this was three months after the match in mexico...


http://rybkaforum.net/cgi-bin/rybkaforum/forum_search.pl?words=Strelka&user=Anthony+C&board=0&field=body&min=&max=&order=desc

It makes interesting reading ... "in my professional opinion Strelka is basically Fritz5 + history reductions + bigger (but not more) mobility/passed pawn terms + super-vasik-material evaluation.  What's amazing to me is how well it works.  I don't know whether to laugh at the silliness of all the speculation surrounding Rybka and how horribly wrong it was, or congratulate you for taking a completely different path than everyone else" (My bold for emphasis) ... and "Which is why I'm surprised no one is talking about the actual source code, which by your own admission is very similar to Rybka 1.0."

In the post of 2008-02-08 20:40 he stated "1. Strelka (Rybka 1.0) is an extremely heavily optimized but otherwise relatively standard (although obviously very effective) chess engine." (My bold for emphsis)

So why did he sign the letter claiming Rybka was a Fruit derivative when at that time in 2008 he did not recognise it as Fruit? In his last post he was referring to Strelka and Rybka as though they were the same and stated it was a relatively standard engine. Based on his professional opinion if he later saw Rybka as Fruit then why did he not state that Fruit must also be a Fritz derivative? If Strelka = Fritz + Rybka then Strelka = Rybka and later Rybka = Fruit?

From my perspective it does not give much confidence in the "professional view"! It seems Rybka somehow mutated from a standard non remarkable chess engine into Fruit without any change of code. Remarkable!

The comment "However, you can take heart that there isn't really anything your competitors can learn from Strelka anyway ;)" ... from my original post ... maybe there was.

The low values for knight, bishop or rook odds did not appear in Strelka 2.0 or earlier versions so this was unique to Rybka and thus the reason for my original post.

PeterG
Parent - - By Charles (**) Date 2013-09-18 10:03 Edited 2013-09-18 10:18

>So why did he sign the letter claiming Rybka was a Fruit derivative when at that time in 2008 he did not recognise it as Fruit?


http://web.archive.org/web/20110521103909/http://www.acoz.net/zappa/rybka-cloning/

The first post in this thread tells the purpose of the above web page:

http://talkchess.com/forum/viewtopic.php?start=0&t=38440&topic_view=flat
Parent - - By Peter Grayson (****) Date 2013-09-18 13:23
Thanks for the links and again very interesting reading but it still does not explain how he could change his professional opinion that Strelka = Fritz 5 + enhancements + Rybka stuff to Strelka = Rybka that was considered standard and nothing special and then the strange transition from Fritz to Fruit based some 2.5 years later. His comments are speculative based on what he believes is or is not possible ignoring that his own engine jumped considerably from Zappa 1.1 to the Mexico versions acording to the CEGT lists that was not too distant in similarity to Rybka.

If the original "professional opinion" statements were without foundation then it gives little scope for credence of the latter statements.

PeterG
Parent - - By Richard Vida (**) Date 2013-09-18 22:55

> opinion that Strelka = Fritz 5 + enhancements + Rybka


The Fritz reference was just a metaphor, meaning "very fast" or "very speed-optimized".
Parent - - By Peter Grayson (****) Date 2013-09-19 10:36
Thank you for clarification. The implication "Fritz-like" puts matters into better perspective.

One curiosity in Rybka 1.0beta is when the rook is removed from the start position, the kN/s jumped by a significant factor but the indicated ply depth also slowed significantly. The indicated node rate was around 520 kN/s in normal start position but with rook removed it displayed 2590 kN/s. How long it stays high seems to relate to how much hash is given. This behaviour is not duplicated by the Zappa engines. I did speculate if the rook removal caused Rybka to bypass the kN/s adjustment to show its true kN/s rate.

PeterG
Parent - - By Rebel (****) Date 2013-09-19 11:17
I am using the patched version that displays the real depth and nodes.

As such R1b gets 2M NPS from the start position and 4M NPS without the rook on a1.

The increase in NPS is due to the Material Imbalance Table and (likely) causes more lazy evals and futility prunings.

To understand the Vas philosophy behind this here are some links.

http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=90083#pid90083

http://rybkaforum.net/cgi-bin/rybkaforum/forum_search.pl?words=winning+percentage&user=Vasik+Rajlich&board=0&field=body&min=&max=&order=desc
Parent - By Peter Grayson (****) Date 2013-09-19 13:28
Thanks for links and again very interesting reading.

Given the change in Zappa's values for the major piece odds start positions does that indicate Zappa made a transition to this method by the time of the Mexico engines?

I mentioned at the time of Rybka's original release these values were considered unique and I had not realised Zappa Mexico had  changed to a similar system until I revisited this a few days ago. Rybka 2.3.2a was the last Rybka engine to display these lower values becasue Rybka 3 on give closer to the traditional major piece pawn equivalence.

Also of interest perhaps is that the Houdini FAQ suggests its evaluation is calibrated to likelihood of a win but must use a different system because the start position piece odds give the more conventional pawn equivalence for the major pieces.

PeterG
Up Topic Rybka Support & Discussion / Rybka Discussion / Unique Rybka evaluation reproduced in Zappa?

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill