Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Why Rybka 3 is not acceptable
1 2 3 4 Previous Next  
- - By Permanent Brain (*****) Date 2008-08-01 19:18 Edited 2008-08-01 19:43
1. At least a half dozen times, it has been promised that the endgame knowledge which was removed from Rybka 2.2 to 2.3.2a (the most stupid, weirdest decison in computer chess history) will be BACK in Rybka 3.

You didn't keep the promise. According to results which have been posted, R3 is an IDIOT in blind bishop positions. I will remember this for future versions & promises. I will never expect that any of your announcements are reliable.

2. You call a primitive, non-specific randomized eval function "Monte Carlo Analysis" which is a JOKE compared to true, scientific MC. approaches. Shame on you. You fail to respect serious approaches to forward board game software strategies, by calling a rediculously stupid feature "Monte Carlo Analysis" which doesn't deserve this name.

3. You enter a top computer chess competition but you don't know about particular problems of an up-to-date CPU. That tells me that Rybka has NOT been tested on a variety of different computers, with different CPUs etc. which is a VERY SIMPLE BASIC REQUIREMENT for any mass market software! Quality assurance is absolutely required if you don't want to sell a piece of crap.

You failed all these tests, 1, 2 and 3.

Rybka 3 is NOT an acceptable product in terms of quality, and not in terms of taking care for the customer.

I have understood (as somebody who can read between the lines) that commercial success is NOT your main goal anyway. But it is NOT FAIR to keep this dedication secret. Vas, speak the truth. What are you going for? I think you do NOT care for anything what a normal chess player, your customer, may want. You do NOT care for details which do not bring Elo points. You do NOT keep promises like putting the blind bishop knowledge back in.

Are you sure that commercial computer chess software business, where the CUSTOMER should be the main factor and not the programmer, is really your best environment to work in? My impression is that you are much too careless and too sloppy.

I can live very good without Rybka.
Parent - - By pawformation (*) Date 2008-08-01 19:49
What is your address? I will send you a box of tissues and some cheese.
Parent - - By Permanent Brain (*****) Date 2008-08-01 19:53
And I will send you the address of a psychiatrist in return.
Parent - - By jejega (**) Date 2008-08-01 22:44
We know Freud was austrian like you. Do you have his address?
Parent - - By Blacknight (**) Date 2008-08-06 04:31
Jejega,

Sigmund Freud was from the Czech Republic.
Parent - By jejega (**) Date 2008-08-06 08:14
Yes but at this time there was not such a thing like Czech Republic. It was all part of the Austrian Empire. That's why he is considered Austrian rather than Czech.
Parent - - By Geomusic (*****) Date 2008-08-01 19:58
"You didn't keep the promise. According to results which have been posted, R3 is an IDIOT in blind bishop positions. I will remember this for future versions & promises. I will never expect that any of your announcements are reliable."

You got to be joking! A 3250+ ELO program is an "IDIOT"?. A program rated 200-300+ higher than its next highest competitor? A program that others try to reverse engineer?  A program that trumps even its previous version by no less than +200 ELO points. You my friend sorry to say are the "IDIOT" not the program.

"You call a primitive, non-specific randomized eval function "Monte Carlo Analysis" which is a JOKE compared to true, scientific MC. approaches. Shame on you. You fail to respect serious approaches to forward board game software strategies, by calling a rediculously stupic feature "Monte Carlo Analysis" which doesn't deserve this name."

And you would know in fact what Scientific Monte Carlo Analysis was because you looked it up for 5 mins on Wikipedia? You haven't the right nor the jurisdiction to levy out insults to Vas and Larry. When you learn of all the work they have done you can talk(just kidding we don't want to hear it)but until then go home.

You my friend have just hurt your own credibility and our intelligence and quite frankly the hate of millions of happy rybka fanboys so it is you who have failed so now it's time to hang your head low and just go and please don't let the door hit you in the "%$#" out the door.

Best Regards,

Mike
Parent - - By theoak (**) Date 2008-08-01 20:04
I might agree with you, but I think this "millions Rybka fanboys" sounds a bad joke. I mean computer chess as a whole does not have so many fans out there :)
Parent - By Geomusic (*****) Date 2008-08-01 22:58
lol the reply was a bit in jest but had some teeth in it.
Parent - - By Permanent Brain (*****) Date 2008-08-01 20:23

> A 3250+ ELO program is an "IDIOT"?.


I was talking about a particular situation, not about an average from all positions.

Btw., what about other flaws in 2.3.2a which have been detected in comparison to 2.2? RESPECT FOR THE CUSTOMER means that there is

1. Clear documentation about all these flaws and how they have been fixed for Rybka 3, and
2. a set of examples with engine output from Rybka 3, to prove this points.

These are VERY SIMPLE REQUIREMENTS any newbie of software distribution and serious(!) advertisement will understand.

Rybka's developement, quality assurance and advertisement is NOT PROFESSIONAL.
Parent - - By Geomusic (*****) Date 2008-08-01 22:57
lol :)
Parent - By Geomusic (*****) Date 2008-08-01 23:48
mmm much better with those two out of my hair...
Parent - - By skulibj (*) Date 2008-08-02 02:58 Edited 2008-08-02 03:07
Im a software engineer and have read alot on this forum so I think my opinion should have some value.

"Rybka's developement, quality assurance and advertisement is NOT PROFESSIONAL"
Those things could of course be better but their resources are limited.
How many employees are programming the engine? Only one isnt it, not a whole team?

So it is really understandable that compromises need to be done, even though they can be painful.

It is my feeling that the development of Rybka is of the highest standard. I dont know the definition of monte carlo analyzis. I would think that this could be called monte carlo analyzis. Just a very rough one.
Parent - - By plicocf (***) Date 2008-08-02 09:04
Skulibj, I do not know exactly what you're talking about, because I am not a
professional.

The fact is that on 05/01/2006 I received from Vasik Rajlich
a message with links to download Rybka 1 Beta until Rybka 1.2.

What impresses me is that today similar message will be send by Convekta,
and we will receive a DVD by Chessbase and Convekta, and Vasik Rajlich continues
selling his software through the Rybka site.
It seems to me that an opening book never was sold without the engine, another novelty.

The most incredible is that I bought the same engine two times, one from Convekta
and one from Chessbase. I also bought Jeroen opening book.

In your opinion, what could happen if they were professional?

Paulo Soares
Parent - - By skulibj (*) Date 2008-08-02 21:57
What are you unhappy with and how would you want to fix it?
Parent - - By plicocf (***) Date 2008-08-02 22:29
I am not sure, but I think that you didnĀ“t understand my message.

Paulo Soares
Parent - - By skulibj (*) Date 2008-08-03 19:20
No I didnt understandi it completely and I dont know too much about the practices of Convecta and Chessbase, and not of Rybka either. I just know that Rybkas team is small so they must be professional, because of the strength of their engine. Maybe that is a wrong assumption but it at least sounds likely. Mayby they should do much better in some areas. But they have no employees that specialize in marketing or sales strategy, so if they make some mistakes there that is not a surprice.
Parent - - By Uri Blass (*****) Date 2008-08-03 19:52
It seems to me that there was a misunderstanding here when everyone thought that the second side complained when it is not the case.

"Rybka's developement, quality assurance and advertisement is NOT PROFESSIONAL" are not the words of you but a quote from the post you responded to.

I am not sure if I understood the discussion and I may be wrong but
it seems to me that paulo thought that these are the words of you and you thought that he criticized rybka for not being professional and both of you are wrong.

Uri
Parent - By skulibj (*) Date 2008-08-04 01:20
thanks
Parent - By plicocf (***) Date 2008-08-04 02:31
You're right, Uri, I thought that the sentence had been written by Skulibj.

Paulo Soares
Parent - By InspectorGadget (*****) Date 2008-08-04 07:07
Mike, I know what you are talking about, it is irritating for a program to display a +3.00 score in a dead draw situation like that one of the blind Bishop. If someone does not raise these issues, nobody will fix it. I wonder if underpromotion to a Bishop has been included.

I wonder why some people are making fun out of your post. No wonder someone called the forum the "unwashed nuckle-dragging masses.."

Someone must really look at these things for customer satisfaction.
Parent - - By turbojuice1122 (Gold) Date 2008-08-01 22:01
Actually, the points he brought up are quite correct and valid.  Of course, he should have been a bit more diplomatic in stating them, as you also should have been--if you had been in this forum for a much longer period, you would know that he is quite intelligent and not just a random troll.

He didn't say the program was an idiot--he said that it was an idiot in evaluating that specific type of endgame.  Opponents can take advantage of this knowledge and thus tune opening books to lead to it, so it is a potentially major problem.  Also, the program is nowhere near having a true Elo level of 3250 calibrated to FIDE--perhaps 3150 on a Skulltrail is more like it.

He is also correct that this is nothing like Monte Carlo analysis as the term is used in computer simulations.  It aims to do much better than that, but due to the nature of chess, I think that the result is a bit worse.  True Monte Carlo analysis will give the absolute truth of a particular result, and the only way this can even be closely approximated is if all of the lines with results that can be avoided are tossed out of the situation, among many other things.

On the other hand, ALL engines are going to have problems and certain types of positions they play extremely poorly.  We see them as more apparent with Rybka because it is probably studied by more intelligent people on a higher level and in a larger variety of situations than any chess engine or system in history.
Parent - - By George Tsavdaris (****) Date 2008-08-01 22:26

>Actually, the points he brought up are quite correct and valid. 


Is his 3rd point valid??
Did Vas ought to know the bug on the AMD Phenom PCs??

Also one of the reasons Mike(Permanent Brain) said he would not buy Rybka is that Vas didn't know about the bug in Phenoms(so that means he does not test in many different hardware, so he is no real professional, etc)
And what about SMK? Did he know about the bug? Apparently not. But Mike said he bought Shredder 11.
Don't you see some kind of contradiction here?

>He didn't say the program was an idiot--he said that it was an idiot in evaluating that specific type of endgame.  Opponents can take advantage >of this knowledge and thus tune opening books to lead to it, so it is a potentially major problem.


I would be extremely surprised if that could be true.
I bet that no book cooker can even manage to lead to such endgame with his lines in 1 out of 1000 games.
Opening from endgame is far away and no book creator can control the type of endgame that would be played.

I would personally don't like to wait more about the release of Rybka 3 for fixing this bug if it takes more than a week.
You would be surprised what Chessbase answered me of the cause of the delay(i'll give it to you with PM) and i wouldn't like even more delay.
And after all, there is one word called: "UPDATE".
Vas said there would be no updates to Rybka 3 but surely if he breaks that word, no one would be unhappy about that unfulfilled promise. :-)
Parent - - By turbojuice1122 (Gold) Date 2008-08-01 22:29
You are correct: there is no reason why Vas should have programmed around buggy Phenoms--I forgot to address that point.

As for opening book cookers, I think that all that is required is a titled player who is very well-rounded in opening theory to exploit this hole.
Parent - - By lkaufman (*****) Date 2008-08-01 22:54
Are you saying that someone would put in his book some line that leads to a material down but drawn endgame in the hope that the Rybka book author would aim for this line based on faulty endgame eval? That would be both incredibly difficult and unlikely to work, given that the book author is a competent player. Maybe I misunderstand you, but it seems almost impossible for a book author to actually benefit from the knowledge that Rybka (unaided) might misevaluate a certain specific endgame.
Parent - - By turbojuice1122 (Gold) Date 2008-08-01 23:18
I am saying this based on the knowledge that there are many large groups of openings that tend to lead to certain general types of endgames, and a book author so inclined (or who already possesses this knowledge) could find, based on statistical analysis, which openings are more likely to lead to that particular type of endgame (or if he already has this knowledge, tune it in at the beginning).  This won't work in every game, of course, but it will increase that program's performance relative to Rybka compared with what it should be based on their relative elos.  The size of this increase is what we don't know.  I have a feeling that Erdo could probably make a non-negligible increase in Zappa's strength by cooking his book this way against Rybka.
Parent - By lkaufman (*****) Date 2008-08-01 23:49
If you are talking specifically about certain endgames that might reduce to B+wrong RP, I would be very impressed if Erdo or anyone could gain even one Elo point by knowing about this (remember, Rybky 3 does know that the basic B+wrong rp or rook pawns is drawn). Now if let's say it turns out that Rybka 3 is generally weak in rook and pawn endings, that might be exploitable.
Parent - By Permanent Brain (*****) Date 2008-08-02 09:26
I bought S11 (SE) some weeks ago. Also, I think Phenoms didn't even exist yet when S11 was released. But that's not the point anyway.

I didn't mean the there should be a "programming around" this bug but that Convekta should do prerelease testing (incl. benchmarking) on current CPUs and OS, like Phenom+Vista SP1. I want to point to the logical assumption that without the Chess960 tournament in Mainz, it would be some the individual Phenom + Rybka users who have this problem, at least those who are not familiar with the TLB bug.

> Vas said there would be no updates to Rybka 3 but surely if he breaks that word, no one would be unhappy about that unfulfilled promise. :-)


Yes... but also, I could add that "no updates" announcement to my list of critical points.
Parent - - By Geomusic (*****) Date 2008-08-01 22:42 Edited 2008-08-01 22:46
"Actually, the points he brought up are quite correct and valid.  Of course, he should have been a bit more diplomatic in stating them, as you also should have been--if you had been in this forum for a much longer period, you would know that he is quite intelligent and not just a random troll."

He isn't being too intelligent right now. Since he is IN-FACT instigating a flame war, not being at all fair to the man-hours put into Rybka and on top of it all being a big cry-baby.
Parent - - By turbojuice1122 (Gold) Date 2008-08-01 22:44
All you're doing is restating your conclusion as a supposed piece of evidence...
Parent - - By Geomusic (*****) Date 2008-08-01 22:53
evidence ? Rybka is +200 elo stronger than 2.3.2a there is your evidence of their hardwork!
Parent - - By turbojuice1122 (Gold) Date 2008-08-01 23:14
I know they did hard work.  Mike's point is that it wasn't with the customers in mind.  I don't know enough to comment on this.  You tried to make the point that he is a troll and cry baby.  You are very wrong--he gave thorough explanations for his complaints (though some of the explanations are not valid, I think).
Parent - - By Geomusic (*****) Date 2008-08-01 23:16
I don't try anything I succeed. They might have been "thorough" but they weren't valid.
Parent - - By turbojuice1122 (Gold) Date 2008-08-01 23:19
You didn't succeed in making a valid point--you were simply wrong.  Some of his points were valid and well-explained, while others were, I think, invalid.
Parent - By Geomusic (*****) Date 2008-08-01 23:43
incorrect again. This is getting boring, I explained originally that it was a poor choice of words for him to come down on Rybka the way he did and as such he was metaphorically termed a "crybaby"
Parent - - By Geomusic (*****) Date 2008-08-01 23:46
Well that is his own warped view of course not a consensus...
Parent - By turbojuice1122 (Gold) Date 2008-08-02 01:21
It's not a warped view--it is a perfectly valid viewpoint.  I don't agree with everything that he said, but it is supported and overall is a fine post, not a troll post as you have labeled it.  This reply is also valid for your other reply.  Yes, this is getting boring--if you can argue with facts, then do that--but you have done none of that, as all you ever did was misstate his points and then come down on your own misstatements.
Parent - - By Permanent Brain (*****) Date 2008-08-02 09:48 Edited 2008-08-02 10:38
Do you see Rybka only as a toy to gain computer Elos?

Of course, the big gain of playing strength is a great achievement. I didn't doubt that hard and good work has been done; that's obvious.

But if the analysis in a blind bishop position is wrong (i.e. suggesting a move which spoils a win), then it doesn't matter if the rest is +200 Elo or whatever. Rybka 2.3.2a also had flaws in some pawn endings. I am insecure about what to expect from Rybka 3 now, related to that.

I would have expected that the developers had collected all these examples, as posted here and elsewhere, to cross-check the fixes/improvements one by one, and provide examples of them in a short "What's new" documentation. This is NOT something to be left over for some Rybka fans to do for fun, after the release. It should be done by the developers and/or other team member, BEFORE the release, including publishing all relevant examples.

Maybe I expect too much? But to me, this seem very obvious steps to do, for the release of a new software version.

In chess software, one advantage is that once particular improvements have been done (where it's not about playing hundreds of games to find a higher Elo) , they can be easily demonstrated by comparing old/new in the same positions.
Parent - By turbojuice1122 (Gold) Date 2008-08-02 14:12
Vas gave a sort of "what's new" post about two weeks ago, though it didn't have everything that is new.  Rybka 3 hasn't been released, so there is still time for such documentation.
Parent - By garyf919 (**) Date 2008-08-03 01:09
Maybe we should all see Rybka 3 for ourselves before this thread goes on much more.
How are you so certain that your assumptions are correct before you have see the finished
product? Did you have a beta version or the final release before it was released? I would be
more angry about that if this were true? :) And, even if all you say is true, how do you know
what the authors circumstances were before this release? It seems to me that they are in time
pressure and making last minute changes before release. This, for a "software professional",
(which I am by the way), is more concerning. This might backfire more than a missing requirement
that couldn't make the release. I cetainly hope that the support for Rybka 3 will be as good
as the program itself!
Parent - Date 2008-08-01 22:44
Parent - - By InspectorGadget (*****) Date 2008-08-04 07:18
mburch1974, I understand Mike's anger on the Blind Bishop knowledge. There's been many posts here on with the Blind Bishop, which Rybka cannot evaluate correctly, and apparently some version had that knowledge of a blind Bishop. So that knowledge could have been brought back into this new version.
Parent - By Geomusic (*****) Date 2008-08-04 17:35 Edited 2008-08-04 17:39
If he is so upset about it, perhaps, someone could send him  condolences in a sympathy card with lots of hugs and kisses. Balloons?

Regards,

Mike
Parent - - By George Tsavdaris (****) Date 2008-08-01 20:00
Your last 2 posts(I & II) came as a surprise to me. They were very uncharacteristic of you.
It's like someone hijacked your password for Rybkaforum. :-)

>1. At least a half dozen times, it has been promised that the endgame knowledge which was removed from Rybka 2.2 to 2.3.2a (the most >stupid, weirdest decison in computer chess history) will be BACK in Rybka 3.


Why do you believe it was the most stupid decision? Perhaps the removal gave to Rybka some ELO points and if they haven't been removed then it may play better in some endgames but the overall loss would be higher.

>You didn't keep the promise. According to results which have been posted, R3 is an IDIOT in blind bishop positions. I will remember this for >future versions & promises. I will never expect that any of your announcements are reliable.


I would agree with you if you remind me and show the promise that the knowledge would be back in the Rybka 3 version.

>2. You call a primitive, non-specific randomized eval function "Monte Carlo Analysis" which is a JOKE compared to true, scientific MC. approaches. >Shame on you. You fail to respect serious approaches to forward board game software strategies, by calling a rediculously stupic feature >"Monte Carlo Analysis" which doesn't deserve this name.


Why do you think it does not deserve the name?
I agree it's not the true Monte Carlo method or anyway anything particularly groundbreaking, but still it may be useful and the name is not important i think.

>3. You enter a top computer chess competition but you don't know about particular problems of an up-to-date CPU. That tells me that Rybka >has NOT been tested on a variety of different computers, with different CPUs etc. which is a VERY SIMPLE BASIC REQUIREMENT for any mass >market software! Quality assurance is absolutely required if you don't want to sell a piece of crap.


As far as i can tell all the other 3 programmers didn't know about the bug in AMD computer either.
In fact why Vasik should have known about it i can't find any reason.

Also your implication that for this reason Rybka hasn't been tested on many different computers, is not a correct one obviously.
It may not have been tested in many different but it will not be due to the reason you said.

>Rybka 3 is NOT an acceptable product in terms of quality, and not in terms of taking care for the customer.


Then you should not buy it if you believe this.

If you set your standards of the quality of a Chess program to be characterized by:
-It can't recognize blind Bishop draws.
-It has a unique feature that no other program has it, but the name is not appropriate!
-The programmer of the engine does not know about all bugs of all different PCs.

Then be my guest to believe that a program to be good has to pass the above 3 tests.
I have other standards it seems, and the above facts i consider them pointless except the first which needs to be fixed.

>Are you sure that commercial computer chess software busineness, where the CUSTOMER should be the main factor and not the programmer, is >really your best environment to work in? My impression is that you are much too careless and too sloppy.


So what do you suggest him to do?
Parent - - By Geomusic (*****) Date 2008-08-01 20:03
wow! George, you are too nice to this guy...
Parent - - By Harvey Williamson (*****) Date 2008-08-01 20:11
If someone wants to be critical of Rybka I do not think we should all jump on them just for that reason. I think George is trying to answer the points made and not just shout at the guy for daring to be critical.

I have one question have the reported KNs and depth been corrected for R3? I am sure it was promised somewhere that this would happen.
Parent - - By turbojuice1122 (Gold) Date 2008-08-01 21:13
Yes, I also recall that it was definitely stated that this would be corrected at the end.
Parent - - By George Tsavdaris (****) Date 2008-08-01 21:21

>Yes, I also recall that it was definitely stated that this would be corrected at the end.


Corrected to what?
As i recall Vasik supports that the nodes searched by the child processes are not counted so that's the reason for the low node count.
Why do you want them to be counted these child nodes also?

About depth, if the actual is equal to reported minus 2(or 3) then it should be fixed of course as i see no real reason not to report the correct depth.
If Vasik of course believes there is a reason then i don't see anything bad with not correcting it.
Parent - By BB (****) Date 2008-08-01 21:41

>then it should be fixed of course as i see no real reason not to report the correct depth.


Aha! But VR and LK have not been unknown (err, perhaps I shouldn't use this circumlocutory verbiage with non-native speakers) to promote the idea that there is no such thing as a "correct depth"... :-P

>As i recall Vasik supports that the nodes searched by the child processes are not counted so that's the reason for the low node count.


I don't recall this exactly. My recollection was that there was some renormalising (of the already subterfuged) node counts in the MP version, so as to estimate better the "real speed-up". For instance, rather than given 4 times the number of "nodes" searched by one CPU, the display would be something like 3.6 times this (see link below, with the "expected benefits').

See http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=5325

>Rybka 2.3 counts nodes differently on multi-processor machines.
>Rybka 2.1 (and also other engines) just count nodes from all children, whether those nodes are useful or not.
>Rybka 2.3 counts only the parent-process nodes, and adds the expected benefits of having "X" child processes. This lets you compare nodes-per-second rates between machines with a different number of cores.
>The problem of course is a psychological one - many users will think that the engine isn't able to exploit the multiple processes properly. It might be better to just count all nodes by default, and allow expert users to choose the 2.3-style scaling.

Parent - - By lkaufman (*****) Date 2008-08-02 00:06
Regarding kNs, Vas decided that he was happy with the way he counts. It might not be the industry norm, but he considers his way more valid. If you want standard count, just add a zero and that should be close enough. Regarding adding two or three to the depth, he planned to do this but apparently changing it was not completely trivial and there was no time to test to make sure no bug was introduced. However the depths below 1 ply can now be set using the UCI limit Elo feature (I hope this is documented) so the main practical reason for the proposed change is no longer applicable. It should also be pointed out that because the ply to ply multipliers are less than in standard programs, even though a Rybka 1 ply search is much better than a Fritz (for example) 1 ply search, a Rybka 20 ply search might not be superior to a Fritz 20 ply search, so adding two or three to the count might be criticized as exaggeration on the upper end of the range.
Up Topic Rybka Support & Discussion / Rybka Discussion / Why Rybka 3 is not acceptable
1 2 3 4 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill