Though, perhaps the most important thing is the quality and character of the dissimilar moves.
But it's not only about move choice, it's also about differing in evaluation. Say, Engine 1 suggests move A with a score of 0.12, and Naum says the score is 0.50. It turns out the variations are very dangerous for Black, and indeed, 0.12 is a too shy score. From the root it doesn't make a difference (move A will be played), but from the distance, it may be critical that you see this variation and aim for it, instead of going for one where Engine 1 suggests an early deviation with 0.24 (twice) the score, which is actually worse.
There's also a factor I'd call "move preferences", that depends on what kind of positions the engine wants to reach. Here's a common scenario of critical positions (keeping in mind critical positions are rare and usually are avoided so they don't appear in the actual game):
All engines, when asked for a move, pick Move A. After interactive examination, Move A is refuted. So, the engines are asked to exclude move A, and give their second preference.
Half the engines suggest move B.
The other half suggest move C.
Naum suggests move D.
Here, Naum's score is very different from the other engines, which forces me to try to refute Naum's idea, and what ends happening is that these engines agree that D is better than C or B. I've found the best move.
People may look at the position and think "all engines pick the same move, nothing to see here", but engine's redundancy requires profundity, and the best engine for the job is only apparent when you ask them for moves when you exclude their initial suggestions (say, most engines have the best move in 5th position, but one has it in 3rd position, this engine has a better understanding of it, even if it's not its first move choice).
Here are two games in where I would have had a drawn result instead of won games if I didn't use Naum, as it was leading the games to victory (at some point, when my advantage was big enough, I had to switch off Naum and put in there a better engine to win the games, but what matters is getting there in the first place).
(Core winning idea by Naum)
(Here, Naum was showing the right 0.00 in positions where other engines thought white had an edge, then Naum was showing black with slight advantage while other engines thought there was equality, so it was like having analysis from the future)
- When I tried this with Rybka 2.3.2a back when Rybka 3 was released (and so, 232a had an easier time), 232a failed miserably, either it suggested moves I already had covered, or it suggested "original ideas" that never worked (mainly, passive moves other engines weren't considering, but for good reason.)
- I've never used Strelka.
> Critter is also in the Ippo family
Didn't know that.
> Critter is also in the Ippo family, although the code is original, not "cloned". But almost all of the ideas and formulas of Ivanhoe can be found in Critter, although in some cases they have been improved upon.
I think I know exactly what do you mean by saying this, I even tend to agree (at the level of "ideas"). Some techniques first published with Ippolit sources became "mainstream" since then. By now most programmers treat them just like killer heuristics, null-move etc. However, just by implementing stuff does not make an engine a "family member". I think it is not a good idea to make such statements in a nontechnical forum where your words will be taken out of context and misquoted. Just look at the attitude in this community and all the flame wars.
> Basically, all the strong programs are in one of two "families". One family contains Rybka, Ippo, Fire, Ivanhoe, Houdini, and Critter (with Critter being relatively the most original of the lot), while the other includes Stockfish and Komodo.
Actually, I think the search mechanics used in Stockfish are not too far from the rest, they are just expressed differently.
> Perhaps in the future we will adopt more Rybka/Ippo ideas into Komodo thus blurring the family distinction, but so far that has largely been a failure, for reasons I have yet to understand.
I am sure Don will find a way to make them work ;)
>Some techniques first published with Ippolit sources became "mainstream" since then. By now most programmers treat them just like killer heuristics, null-move etc.
Can you give some examples?
Critter as well as all of the Ippos share certain traits that are not found in Stockfish or Komodo (or any other programs not suspected of being Ippo-derived). They are relatively much stronger at bullet or hyperbullet chess than at slower play, they tend to stick with the same move from iteration to iteration more than do all other programs, the evals don't swing nearly as much from iteration to iteration in general, etc. I would be most interested in your opinion as to which idea or ideas in Critter and Ippo are primarily responsible for this behavior.
I think that what has happened with Komodo is that although we could not make Ippo ideas work for us, we found our own ways to accomplish much of what the Ippo ideas do. So in that sense perhaps I should say that Komodo is intermediate between the Stockfish family and the Ippo family.
> ... or copy PST or other eval numbers it makes the program a derivative (in some sense anyway) of the other one, and would make Critter ineligible to play in ICGA events, for example.
Please don't get me started about PSTs :)
I used PST values from Ippolit just because of lazyness and never tried to hide this fact. I don't really care about ICGA rule #2. But do you seriously think Komodo3 PST values would be seen as original in a case of Rybka1.0 style ICGA investigation? AFAIK Komodo builds them using similar formulas like fruit/rybka/ippolit, with the same ideas of file/rank ramping and centralization, albeit with different constants/weights. I think that inventing new PST formulas/weights simply does not worth the effort if done just for the sake of perceived 'originality'.
> ... the evals don't swing nearly as much from iteration to iteration in general, etc. I would be most interested in your opinion as to which idea or ideas in Critter and Ippo are primarily responsible for this behavior.
I thought this is the normal/desired behavior. Instead, I would be curious what causes Stockfish evals swing up and down like crazy.
> ... So in that sense perhaps I should say that Komodo is intermediate between the Stockfish family and the Ippo family.
Not very wise choice of wording. Some people are very eager to misinterpret such quotes ;)
Regarding the swings in value from iteration to iteration, I quite agree that the behavior of Critter, Rybka, and the Ippos is indeed desirable behavior. But it is my opinion that the relatively "wild" behavior of Stockfish is the way all programs behaved before Rybka, and is still the way most programs other than the above ones behave. So I think there is nothing to explain about Stockfish; my question is what is the idea in Rybka/Ippo/Critter that "cured" this formerly universal problem? As for Komodo, I think it is in between Stockfish and Critter in this specific area.
You are probably right about my wording. The only way to avoid being misinterpreted is to shut up!
Anyway, good luck with Critter. I mean that sincerely; while of course I hope Komodo can stay on top, I would be very pleased if both Komodo and Critter pass Houdini in the near future. It would be nice to demonstrate that "copy and paste" is not the key to success in this field.
Look at what happen to Rybka....
> Critter as well as all of the Ippos share certain traits that are not found in Stockfish or Komodo (or any other programs not suspected of being Ippo-derived). They are relatively much stronger at bullet or hyperbullet chess than at slower play, they tend to stick with the same move from iteration to iteration more than do all other programs, the evals don't swing nearly as much from iteration to iteration in general, etc. I would be most interested in your opinion as to which idea or ideas in Critter and Ippo are primarily responsible for this behavior.
> I think that what has happened with Komodo is that although we could not make Ippo ideas work for us, we found our own ways to accomplish much of what the Ippo ideas do. So in that sense perhaps I should say that Komodo is intermediate between the Stockfish family and the Ippo family.
See, there you go again! You are subtly lumping Critter in with the ippolit engines and in a thinly veiled attempt to make you and Don look like some shining white knights.
As I have said before you should be proud that you have been a major part of the development of two great and strong engines (Rybka and Komodo) but don't pull this silly word play like some 2nd rate politician.
Just come on here, state that Komodo 4 is ready and that you and Don believe that it can make a significant contribution to anyone's computer analysis studies and work. Don't bad mouth others.
Why is pointing out similarities of Critter and Houdini a criticism? Anyway the point of my question is to better understand the cause of the behaviors in question, so as to use this knowledge to improve Komodo. Houdini and Critter do some things better than Komodo, while Komodo does other things better than they do. If I can understand why they are better at certain things (such as smoother eval changes) we can probably make Komodo stronger in those areas.
Since it seems that Komodo catches up a little over Houdini the longer the time control is.... Can we expect to see the MP version of Komodo 4 even closer to Houdini ?
>See, there you go again! You are subtly lumping Critter in with the ippolit engines and in a thinly veiled attempt to make you and Don look like some shining white knights.
This is like, severe "drama-queenism" for lack of a better term.
Larry is speaking entirely objectively, it's got nothing to do with any controversy past or present. It's not bad mouthing or berating, it is a frank analysis from someone with some pieces to the puzzle.
Plus Larry and Don disagree on some fundamental points of the Vas issue anyways so it would be inaccurate to loop together their "political" views anyways.
>> See, there you go again! You are subtly lumping Critter in with the ippolit engines and in a thinly veiled attempt to make you and Don look like some shining white knights.
> This is like, severe "drama-queenism" for lack of a better term.
> Larry is speaking entirely objectively, it's got nothing to do with any controversy past or present. It's not bad mouthing or berating, it is a frank analysis from someone with some pieces to the puzzle.
> Plus Larry and Don disagree on some fundamental points of the Vas issue anyways so it would be inaccurate to loop together their "political" views anyways.
Actually if you look at the history of the posts it isn't until I called him on it that Larry qualified what he was saying. In his first posts he was trying to subtly lump Critter with the ippolit engines without much of an explaination. Even Richard Vida felt the need to speak up.
After calling him on it Larry did indeed give very detailed and objective reasons for his thoughts. Which he should have done in the first place.
> Larry is speaking entirely objectively
No, Larry himself accepted that his views on the Houdini-Critter connection are subjective.
Trust your health is fine now Larry.
> 2) Houdini has got nothing to do with Fruit. Zero, zilch, nada.
But there might be the line: Fruit-Rybka-Ippolit-Houdini.
Then Houdini and Rybka4.1 might have a similar relationship to Fruit.
> But there might be the line: Fruit-Rybka-Ippolit-Houdini.
Fruit and Ippolit are open source engines, it should be very easy to demonstrate by direct comparison that Ippolit is related to Fruit.
Please do so.
> Don, the author of Komodo seems to think there is still around 1000 ELO to be gained by engines. That is a mind boggling number and I am not so sure it is accurate. I have no doubt that Vas has a Rybka that can run circles against any engine out there, but wanting to release it to the public is a totally different issue. If you look at how fast Houdini was quickly RE, then you really can't blame Vas to not want to have to go through another Ippo debacle. The best possible scenario for us is if a secure hardware card is possible to port an engine to. That would be great as it would give us the best possible engine strength while still keeping the code safe.
100% agree, excellent post. The only thing I'd add is an FPGA card instead of a security hardware card so we get more processing power! :)
You guys are funny. Last I checked, chess was just a board game.
> And maybe we should add iris and voiceprint scanners to the Aquarium startup sequence.
> You guys are funny. Last I checked, chess was just a board game.
No, no, no. Chess is sooooo much more than just a board game. I like your list but I'd add fingerprint recognition as well - just to be safe.
I do think that FPGA cards have some possibilities with regards to specialized modules ... maybe endgame modules or real time MC generation modules. If you could put all the information of a rook endgame book on an FPGA card and you consider than more than 10% of chess games are rook endgames, it would be a cool thing to have and probably worth some good ELO points. As it is now, there are still a lot of "easy" ELO points to have by simply improving the evaluation and efficiency of today's engines ... but IMHO that will end in about 200 ELO points.
So, yeah, from time to time, what I perceive as an extreme lack of perspective on particularly this, but also on other fora (including mine), drives me a bit crazy. Despite what Vincent Diepeveen implies, no one is making weapons-grade chess software that can hack into foreign bank accounts and pull satellites out of their orbits. Anand's WC prep won't change the course of the Middle East peace process, nor will Kasparov's laptop halt the accelerated melting of glaciers. Rybka's source code won't help determine who sits in the White House or the Kremlin.
Sure, lots of people around the world play chess, some quite seriously, and there's plenty of money to be made off of them. But that's the only value locked up in these secret sources, and there's probably comparatively more profit to be made in developing and hosting playing servers than in analysis software or platforms.
So ultimately, all the secrecy, dedicated hardware hosting, paranoia, etc. is fairly silly, at least as it's justified by people around here: if you think that chess is much more than a board game or a competitive sport played with same, if you think that chess software developers are geniuses contributing to the expansion of the human spirit and the further development of the human mind, than you'd also have to insist that their findings be publicized, and not secretly hoarded for base profit motives. If you agree that it's just (small) business, the mystique is out of place.
But your point is taken, and I shouldn't make fun.
Point of information: According to Richard Vida (Critter) Houdini was quickly RE'd because it's so closely based on Robbolito that there was a simple open-source roadmap for REers to follow.
>There are so many options, if the goal is simply to protect Vas' IP. Here's just one.
That toy will protect nothing.
>Don, the author of Komodo seems to think there is still around 1000 ELO to be gained by engines.
To be better by 1000 elo, you have to win at least 99% of the games. And if you talking vs houdini on a 12 core today on classic time controls, I think it is very unlikely.
Now, over the course of games between these two entities, we know A will never make a move that surrenders a half-point. (An engine that did play less than perfectly in order to get an opponent into positions he would definitely make a fatal move would be even stronger yet, but let's set that aside for this discussion.)
Assuming the game of chess is drawn, how many moves would Player B get into the game without giving up a half-point, i.e. going from a drawn position to a losing position?
Let's say it can make 12 moves before it makes a move that loses. The challenge for the programmer is finding a move that holds the draw. Until it can do that the ELO difference between A and B is only limited by the ELO formula. In practicality, the ELO difference is infinite. If Player B always loses, every single time, then there can be no comparison.
Let's suppose Player B does find a drawing move on move 13. OK then, how many more moves can it go before it makes another losing move? And now I repeat what I just said above.
The point is that today's Player B (the strongest chess entity in the world) may make any number of objectively losing moves over the course of a full game. In order for ELO to be equal with Player A, every single one of those losing moves has to be replaced with a drawing move, otherwise the ELO difference is infinite. It isn't a question of relative ELO; at the true extremes it is a question of draw or lose. It is pass-fail. ELO doesn't enter into it.
Now, needless to say, if there even existed a Player A then Player B, C, D...Z would all rapidly improve. Because Player A would reveal, though its moves, the correct path in every position. That information would be invaluable to the competition, and in due course there would be many engines who, through trial-and-error, could find ways to draw Player A. At that point the ELO measurement would become relevant again.
Unfortunately, I have no good idea how to guess the probability that Rybka or Houdini play a game without error. I only see an upper limit for this probability. For example, if it was 90% then the draw rate between these programs would be greater than 81%.
But this logic works only in one direction. Even if we observed a draw rate as high as this, we could not conclude anything about the error probability.
Maybe an expert of Bayesian probability is reading this and can help us.
Why? random eval is kin to mobility. If you do a minimax (alpha/beta) search, you will evaluate positions where you have more mobility as a higher value, because with more moves out of that node, you have a higher chance of getting a larger random eval back. Fewer options reduce this. So a random player will never be a sub-1000 level player. To make this happen in Crafty, I had to modify the skill command to burn cycles in addition to the random stuff, to get it down to the 1000 level.
One day a random eval will hit GM level or beyond, which will be strange and interesting at the same time.
You cannot assume a perfect player will never lose, simply because we don't know whether white, black wins, or whether chess is a draw.
> Don, the author of Komodo seems to think there is still around 1000 ELO to be gained by engines.
Imagine there suddenly would be several Engines playin some kind of ideal-perfect chess.
And the work among humans.
Which ELO-rating might get reached then by those Engines?
Is Vas really willing to spend much energy, heart's blood and time in Rybka-Development today?
I'm not convinced that this is fact. :-(
Maybe during the last months things happened in a very disappointing, demotivating way for Vas.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill