![[se]](/mwf/v12/flags/se.png "Sweden")
<"All improvements I make are thoroughly RE'd by the likes of Richard Vida and Yuri Osipov">
Ah yes, you get a taste of your own medicine
KARMA! and I don't even believe in that crap: maybe I should reconsider
Ah yes, you get a taste of your own medicine
KARMA! and I don't even believe in that crap
: maybe I should reconsider
> Ah yes, you get a taste of your own medicine
I don't RE other engines, so I'm not sure what you're alluding to.
"I don't RE other engines, so I'm not sure what you're alluding to."
I was refering to your own qoute about <taking improvements> not engines.
I do believe this distinction is non existent by most definitions in your own case but that's a seperate discussion.
I was refering to your own qoute about <taking improvements> not engines.
I do believe this distinction is non existent by most definitions in your own case but that's a seperate discussion.
Note also that in the current environment I see little point in Houdini being more than 50 points stronger that the competition. All improvements I make are thoroughly RE'd by the likes of Richard Vida and Yuri Osipov, then discussed on various forums (mostly by Richard), implemented in Critter or Strelka, or passed on to the Komodo or Stockfish team through the Critter source code Richard so generously distributes. It's a strange world :).
At least Richard gives you credit for having good ideas, while he's revealing them. I always get a good chuckle when I read that Komodo is trying all of these ideas, but that none of them work for them, so they aren't being used...
At least Richard gives you credit for having good ideas, while he's revealing them. I always get a good chuckle when I read that Komodo is trying all of these ideas, but that none of them work for them, so they aren't being used...
![[us]](/mwf/v12/flags/us.png "United States")
Larry, I hope when it comes to Komodo's MP version you're not going to fall back on voodoo ratings when it comes to measuring its performance!
P.S. This is starting to look like a lot of hype. (Maybe it's not too late for Don to just start copying some code.eh!)
P.S. This is starting to look like a lot of hype. (Maybe it's not too late for Don to just start copying some code.eh!)
I consider the CCRL 40/40 list and the CEGT 40/20 list to be the gold standard of computer chess ratings. The 40/120 list would be best, but by the time an engine gets rated with enough games there a new version will be coming out. The IPON list has the best conditions regarding uniformity of opposition, books, hardware etc. but it uses a blitz time control so I would say it deserves priority over the CCRL and CEGT blitz lists, but not over their 40/20 and 40/40 lists.
So to be clear, I will claim Komodo to be the strongest engine only when its average rating on those last two lists exceeds the average rating of other top engines. Fair enough?
So to be clear, I will claim Komodo to be the strongest engine only when its average rating on those last two lists exceeds the average rating of other top engines. Fair enough?
![[gb]](/mwf/v12/flags/gb.png "United Kingdom")
Doesn't work for me Larry. When the new version of Komodo performs better than Houdini in the majority of positions I give them to analyse then I would be the first to say Komodo is 'stronger' at chess than 'Houdini' - all a ranking list gives you is a prediction of how engine X will perform against engine Y over a number of games played - not some absolute value of strength - for that you need to see what the engine plays in a given position and at the moment, Houdini still outperforms Komodo as far as I'm concerned - perhaps when you launch K4.1 or K5 things will change.
Well I'd be happy to go by such a metric if everyone agreed on which positions to use and how to score them. But of course there is no agreement; probably Houdini will do better on more tactical positions and Komodo on more quiet ones. The problem is that it is much harder to prove what is the best move in a quiet position. We do have a huge set of 150,000 positions I could use to compare them; I haven't done this in a long time but perhaps I will try to resurrect it. But as far as an acknowledged standard way to compare engines is concerned, I don't know of a better way than CCRL/CEGT 40/40 and 40/20.
If this is info you can provide and you don't have to give names, but outside of yourself and Don. Is there other people who help you beta test Komodo SP and MP? When reading some of the comments it seems as if you are the only two or test. Just curiosity
We have other people test major beta versions we make for Windows, mostly at "real" time controls like 5 min or so, but we are the only ones testing the frequent Linux versions where the development occurs. In other words, the main function of the outside testers is to prove that changes we make that look good at hyperspeed are also good at normal time controls.
I agree, the only independent tests come from test houses, however, for people like me who already play chess, we tend to trust our own experiences and what you call 'tactics' I guess is what I call looking for the imbalances in the position. At the moment, Houdini still has the edge on others I have used {apart from Komodo, I have also used Critter, Hiarcs, Rybka, Stockfish, Zappa} - I only use single core and in the majority of positions I have analysed, it has been Houdini that has been the greatest help to me.
All, kidding aside. There seems to be just so much more controversy over rating lists and numbers lately when it comes to Komodo's performance. I've never seen so much confusion arise even in testing -as it has even here over this engines performance-nothing seems to be coming out that is straight forward. That is just my perception.
Part of it is just sample error. You need a lot of games to prove which engine is better if the gap is under ten elo.
I think a great deal more of the controversy swings around having a single core version prior to the release of an MP version.
I don't see much controversy about that. One program can be the best single core engine, but until it has an MP version it won't be the strongest engine for MP users (other than IDeA users).
That may very well be true if it were not being judged as a precursor to an MP version, which it is...against the MP versions of other engines on single core.
Well, of course the natural assumption is that one program will benefit about as much as MP from another (assuming a correspondingly longer time limit for the SP test), but until we actually have a testable MP program I can't say much more than that about it. In other words, the best prediction for Komodo MP on 8 cores at 40/4' is the 40/20' rating for one core, if we assume 8 cores gives 5 to 1 speedup.
> All, kidding aside. There seems to be just so much more controversy over rating lists and numbers lately when it comes to Komodo's performance. I've never seen so much confusion arise even in testing -as it has even here over this engines performance-nothing seems to be coming out that is straight forward. That is just my perception.
Well at least there is no controversy about the latest Junior release.
(and it pains me a bit to have fun at Junior's expense)
+1
> I consider the CCRL 40/40 list and the CEGT 40/20 list to be the gold standard of computer chess ratings. The 40/120 list would be best, but by the time an engine gets rated with enough games there a new version will be coming out. The IPON list has the best conditions regarding uniformity of opposition, books, hardware etc. but it uses a blitz time control so I would say it deserves priority over the CCRL and CEGT blitz lists, but not over their 40/20 and 40/40 lists. So to be clear, I will claim Komodo to be the strongest engine only when its average rating on those last two lists exceeds the average rating of other top engines. Fair enough?
Larry, Do you download the CCRL 40/40 and CEGT 40/20 games to analyse, e.g. looking for weaknesses or positions where Komodo didn't play so well, so that you can improve the engine ? Or do you have enough games of your own for that ?
We have far more games than time to look at them.
IMHO Houdini will not improve in strength.i believe Komodo will as Larry at least is a top Chess Player like Vas and Larry is one of worlds top evaluation experts,also i expect Vas is missing him now.Komodo MP could easily have been released a while back and made Don and Larry a lot of bucks.I admire both of them for turning down easy money and trying to improve their engine to make it a world beater.
i am happy to do some testing for them as well as purchase it.
Stop knocking Larry,he is one of the good guys.!!!!
i am happy to do some testing for them as well as purchase it.
Stop knocking Larry,he is one of the good guys.!!!!
I agree with your last statement, but I think that you need a number ingredients to build a great engine today (not necessarily in any particular order):
- Algorithmic skill to provide a detailed understanding of the intricacies of MP search,
- Software skills to write clean and efficient code,
- Reverse engineering skills to allow mining of great ideas from others, and
- Evaluation tuning skills.
Guys like Vas and Robert are really good at the first two. Vas is also very good at automated eval tuning, and of course is a very good chess player as well. Larry is the best at evaluation tuning of course, not only because of his chess skills, but also because he actually evaluates positions by the numbers himself. Richard is probably the best at reverse engineering.
I believe that all of the current top engines, including Komodo, have benefited from Ippo (and indirectly from Rybka). But future progress will likely be more difficult. I don't think you can count out people who have advanced the state of the art in the past though.
- Algorithmic skill to provide a detailed understanding of the intricacies of MP search,
- Software skills to write clean and efficient code,
- Reverse engineering skills to allow mining of great ideas from others, and
- Evaluation tuning skills.
Guys like Vas and Robert are really good at the first two. Vas is also very good at automated eval tuning, and of course is a very good chess player as well. Larry is the best at evaluation tuning of course, not only because of his chess skills, but also because he actually evaluates positions by the numbers himself. Richard is probably the best at reverse engineering.
I believe that all of the current top engines, including Komodo, have benefited from Ippo (and indirectly from Rybka). But future progress will likely be more difficult. I don't think you can count out people who have advanced the state of the art in the past though.
You forgot to mention Fruit!
And even more importantly, Crafty. All good ideas have always come from Crafty!
> Richard is probably the best at reverse engineering.
> Guys like Vas and Robert are really good at the first two. Vas is also very good at automated eval tuning, and of course is a very good chess player as well. Larry is the best at evaluation tuning of course, not only because of his chess skills, but also because he actually evaluates positions by the numbers himself. Richard is probably the best at reverse engineering.
One of the dangers of reverse engineering is that you lose creativity and originality.
A result of Richard's RE-ing Houdini 1.5 is that now Critter 1.4 is virtually indistinguishable from Houdini 1.5 in the similarity diagram.
Sure, Critter has become stronger, but at what cost?
The biggest danger from reverse engineering is that if trade secrets are revealed too quickly, the innovators (guys like you and Vas) don't want to put their best stuff out. In an ideal world (at least my ideal world) RE would be possible, but would take ~5 years and a fair amount of effort. If RE can be done in three to six months, there isn't sufficient time to capitalize on innovation, and the rate of advancement slows. It's well known that Vas didn't put out his best stuff in R4 and is looking to a cloud model to get away from RE. No doubt you have your own plans to deal with this issue.
This isn't a problem just for chess engine development of course. I have the same issues in my business (developing weapon systems), but with much more money at stake. As a matter of fact, I would have already tried to recruit Richard if he were a US citizen.
This isn't a problem just for chess engine development of course. I have the same issues in my business (developing weapon systems), but with much more money at stake. As a matter of fact, I would have already tried to recruit Richard if he were a US citizen.
> Stop knocking Larry,he is one of the good guys.!!!!
Sure! When he stops playing both ends against the middle!
There is no correlation between streight of the programmer as a chess player and the engines prospects.. chess engine programing has more to do with technicalitys and search. The fruit author who made the evaluation philosophy of which most programmers currently implement is not a good player.
> There is no correlation between streight of the programmer as a chess player and the engines prospects.. chess engine programing has more to do with technicalitys and search. The fruit author who made the evaluation philosophy of which most programmers currently implement is not a good player
It's true that there is no direct correlation, but it sure helps to be a "decent" chess player (say, at least 2000 Elo) to evaluate the situations where the engine is producing undesirable evaluations, or to design new evaluation terms.
BTW, you're overestimating the importance of the "Fruit author" - Fruit is a very weak engine in nowadays context. The most important person for the development of computer chess since 2005 is, without any doubt, Vasik, and he's a lot better than just a "decent" chess player.
![[nl]](/mwf/v12/flags/nl.png "Netherlands")
a lot better to put it mildly...
My best bet is that in a simul with the 16 voters for the guilty verdict he would get 16 out of 16 with his hands on his back.
My best bet is that in a simul with the 16 voters for the guilty verdict he would get 16 out of 16 with his hands on his back.
"Fruit is a very weak engine in nowadays context"
The basis or call it philosophy from fruits evaluation is still in use by most but not all top engines.
Practically it's a pretty weak engine by todays standards, reaches lower depth, weaker tactically, weaker search e.t.c
"sure helps to be a "decent" chess player (say, at least 2000 Elo) to evaluate the situations where the engine is producing undesirable evaluations, or to design new evaluation terms."
Ok but that's not ugm level. I think most enugine authors are around 1800, Frans morch 1800 , Stefan meyle kahleen 1900/2000.
The basis or call it philosophy from fruits evaluation is still in use by most but not all top engines.
Practically it's a pretty weak engine by todays standards, reaches lower depth, weaker tactically, weaker search e.t.c
"sure helps to be a "decent" chess player (say, at least 2000 Elo) to evaluate the situations where the engine is producing undesirable evaluations, or to design new evaluation terms."
Ok but that's not ugm level. I think most enugine authors are around 1800, Frans morch 1800 , Stefan meyle kahleen 1900/2000.
> The basis or call it philosophy from fruits evaluation is still in use by most but not all top engines.
Can you provide any evidence to support this claim?
What exactly is, according to you, the "philosophy from Fruit's evaluation"?
>> The basis or call it philosophy from fruits evaluation is still in use by most but not all top engines.
> Can you provide any evidence to support this claim?
> What exactly is, according to you, the "philosophy from Fruit's evaluation"?
Should we understand from your silence that you haven't got a clue?
True, although I suspect his wife is stronger!
> IMHO Houdini will not improve in strength.
LOL. You'll get the answer in September :).
i would love to be proved wrong.
I see...several months after the probable release of Rybka 5.
> I see...several months after the probable release of Rybka 5.
You'll note that unlike Rybka 5 I announce a release date.
Yes...after you already know that Rybka will be released in June or July.
How can he possibly know when Rybka will be released?
Because I think I recall Vas saying that it would be around June or July...or is this just me imagining?
He may well have said that, but past history being what it is, this unfortunately gives no indiciation of when it will actually be released.
![[mx]](/mwf/v12/flags/mx.png "Mexico")
You may have missed Lukas's message that Vas is behind schedule.
> Yes...after you already know that Rybka will be released in June or July.
No mention of Rybka 5 on the Houdini Facebook page: http://www.facebook.com/pages/Houdini-Chess-Engine/164560926948947
>"We've"
Hmm?....
Oh, I missed this completely--the Houdini Facebook page must be the best source of up-to-date news on Rybka products.
Consider this a challenge--if you release something significantly stronger than Rybka 5 before, during, or very soon after the release of Rybka 5, doubts will fade away.
Consider this a challenge--if you release something significantly stronger than Rybka 5 before, during, or very soon after the release of Rybka 5, doubts will fade away.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill