![[us]](/mwf/v12/flags/us.png "United States")
Do anyone have any information how well Houdini performs against Rybka at 40/120 timecontrol? I know a lot of talk about Houdini at fast time controls but computers like humans finds the better move at slower time controls.
![[se]](/mwf/v12/flags/se.png "Sweden")
http://www.husvankempen.de/nunn//40120new/40_120_ratinglist/40_120_AllVersion/rangliste.html
Error margin is around 25 points.
Error margin is around 25 points.
Komodo in first place, ahead of Houdini by a crushing margin (an entire elo point!).
>Komodo in first place, ahead of Houdini by a crushing margin (an entire elo point!).
ahead what version oh Houdini??you forgot mention that??
Komodo in first place,and where is Houdini 2?want you bet that if it was Houdini 2,you wont have the first place in that list today?
<"Komodo in first place,and where is Houdini 2?want you bet that if it was Houdini 2,you wont have the first place in that list today?">
I suspect H2 is around the same rating here as well as you can see by the blitz comparisons were the difference is around 5 points between versions:
I believe CEGT keep the same openings for all versions without major changes each month, so they should be quite reliable if theres major improvements made between versions.
I suspect H2 is around the same rating here as well as you can see by the blitz comparisons were the difference is around 5 points between versions:
I believe CEGT keep the same openings for all versions without major changes each month, so they should be quite reliable if theres major improvements made between versions.
you saw carefully at that list?Houdini 2.0 isnt at that list,only Houdini 1.5,so the claims that Komodo is top 1 at that list,is right,but false at the same time
again,why Houdini 2.0c isnt at that list?
again,why Houdini 2.0c isnt at that list?
Who cares it's not practically stronger than 1.5 anyway.... as you can see on 40/20 list.
2 Houdini 2.0c x64 4CPU 3097
3 Houdini 1.5a x64 4CPU 3093
2 Houdini 2.0c x64 4CPU 3097
3 Houdini 1.5a x64 4CPU 3093
that means Komodo ISNT TOP 1 at that list as Larry claims...
when he posted Komodo is number one,he forgot to mention that Houdini 2 werent in that list...too smart from him
i think that with the time,and with smaller controls,Komodo overpass Houdini,(Larry is a Gm,and Houdart isnt,so Larry can improve a lot more the knowledge than Houdart),but as rybka showed us,its strongest comes from its speed,not from its knowledge,so it isnt all lost for Houdart
at least now,still Houdini n. 1 at the list....Larry,correct your post please,as you forgot to mention that Houdine 2.0 wasnt at the list
when he posted Komodo is number one,he forgot to mention that Houdini 2 werent in that list...too smart from him
i think that with the time,and with smaller controls,Komodo overpass Houdini,(Larry is a Gm,and Houdart isnt,so Larry can improve a lot more the knowledge than Houdart),but as rybka showed us,its strongest comes from its speed,not from its knowledge,so it isnt all lost for Houdart
at least now,still Houdini n. 1 at the list....Larry,correct your post please,as you forgot to mention that Houdine 2.0 wasnt at the list
"that means Komodo ISNT TOP 1 at that list as Larry claims..."
Why do you care the placement of a machine?? Even chess engines have fanboys nowdays....
H2 might very well score below 1.5.. they are so close between each other
Why do you care the placement of a machine?? Even chess engines have fanboys nowdays....
H2 might very well score below 1.5.. they are so close between each other
i dont care the placement of an engine...is stupid for a long time controls care for the placement of a engine,and more if the engine is only rated 6-10 elo above other engine...
i just recalled one and another time the sema message because Larry Kaufman posted here that Komodo was rated top 1 in that list...he was anxious to tell everyone that his engine was top 1 in one list,and when he saw it,posted here...BUThe forgot to tell us that in the list that his engine was top 1,Houdini 2.0 was missing
just that,i was laughing about is hurry and his anxiety
i just recalled one and another time the sema message because Larry Kaufman posted here that Komodo was rated top 1 in that list...he was anxious to tell everyone that his engine was top 1 in one list,and when he saw it,posted here...BUThe forgot to tell us that in the list that his engine was top 1,Houdini 2.0 was missing
just that,i was laughing about is hurry and his anxiety
Houdini 2 was not tested by CEGT at 40/2 hours and won't be because they only test versions at that level that were rated above previous ones on their 40/20' list. Houdini 2 (on 1 core) was rated below 1.5 on that list. This policy is quite correct: it would be unfair to give Houdini two chances to be top if the newer engine is not provably stronger at the next closest time limit. Furthermore CCRL also found that Houdini 2 on 1 core was below version 1.5. So I think it is quite correct to say that Komodo is the number one single-core engine based on 40/2 hour CEGT testing.
I expect our next release will remove any doubt about which is strongest at longer time controls.
I expect our next release will remove any doubt about which is strongest at longer time controls.
It's possible that H2c's optimizations for best MP scaling necessitated compromises that left single core at the same strength at H1.5a. I'm not sure that's particularly important to the majority (outside of heavy IDeA users) who are interested in MP performance.
agree Alan...me for example,using Fritz GUI,im interested in MP performance,not running 2 instances of IDea
"It's possible that H2c's optimizations for best MP scaling necessitated compromises that left single core at the same strength at H1.5a."
The difference between Mp Houdini 1.5 vs Mp H2 is as slim as the single core counterpart comparisons... as you can see for yourself.
Your compromise theory for that matter makes no sense whatsoever.
The difference between Mp Houdini 1.5 vs Mp H2 is as slim as the single core counterpart comparisons... as you can see for yourself.
Your compromise theory for that matter makes no sense whatsoever.
![[gb]](/mwf/v12/flags/gb.png "United Kingdom")
i cant find any difference between H1.5 and H2 on 12 core at any time control or at corr level.Just my 2 cents.
![[mx]](/mwf/v12/flags/mx.png "Mexico")
Well, a huge difference may be that one engine has a learning feature, while the other isn't. I got a report that this feature makes the engine the best for analysis once it is "warm" and that's what I expected as I found Rybka 3 to provide better analysis than Rybka 4.1 once its Persistent Hash is "warm" (so the time spent warming it up is worth it in the long run.)
imho,the only reason you get any ELO diff between H1.5 and H2 is because when you get same eval lets say 0-00 to keep it simple and a choice of 2 moves its pot luck which move the engine chooses,one may well be better than the other but if after X depth the engine cannot decide it will stick with the original move it choosed.
i remember 3 or 4 years ago both Lukas and i both looked at same line in a match on Hiarcs forum,we both had Skulltrails.we had different lines but equal eval at same depth.i think you were there at the time.
i remember 3 or 4 years ago both Lukas and i both looked at same line in a match on Hiarcs forum,we both had Skulltrails.we had different lines but equal eval at same depth.i think you were there at the time.
Ditto... I found no difference either beyond randomness.
if your reasoning is right(about being Houdini 2.0 worst than 1.5),then yes,you was right and i was who spoke more than needed,that is something that i asked and no one answered to me,so my apologyses
about your anxiety to tell people your engine is number one in some list,i was right;you seem very very anxious to catch Houdini and tell it to everyone,but dont be bothered by my words,every person that can catch n.1 will be anxious to do it and tell everyone
if im sure you will do it(catch Houdini and overpass it)?sure,since you are a GM,and you can put a lot of knowledge into the engine,and Don can 'improve' the engine making it ''fast''(pruning it,or as he thinks is correct,im not a programmer and i wouldnt speak about i dont know),and the difference between Komodo and Houdini,at least at lont time controls,are very very little...
now another question,NOT relative about long time controls:in a short time controls,and using single core(like ipon list),do you think you can catch Houdini in a relative short time?
about your anxiety to tell people your engine is number one in some list,i was right;you seem very very anxious to catch Houdini and tell it to everyone,but dont be bothered by my words,every person that can catch n.1 will be anxious to do it and tell everyone
if im sure you will do it(catch Houdini and overpass it)?sure,since you are a GM,and you can put a lot of knowledge into the engine,and Don can 'improve' the engine making it ''fast''(pruning it,or as he thinks is correct,im not a programmer and i wouldnt speak about i dont know),and the difference between Komodo and Houdini,at least at lont time controls,are very very little...
now another question,NOT relative about long time controls:in a short time controls,and using single core(like ipon list),do you think you can catch Houdini in a relative short time?
To catch Houdini at blitz, we might need to know why it is stronger at blitz. I am convinced that all of the Ippolit-derived programs are relatively better than all the others at blitz versus longer time control chess. This also applies to Critter which although not Ippolit-derived uses all the pruning and extension ideas from Ippolit. So the key is in one or more of those ideas which we are not currently using. I'm trying to figure this out. Anyway, even without this "secret", I expect we will overtake Houdini even at blitz controls in a few months just based on our rapid rate of progress. Perhaps with MP it will happen sooner, though this is just conjecture at this point.
"To catch Houdini at blitz, we might need to know why it is stronger at blitz."
When two engines have <similiar> evaluation streight, tactical streight decides which engine will come out on top at speed chess.
And Houdini and Critter accordingly always outperform Komodo at tactical test suites by a large margin while both have similiarly solid evals....
And this benefit matters very little in slow chess when they have 3 minutes per move instead of seconds since less tactics go unnoticed even for the "slower" engine.
When two engines have <similiar> evaluation streight, tactical streight decides which engine will come out on top at speed chess.
And Houdini and Critter accordingly always outperform Komodo at tactical test suites by a large margin while both have similiarly solid evals....
And this benefit matters very little in slow chess when they have 3 minutes per move instead of seconds since less tactics go unnoticed even for the "slower" engine.
Yes, this is probably a correct explanation. But I need to know WHY they are stronger at tactics. I do know some reasons, and we will probably address these issues in the near future. But I feel that I'm missing the major reason for their tactical edge. Since even Ippolit (Ivanhoe) is especially strong at bullet chess, the answer should be something known, not some new idea in Houdini.
Presumably developers are already reverse engineering Komodo to look for your improvements to evaluation. Is there any reason to believe that other engines won't have the same improvements six months later?
I would imagine that most people inclined to do reverse engineering will focus on the top rated engine, so this will probably only become a big problem once we achieve that spot. Anyway, it is likely that many ideas we have discovered won't work so well in other programs. Perhaps they work for us because they accomplish the same goal by different means than things already in Houdini and/or Critter.
You underestimate your reputation for being the maven of evaluation weights. I suspect that smart developers will already be looking at Houdini for search improvements, and Komodo for evaluation improvements. It would be interesting to get Mr. Vida's take on this...
Richard said that he looked at Komodo's search, but not at its eval. I don't know if he used any Komodo ideas. Of course he is free to do so.
> Richard said that he looked at Komodo's search, but not at its eval. I don't know if he used any Komodo ideas. Of course he is free to do so.
You mean RE Komodo? This doesn't bother you? I know it's legal, but if someone did that to my program...I don't know... I'd be a bit rattled.
![[cn]](/mwf/v12/flags/cn.png "China")
I think they exchanged source.
> I think they exchanged source.
Ah....ok.
Was it an exchange or did Richard give his source to Don! I think it went in one direction only!
Judging from Larry's response, you are correct.
dont think is a bit stupid exchange the source code of a new and recent released engine like Komodo 4?
or if Larry or Don exchanged code with Richard,must be an older version of Komodo,like the 1st version,but not the actual version,that seems stupid...if it is the actual version,must be a RE
or if Larry or Don exchanged code with Richard,must be an older version of Komodo,like the 1st version,but not the actual version,that seems stupid...if it is the actual version,must be a RE
> dont think is a bit stupid exchange the source code of a new and recent released engine like Komodo 4?
There was no exchange. But why do you think it would be stupid?
for a simply reason;it would be stupid exchanging source code of a recently released version of a comercial engine (Komodo 4) to another programmer competitor (you),even if you offer a free program engine...
different is if you,releasing a free engine (and as you said,always will be free),you offer your source code to another programmer to see your ideas
different is if you,releasing a free engine (and as you said,always will be free),you offer your source code to another programmer to see your ideas
Don't you think that competing chess programmers (commercial or otherwise) can be friends? And may have different motives than ripping off each other, or they may even (heavens forbid) share ideas?
yes,can be friends...so tell me Richard,if you are friend of Don and Larry,why RE Komodo to look at source code instead of asking them for the source code?
![[dk]](/mwf/v12/flags/dk.png "Denmark")
Barnard,
You clearly should think before you jump, or better yet, get your head out of you ass.
Richard freely gave his source of the latest and strongest Critter to the Komodo team to help them improve their engine, why should not they do the same ? BTW, even if they don't, he can always decompile the engine and look what's inside.. The point is he would be wasting a lot of time by doing so. It will also show the Komodo Team untrustable, if they got his source, and he himself got nothing in return. Is it not much easier to be on a friendly terms, like sharing sources and ideas, from one programmer to another.
Richard is not commercial author, so Komodo Team have nothing to worry about.
You clearly should think before you jump, or better yet, get your head out of you ass.
Richard freely gave his source of the latest and strongest Critter to the Komodo team to help them improve their engine, why should not they do the same ? BTW, even if they don't, he can always decompile the engine and look what's inside.. The point is he would be wasting a lot of time by doing so. It will also show the Komodo Team untrustable, if they got his source, and he himself got nothing in return. Is it not much easier to be on a friendly terms, like sharing sources and ideas, from one programmer to another.
Richard is not commercial author, so Komodo Team have nothing to worry about.
i will get my head out of my ass the day you will get your finger out of your ass and after you taste it stupid ignorant...i explained myself enough clear?
We didn't send him source code, but he can get some idea about our search using a decompiler. I don't mean reverse engineering Komodo fully, that would be a major project, since unlike Houdini there is no open-source code that was copied from. But Richard has the skills to be able to spot key search techniques just from a quick look at the output of a decompiler. There's nothing to be done about this, so there's no point in getting upset if other people do this, especially in this case since Richard did sent us Critter source code voluntarily. We agree that it is okay to use ideas from other programs; we just disagree about what constitutes an "idea".
disagree about what constitutes an ''idea''?
is sooooo easy:idea is a NEW aproach to something (engine in that case),that you implement with a code...
i dont know program,but lets see an example:you dont prune a lot your programm,and you trust in the knowledge of your program to be strong (is a stupid idea,i know,just to explain my concep of idea),but someone discovers that making a drastic prunning AND adding a bonus for the bishops,that increases the elo about 100 elo points... that is a new idea to make your program stronger,no one had before that 'idea',so that idea is a new aproach,and you only need to implement it
problem?someone will decompile you program,will see your 'idea',will take it,and implement in his engine,and make his engine strong with your idea...
some people are specialist with doing that Larry
edited:if you make your new idea,is yours,even if others take it...if you take that idea from others(being legal or not),that idea isnt yours,is from the guy you took it...but people never give credit from who took ideas(or only a few times do it),so they give credits to himselfes,telling ''how smart i am,i improved my engine X elo points in X years'',but forget to tell ''taking ideas from the RE engine X''...
is sooooo easy:idea is a NEW aproach to something (engine in that case),that you implement with a code...
i dont know program,but lets see an example:you dont prune a lot your programm,and you trust in the knowledge of your program to be strong (is a stupid idea,i know,just to explain my concep of idea),but someone discovers that making a drastic prunning AND adding a bonus for the bishops,that increases the elo about 100 elo points... that is a new idea to make your program stronger,no one had before that 'idea',so that idea is a new aproach,and you only need to implement it
problem?someone will decompile you program,will see your 'idea',will take it,and implement in his engine,and make his engine strong with your idea...
some people are specialist with doing that Larry
edited:if you make your new idea,is yours,even if others take it...if you take that idea from others(being legal or not),that idea isnt yours,is from the guy you took it...but people never give credit from who took ideas(or only a few times do it),so they give credits to himselfes,telling ''how smart i am,i improved my engine X elo points in X years'',but forget to tell ''taking ideas from the RE engine X''...
To illustrate the question, let's say I add code that says "if I have a bishop that attacks the square in front of your king, I get a bonus of 0.17 pawn." So, the new idea would be to reward bishops that attack the frontal square of the king. If someone implements this idea and uses my 0.17 value, I consider that to be more than using the idea, it is using part of the code (the numerical value). Richard would say that the 0.17 value is part of the idea. That is the point of disagreement.
i think,in the form you presented me that idea,i must agree with Richard;you presented the 0,17 bonus in a global of the idea,so is part of the idea...different is if that idea implemented,and coded,when the code is computed by the computer,produce a result of 0,17 unknow before to you,and with that 0,17 unknow before to you,yes,i must agree that is part of the code
Hi Larry
take a look at my running game at WBCCC 2 agains Sebastian Boheme,and you will see the type of position where it will be usefull that Komodo has implemented the second featured that we spoke (about pawns break when nothing is on the board)...
i mean at 22 move of black,just before i made my 23 move 23.c4
p.s.i just that you want see the position,i wouldnt any help or comment,to avoid Sebi thinking im receiving help from you...just say if is the type of position Komodo can be improved with the feature we spoke(breaking pawns),so Komodo can gain elo from that feature in the future if is well implemented and surpass Houdini
take a look at my running game at WBCCC 2 agains Sebastian Boheme,and you will see the type of position where it will be usefull that Komodo has implemented the second featured that we spoke (about pawns break when nothing is on the board)...
i mean at 22 move of black,just before i made my 23 move 23.c4
p.s.i just that you want see the position,i wouldnt any help or comment,to avoid Sebi thinking im receiving help from you...just say if is the type of position Komodo can be improved with the feature we spoke(breaking pawns),so Komodo can gain elo from that feature in the future if is well implemented and surpass Houdini
What works in one position does not necessarily mean it will work in another - try to remember that all chess engines are tuned to give a 'best fit' approach across a range of tests. Two things to note:
- this approach will never provide a 'perfect' solution as it is a 'best fit' approach
- we don't have enough time left on this planet to provide tests that cover all possible scenarios, let alone have the time to test them!
So, adjusting the way a program plays for a given situation may not end up with a better solution across all the tests carried out and ultimately this could mean that the position on a ranking list {CCRL, CEGT, IPON, etc.} is not as good - this is ultimately what chess programmers use to judge their achievements when they take their efforts from the lab in to the public domain.
Make sense?
- this approach will never provide a 'perfect' solution as it is a 'best fit' approach
- we don't have enough time left on this planet to provide tests that cover all possible scenarios, let alone have the time to test them!
So, adjusting the way a program plays for a given situation may not end up with a better solution across all the tests carried out and ultimately this could mean that the position on a ranking list {CCRL, CEGT, IPON, etc.} is not as good - this is ultimately what chess programmers use to judge their achievements when they take their efforts from the lab in to the public domain.
Make sense?
you must understand that look at one position is just an example,and to implement 'code' to an engine you must put the 'human' knowledge to an engine and try to put that knowledge is try to put concret examples into a general terms...
Barnard,
All changes bring good/bad on a 'best fit' approach - every engine that is built is based on some form of selective search that helps it to evaluate and establish a table of moves prioritised by the scores coming from the search/evaluation function. So one small improvement in one type of chess position may not be good in another. This though is the approach that most if not all chess programmers take. Sometimes they get lucky and discover one change that brings more plusses than minuses. This doesn't happen very often - the chess programmer is indeed a resilient beast!
If one imagines for a moment that chess is mathematically solvable then we obviously wouldn't need search and engine chess as you know it today would cease as a calculator/spreadsheet would give the right answer every time. Chess software as you know it today would survive only because of the features it contains, not because of the moves it plays as these would be 'perfectly' and 'instantly' calculated. Chess tournaments where person A is playing against person B, both pretending to each other that they are playing {as opposed to a chess engine} would also cease - just to end with some good news!
All changes bring good/bad on a 'best fit' approach - every engine that is built is based on some form of selective search that helps it to evaluate and establish a table of moves prioritised by the scores coming from the search/evaluation function. So one small improvement in one type of chess position may not be good in another. This though is the approach that most if not all chess programmers take. Sometimes they get lucky and discover one change that brings more plusses than minuses. This doesn't happen very often - the chess programmer is indeed a resilient beast!
If one imagines for a moment that chess is mathematically solvable then we obviously wouldn't need search and engine chess as you know it today would cease as a calculator/spreadsheet would give the right answer every time. Chess software as you know it today would survive only because of the features it contains, not because of the moves it plays as these would be 'perfectly' and 'instantly' calculated. Chess tournaments where person A is playing against person B, both pretending to each other that they are playing {as opposed to a chess engine} would also cease - just to end with some good news!
>So one small improvement in one type of chess position may not be good in another
one chess engine can know if it must apply that knowledge or not recognising one type of position or another,right?
Each chess engine in the public domain today has strengths and weaknesses over their competition and therefore if you offer a particular position to say the top-20 engines you will find anything from the same, through similar to different results. So yes, for a given position you may find, lets say, Houdini appears to understand {please remember a chess engine doesn't really understand, simply coded to calculate in a certain way} the position more than lets say, Shredder. However, you may also find that Zappa also appears to understand the position. So, here you have an example where the highest ranked engine Houdini and an engine ranked almost at 20th are in alignment, yet an engine ranked around 9th doesn't agree. There are many examples like this and if we were to create say, 200 test positions, {something I have been doing for years and I'm sure those who test engines have done also} and allow each engine to provide answers against these positions after lets say, 5min of thinking time, and ranked these engines by the performance of the results, then would this list look different to say the one supplied by CCRL, CEGT or IPON?
I have certainly found Zappa to perform much better in this type of position analysis than its ranking position would have us believe it should. So in the end, the programmer has to make a choice and currently that choice appears to be to create an engine that can achieve a ranking of 1st on published lists from at least the above and this is not necessarily the same as providing the chess analyst with the most solid piece of chess analysis for every position encountered. A 'best fit' approach can never do this.
I have certainly found Zappa to perform much better in this type of position analysis than its ranking position would have us believe it should. So in the end, the programmer has to make a choice and currently that choice appears to be to create an engine that can achieve a ranking of 1st on published lists from at least the above and this is not necessarily the same as providing the chess analyst with the most solid piece of chess analysis for every position encountered. A 'best fit' approach can never do this.
well...all that you wrote,in esence,is right...but what Larry and me were speaking is add code to Komodo to make it start 'actions' when,and is an example,'nothing occurs' on the board,so start 'breaking pawns',for example...and that isnt put one position into the engine,that is code the engine to see if it can start a breaking pawn,and if it can be 'positive'just that and only that
Do you remember Rybka 3 Dynamic? Assuming you do you will remember that this was the most aggressive Rybka engine from the 3 series and yes, in certain positions it outperformed Rybka 3 and Rybka 3 Human. However, from a ranking viewpoint it achieved a lower rating then the other two engines.
I hope this gives you food for thought around how change works, i.e., that every change brings a range of outcomes, some good, some less good, some bad, and so on. The R3 series I hope will convince you that what works well in one dimension of chess does not mean that it will work well in all dimensions of chess. If it doesn't convince you then I guess I will leave you to your quest of convincing others. Good luck.
I hope this gives you food for thought around how change works, i.e., that every change brings a range of outcomes, some good, some less good, some bad, and so on. The R3 series I hope will convince you that what works well in one dimension of chess does not mean that it will work well in all dimensions of chess. If it doesn't convince you then I guess I will leave you to your quest of convincing others. Good luck.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill