Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka and Houdini at 40/120 Timecontrol?
1 2 3 4 5 6 Previous Next  
Parent - - By Barnard (Bronze) Date 2012-02-21 21:40
yes,i remember rybka 3 dynamic,and yes,i understand your point fo view that every chnage bring outcomes,some good and some wrong...
now answer me just one question,just one:Larry will implement 'break pawns',and when it will be implemented,in some positions it will be good,and of course,in some positions will be wrong,but in total,will have a score,lets say 1)negative--Larry will delete that code from Komodo 2)positive--Larry will continue with that code in Komodo...so the question:why dont try to implement that function?if it works,it will continue,if it doesnt wotk,it will be deleted...just that
Parent - - By Razor (****) [gb] Date 2012-02-21 21:53
Because the approach being used is 'best fit' - this means that all change will bring a variety of outcomes.  You seem to believe that one change followed by another will remove all that is bad and only end up with good outcomes across the whole sphere of chess.  This is just not possible I'm afraid - I wish you and Larry {and other chess programmers} the best of luck with this approach.
Parent - - By Barnard (Bronze) Date 2012-02-21 21:57
well,you seem know a lot more than me about programming...so please,tell me,how you will improve an engine to make it strong?
Parent - - By Razor (****) [gb] Date 2012-02-21 22:07
I know a great deal about chess; I know a reasonable amount about testing software; very little about programming a chess engine other than what I have read in books so I'm afraid I cannot provide any insight at a coding level.  Always happy to help with ideas and most of my responses to you have been around providing feedback based on my experience as a software tester and knowing the consequence of code/data changes on the same.  Whilst these solutions are not chess solutions the points I have made hold true for all software solutions; the impact a code/data change has can be far-reaching and very often manifest itself in symptoms that are not always positive ones.

Bye for now.
Parent - By Barnard (Bronze) Date 2012-02-21 22:15
thanks for sharing you knowledge...

bye for now
Parent - - By Lukas Cimiotti (Bronze) [de] Date 2012-02-17 19:55
Would it work to use .15 pawn instead of your .17 to make Critter clean?
Parent - By Banned for Life (Gold) Date 2012-02-17 21:29
What if there are five engines, and they have staked out claims to 0.15, 0.16, 0.17, 0.18, and 0.19. Is one forced to use a crappy value to avoid unoriginality? :smile: I find this to be a very weak argument...
Parent - - By lkaufman (*****) Date 2012-02-18 17:36
My view is that if you use the same value as another engine, or one extremely close to it, that could be a legitimate coincidence, or it could be that the original value was optimum. But if you do it repeatedly thruout the program you are not just using ideas, but are copying the results of tuning, which is not in my view an "idea". For example, in Komodo we have some parameters that are identical to ones found in Ippo, but they are small integers where we have determined by our own testing that those values really are best. But we never use the same values or even very close ones for parameters whose value is hard to determine, because our own testing never produces the same or similar values. In short, using someone else's parameter values is more like copying code than like copying ideas, but an isolated instance or two proves nothing; it must be a pattern. Of course there can be cases where it is a matter of opinion; for example someone could copy another program's values and randomly perturb them by a few percent, just enough to make it less than certain that they were copied. No easy answers...
Parent - - By Banned for Life (Gold) Date 2012-02-18 18:33
But we never use the same values or even very close ones for parameters whose value is hard to determine, because our own testing never produces the same or similar values.

Ugh. This is self serving nonsense. What if it did produce the same or similar values? Would you detune your weights to weaken your product so that you could continue to claim 'originality'? And does it bother you at all that neither you nor Don came up with 90+% of the features that you are using?

Of course there can be cases where it is a matter of opinion; for example someone could copy another program's values and randomly perturb them by a few percent, just enough to make it less than certain that they were copied. No easy answers...

It's actually very easy to check the correlation of static evaluation between evaluation functions, and over a large number of positions (on the order of a million) these correlation values will be stable with an RMS error of less than 0.1%. This exercise also shows how close different top engines are, with all top engines being at least at the 0.975 level and with similar engines being at 0.99 and above.

Statistical techniques can also be used to generate weights that look very different but produce exactly the same results. The simplest method of doing this would be to decorrelate the features.
Parent - - By lkaufman (*****) Date 2012-02-20 00:04
"This exercise also shows how close different top engines are, with all top engines being at least at the 0.975 level and with similar engines being at 0.99 and above." That's almost certainly because with the sole exception of Stockfish, every top program was indirectly based on my evaluation work, either for Rybka or for Komodo.
Parent - - By Banned for Life (Gold) Date 2012-02-20 00:12
every top program was indirectly based on my evaluation work, either for Rybka or for Komodo.

OK, that explains why Stockfish is an outlier. The Ippos are closer to R3 than R1B was to Fruit. Unfortunately, it's highly unlikely that BB will collect the static evals for the million positions for Komodo and Critter to compare these fine engines to Crafty, Fruit, Rybka, the Ippos and Stockfish...
Parent - - By lkaufman (*****) Date 2012-02-20 00:24
I'm not sure what the point would be. Komodo will bear some similarity in eval to R3 and the Ippos since I worked on both Komodo and R3, but I'm certain that the similarity of R3 to the ippos is far greater than either to Komodo. As for Critter, I suppose the similarity to the Ippos declines with each new version, as Richard is constantly improving Critter. The same is true of Houdini, but in that case the eval of the first version was not similar to Ippo, it was identical. It is in the area of search where it is easier and perhaps more meaningful to identify similarities and differences. After all eval is based on accepted chess wisdom, and so the terms are fairly similar in most programs, but there are a zillion ways to do things in search.
Parent - By TheHug (Bronze) [us] Date 2012-02-20 00:33
I know this is a little off-topic and I did make a thread about it on the computer chess forum here. But I wanted to wish you and Dan a tournament next weekend. :smile:
Parent - - By Banned for Life (Gold) Date 2012-02-20 00:48
All features are not created equal. Some are much more influential than others, due to weighting and frequency of occurrence. I computed a table of correlations of static evaluations from multiple engines a while back based on a million positions (evaluations were collected by BB), and the table values were stable to the fourth decimal place when testing with disjoint subsets.

The table shows, for instance, that your tuning for R3 Human had very little affect on the normalized correlation with R3 default (0.9979), and that the early ippo was very close to R3 (0.9888). Despite claims that the eval is virtually identical, Fruit 2.1 isn't that closely correlated to Strelka (by the standards of strong engines) at (0.9751). The original Ippo was actually a little closer at 0.9774, although nobody claimed that Ippo's eval was based on Fruit. Stockfish's eval is separated from the other top engines (probably because you weren't involved with the development of its eval) and Crafty even more so (which may partially explain why Crafty is not a top engine).

    R3          R3H       R3D         IPP        SF 1.6.2  Fruit 2.1  Strelka    Crafty     R3H C15
    1.0000    0.9979    0.9944    0.9888    0.9536    0.9726    0.9678    0.9535    0.9935
    0.9979    1.0000    0.9939    0.9867    0.9525    0.9697    0.9657    0.9539    0.9950
    0.9944    0.9939    1.0000    0.9790    0.9318    0.9644    0.9652    0.9442    0.9912
    0.9888    0.9867    0.9790    1.0000    0.9572    0.9774    0.9652    0.9555    0.9823
    0.9536    0.9525    0.9318    0.9572    1.0000    0.9531    0.9407    0.9314    0.9461
    0.9726    0.9697    0.9644    0.9774    0.9531    1.0000    0.9751    0.9627    0.9675
    0.9678    0.9657    0.9652    0.9652    0.9407    0.9751    1.0000    0.9532    0.9645
    0.9535    0.9539    0.9442    0.9555    0.9314    0.9627    0.9532    1.0000    0.9534
    0.9935    0.9950    0.9912    0.9823    0.9461    0.9675    0.9645    0.9534    1.0000

In any event, this comparison of eval function via normalized correlation is, in my opinion, a much better estimator of how close the eval functions are than something relying on counting similar features, which is rather arbitrary and doesn't take into account frequency of occurrence.
Parent - - By lkaufman (*****) Date 2012-02-20 01:26
Rybka 3 human mainly differed from normal Rybka 3 in the relative weighting of material vs. positional terms. Apparently your method is rather blind to a large difference in this respect. But maybe it's correct in a sense, because most of the terms were identical or nearly so. So here is an example of two near-clones that nevertheless played very different chess.
Parent - By Banned for Life (Gold) Date 2012-02-21 06:58
Rybka 3 human mainly differed from normal Rybka 3 in the relative weighting of material vs. positional terms. Apparently your method is rather blind to a large difference in this respect.

This 'large difference' results in a fairly small change in the average static eval output, resulting in almost perfect correlation. This is either because the differences are uniformly small, or because they are larger, but occur in a small percentage of the million tested positions.

But maybe it's correct in a sense, because most of the terms were identical or nearly so.

This test is agnostic to what terms are used or not used. It looks only at the covariance of the static evaluation outputs for the set of engines. In fact, one method of attempting to improve chess engine performance might be to find the more time consuming features, and look at minimizing changes to the static eval when removing this feature. This can be done in a reasonable time frame because it does not require any games to be played (i.e. the optimization is performed on the previously evaluated large set of positions). Another useful feature of the covariance matrix is that it provides a method of ordering features by average importance to the static eval over the set of positions. This ordering by importance to the final result might be compared to another list showing how much time is required to calculate each feature to give a quantitative measure of 'bang for the buck' for each feature.
Parent - By patrick delaurentis (**) Date 2012-02-21 03:01
i think someone with a bit of programming knowledge and a bit of cash can easily copy other programmers- im am, and i feel alot of people also, who want better quality chess moves, look to hardware- being able to get to ply 25 by the time another program reaches 20 ply reaps amazing results, but i have noticed that it usually is with different programs where the most significance is seen. via, rybka vs rybka with the weak and strong computer in comparison to rybka vs komodo or rybka vs houdini, you will see better results even with the weaker program, as it will beat the stronger one when it has the power of a beast machine to drive it.

as a matter of fact chess programs are so much stronger then humans that i feel the main aspects of chess improvements has to do with mainly with depth. for instance, with rybka on 100 cores as vas has, all he needs to do is get the moves of games where one side plays on a depth of 20 the other on 21, and so on, the next game with one sides depth at 22 and the other 23. so i feel this is done alot and that the search is artificially fabricated to seem like it is reaching ply 22 when the programmer already has the set moves of a givin depth on his personal machine.
Parent - By patrick delaurentis (**) Date 2012-02-21 03:15
i think all chess improvement these days have to do with depth/ply- vas has a 100 core beast- he can play multiple sets of games and just up the ply to -23n-24-25-26 on different games. i feel the search we have is artificially fabricated once it reaches very high depths/ply and that vas has most of these super high plys on his personal machine and he gives us some of the higher plys in the next program and makes that program reach them faster in a fabricated search (meaning false) there is alot of deception in programming. i feel the top programmers work together and that they milk them slowly on purpose- i think they have chess codes capable of elo 3500 play right now.
Parent - - By Banned for Life (Gold) Date 2012-02-17 21:26
One would think that once a new feature was implemented, some optimization routine would be run to come up with a weight. If the weights are properly normalized (which is admittedly unlikely in a chess engine), and both engines are well optimized, it shouldn't be surprising to see weights that or similar or even the same, just as it wouldn't be surprising to see both engines sharing the same weights for piece values, or bishop pair bonus, or any other value that can be set via optimization.
Parent - - By lkaufman (*****) Date 2012-02-18 17:43
For any one term, I agree. For a hundred terms, a close fit would be statistically impossible.
Parent - - By Banned for Life (Gold) Date 2012-02-18 18:12
On the contrary, if both cases used the same features and effective optimization routines, a close fit after optimization would be very likely. This is the norm with many systems which use common weighted features for important tasks like voice recognition. It's only in the chess engine development realm with its Hyattian influence, where people claim license to things that are not protected by copyright or patent.
Parent - By lkaufman (*****) Date 2012-02-19 23:57
There should be ways to estimate the likely standard deviation of parameter values determined by optimization. If program B matches program A far too closely compared to the estimated SD, then we can be reasonably sure that program B just copied the values (with a little fudging perhaps) rather than optimized them himself. That would presumably be the criterion for entry in an ICGA event, I would think.
Parent - - By AWRIST (****) Date 2012-02-18 08:15
GM Larry, I'm just an observer, but if I observe your Komodo results in some test ranking lists, I am afraid I must conclude that you and Don certainly have no ideas that would mean 200 Elo points above the mediocre rest of the field? What you have is what you got from Rybka with the help of some byproducts. So, your next step would only come after Rybka 5. If I were Vas I wouldnt want to enter into slavery mode for political loudmouths while I were put under ban by these hooligans. So, either you convince your master of the necessity to rehabilitate Vas completely or I would hope that anytime a commercial Kommody is first or second place that then a copy came along of course together with open source of what Don Dailey is actually working on? Action and reaction! So, perhaps Richard gives you some warmth against the cold outside?
Parent - By lkaufman (*****) Date 2012-02-18 17:49
Of course we use many good ideas from Ippolit and from Stockfish, plus one or two ideas from Critter, but we always tune the values ourselves and never end up with the same numbers except for small integers. I would estimate that our original ideas added about 50 elo, and of course our eval is completely original and of unknown value compared to others. Many ideas from these other engines don't work for us, at least so far. In short, we use all publicly available knowledge to make Komodo stronger without assuming that any particular idea or value is a good one.
Parent - - By Labyrinth (****) [us] Date 2012-02-13 00:29

>But I feel that I'm missing the major reason for their tactical edge.


Could it be just good ol' fashioned bean counting? I was under the impression that Komodo had a heavy eval which allows it to give more accurate evaluations and play stronger at longer time controls at the cost of blitz/tactical performance. How is the perft on Komodo?
Parent - - By lkaufman (*****) Date 2012-02-13 02:52
I'm sure that this is part of the problem, our nodes per second are rather low, no matter how they are counted. But I don't think this is the only issue, or even the main one. Our Singular Extension is rather conservative, so that could be the problem. We are working on that now.
Parent - - By rarara (**) Date 2012-02-14 17:19
Interesting to read. Always nice to hera something directly from a developer team! :smile:

Thumbs up
rarara
Parent - By Werewolf (*****) [gb] Date 2012-02-16 08:52
Yes it's great having Larry on the forum giving information. Can't wait for Komodo MP
Parent - - By Barnard (Bronze) Date 2012-02-12 21:48
thanks for your answer Larry

so,in ensence,you mean you will catch Houdini without knowing his 'secret',due to your fast rate of progress,and without using any of the pruning and extension,etc...ideas that it surely uses?

well,if it happens in short time controls(and of course,in long time controls),i will be a fan of Komodo...

just 2 questions for you,if you have the patience to answer me:

1)it seems that Don is having problems developing a MP version of Komodo,since you are in version 4,and still we havent a MP of it,im right?

2)is possible to develop Komodo more agressive,or at least,allow us to 'play' with the configuration,to make it more agressive?

thanks
Parent - - By lkaufman (*****) Date 2012-02-12 22:04
I don't say we don't use pruning ideas from other programs if known. What I do say is that we have found that only some of them work for us, many do not. So we develop our own ideas.
     I don't know if problems is the right term; it's clear that Don underestimated the time MP would take, and he cannot resist spending time trying to improve the program, since that's much more fun than MP. But now that he has an MP version that at least compiles, the end should be in sight.
     We made a huge change in Komodo 3 to make it more aggressive. We could perhaps offer options for the user. But we need a good, rigorous definition of "aggressive".
Parent - - By Barnard (Bronze) Date 2012-02-12 22:13
well,im gonna give you a ggod,rigorous definiition of ''agressive'';the play style of rybka is more agressive,in general terms,than Komodo...i dont mean that Rybka is stronger,i mean that i mean,that is more agressive...

agressive is that atacks more,preffer more agressive positions instead of positional positions,if the engine can,sacrifice instead of choose another line that can also win but WITHOUT the sacrifice,press the opponent and dont 'sleep' into drawish positions (i think that can be set with the contempt factor),etc...

an example:using an high contempt factor;Komodo will refuse draws...of course,against a strong engine,at risk of losse,BUT improve the engine,so you can eliminate that risk of losse,so Komodo can ALWAYS play for win,like if he plays with an high contemt factor

i know that i dont explain myself properly;but you know perfectly what means ''agressive'' and ''passive'';rybka is an agressive engine,Hiarcs is a passive engine(plays too positionally,dont risk,etc...)
Parent - - By lkaufman (*****) Date 2012-02-12 23:39
It's funny, but I used to hear that Rybka was too passive compared to some other programs. Maybe that's changed now. But even your reply includes some unrelated notions. One point is willingness to sacrifice material. Another is to play actively, which to me mostly means to initiate pawn breaks if nothing is happening. These two issues are pretty much unrelated though. I suspect that Komodo is aggressive enough in the first sense, but probably not in the second.
Parent - - By Barnard (Bronze) Date 2012-02-12 23:46
so even with my 'unrelated notions',you understood me...

well,is possible to make Komodo,taking your example,initiate pawn breaks if nothing is happening?

and second,you are a grand master,you know perfectly well,even with my poor english,what is play agressive,and what is play passive...so,with your knowledge of chess,is possible make Komodo plays agressive in ALL of the aspects that he can?or at least give the users the possibility to configure it to play agressive,like agressive means for the understanding of a GM?
Parent - - By lkaufman (*****) Date 2012-02-12 23:54
Komodo is already aggessive in terms of pawn sacrifices; we could make a user option to increase or decrease that. In terms of initiating pawn breaks, it is very easy to make Komodo do this, but hard to make Komodo do this without losing elo points. Perhaps this could also be an option; I rather like that idea, as it would be especially effective in matches against human GMs. Do you have any other ways in which you think Komodo should be more aggressive?
Parent - - By Barnard (Bronze) Date 2012-02-13 00:03 Edited 2012-02-13 00:08
not,i havent

about a user option to increase/decrease pawn sacrifices:are you gonna tell Don to implement that option?it will make losse elo to Komodo as pawn breaks?

about pawn breaks(question for you)                            :are you gonna study how implement your chess knowledge to make possible implement a future more agressive 'pawn
break',without losing elo points?

i think at least that last feature will make Komodo play stronger,and if it plas strong agains GM's,maybe will improve its plays in long time controls against other engines(correct me if im wrong)
Parent - - By lkaufman (*****) Date 2012-02-13 02:45
Increasing or decreasing pawn sacrifices by any significant amount will hurt the elo. We may make that a user option, probably not in the next release. The pawn break idea is more interesting, both as a feature and because there is at least a possibility that if we do it right it could help the elo. Actually, now that Komodo is among the top three engines, just making it try for more breaks might help the elo on the rating lists even though it might hurt very slightly when playing other top-3 engines, because avoiding draws with the many weaker engines will help the elo. We should do this in the near future one way or another I think. If we do it in a "smart" way it could even help against equal engines (if there are any ...).
Parent - - By Barnard (Bronze) Date 2012-02-13 02:55
well,if improving more pawn breaks is interesting,just do it :wink:

i think your reasoning is right,and maybe with the other top-3 engines even hurt slightly,but with the other engines,it will take more elo points,so in global terms,i think that Komodo can gain more than losse elo points,so that 'agressive break pawn' can be decisive to overpass Houdini at the top lists

and being you a GM,i think you have the knowledge to do it
Parent - - By lkaufman (*****) Date 2012-02-13 03:08
The only delay is that this is not something I can code without Don, and Don needs to stay on MP until we can release it. After that this can be a high priority item.
Parent - - By Barnard (Bronze) Date 2012-02-13 03:15
well,is supossed(correct me if im wrong),that releasing a MP version of Komodo is only matter of a few days/less than a month,so if you want catch Houdini,and is right that Houdart said here that Houdini 3 is scheduled to be out in september,you will have about 5 months to do the work;you think it will be enough?it will be very important an high agressive and top rated Komodo 5,or even a updated Komodo 4 with that pawn break to catch Houdini before 5 version is released
Parent - - By lkaufman (*****) Date 2012-02-13 03:23
The pawn break could be implemented and tested in a day, but optimizing it could take a week. I don't think we'll need 5 months to catch Houdini, except perhaps at bullet chess.
Parent - By Barnard (Bronze) Date 2012-02-13 03:27
just one day to implement,and one week to implement?too fast...well,if you catch Houdini at bullet chess,im sure at long time controls Houdini will be at least 40 elo points above than Houdini
Parent - By leavenfish (***) [us] Date 2012-02-15 01:42
As a paying customer who purchased Komodo4 specifically for anaylsis and because of your trying to make it best at anaylsis first and foremost, I say please just do what you think is best.
Parent - - By Werewolf (*****) [gb] Date 2012-02-13 08:56

> Do you have any other ways in which you think Komodo should be more aggressive?


Well, what about the simple bonus of swarming the enemy king with pieces? Do you think this is what HIARCS does? I can't decide whether HIARCS has superior King-safety (king-attack) knowledge or whether it just gives a high bonus to pieces near the enemy king. It's just that usually the beginning of an attack lies with pieces being in range of the enemy king, even when there is no direct threat to start with.

Presumably though you've experimented with the co-efficient of doing this quite a lot.
Parent - By lkaufman (*****) Date 2012-02-13 16:18
Yes, of course we already have this sort of thing in Komodo, in fact our method is quite unique and complex. We can easily make the coefficient higher for more aggressive play, but it is already optimized so that would lower the elo.
Parent - - By Master Om (Bronze) [in] Date 2012-02-16 08:46
I have few points to make it aggressive.

As you said what is Aggressiveness ?
For me its the King Hunt. An engine which knows opponents king will be called Aggressive. Chess's has only one aim of killing the king. Thats what An aggressive program does!!.

Remember I am not saying playing like tal is aggressive, But rather Playing like Rashid Nezmetedinov or Lenoid Stein is aggressive.
Playing positional being tactical and playing tactical being positional is totally different.
For example , for 1st one its like Blackburne , Spielmann , Nezmetedinov ...who actively placed pieces where they correlate enourmously seeking tactics and when situation demand they pounce with force. Inbetween also they play silent positional moves. Rybka 3 , Fritz 10.1 , Stockfish 1.8 was like that.
2nd one is like Petrosian, Spassky , Alekhine , Capablanca..... who play positionaly like play safe and never let tactics miss.

So the thin line is the approach. A Tactics Seeker and A Tactics Waiter is what the difference. Komodo is a Tactics waiter but not much. Its The Pyhons way all long to the endgame. When Komodo wins Its convincing always.

I still feel Houdinis method of play is like Capablancas who was Bullet fast in playing games. Still looks mysterious how he could be so spot on as fast as possible even in complicated situations. Thats the way he searches the given position and he was good in finding simple continuations leading to win. But Alekhine who beat him was not good at all in bullet/blitz chess rather good in blindfold chess where it takes lots of time to think ( Komodo's approach ). So as A GM u must be aware how capa would have been playing fast ?

So its a request please add king safety evaluation in Komodo 4.
Parent - - By lkaufman (*****) Date 2012-02-16 17:25
There is already quite a lot of king safety stuff in Komodo, but of course it can always be improved.
Parent - - By Master Om (Bronze) [in] Date 2012-02-17 01:19
The setting is not in Komodo 4 but in Komodo 3.
Parent - - By lkaufman (*****) Date 2012-02-17 02:37
Oh, you mean to make it a user option. I misunderstood.
Parent - By Master Om (Bronze) [in] Date 2012-02-17 07:36
yes.
Parent - - By Razor (****) [gb] Date 2012-02-17 06:07
Sorry to inject a comment here in to your thread with Barnard; couldn't help it!  :smile:

When I use K4 {actually this feels true for all engines I use/have used} I get a sense that the style is fairly fixed.  What I mean by this is that there is no mechanism for adjusting the style of play dynamically as part of game play - true?

On the assumption that what I sense is true I would spend some energy on considering this.  Dynamically adjusting the style of play based on a number of factors {including opponent's play} would seem to me to bring a richness missing from engines that play chess.  Whether or not this adds any ELO I couldn't say, although perhaps it would for certain factors and perhaps a checkbox {On/Off} could be made as a 'User Parameter'.

The above approach is one I and I'm sure you use/have used when playing chess, as we adapt our play to what is in front of us.  Those that do this well are certainly much more difficult to beat!

Just a thought!  :smile:
Up Topic Rybka Support & Discussion / Rybka Discussion / Rybka and Houdini at 40/120 Timecontrol?
1 2 3 4 5 6 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill