Not logged inRybka Chess Community Forum
- - By Kapaun (****) [de] Date 2007-10-09 10:45
What is the valid margin of centipawns you can set up the randomizer with? 1 - 99? Or can you go higher than 100? If so, how high?
Parent - - By Felix Kling (Gold) [de] Date 2007-10-09 12:57
You can set it also to very high values, this would lead to strange results since the engine would play EVERY move then :)
Maybe 29900 is the limit if there is any :)
Parent - - By Kapaun (****) [de] Date 2007-10-09 13:34 Edited 2007-10-09 13:47
Well, these days I'm experimenting a bit with the randomizer (due to your comment a while ago), and there is a funny thing I don't comprehend - wondering if I'm doing something wrong:

If I set up a match between two randomized Rybkas (fixed depth of 7) and let them work on the starting position, while the centipawns are set to 80 - then nearly each and any move is used (14 or 16 of the 20, or so), but NOT (!) 1.e4....

Huh? What's that? Should simply be impossible. No way 1.e4 is considered more than 80 cps worse than the best move!

And by the way, I don't believe that any of the starting moves is considered that low, so they should all be used, right?

Any ideas?
Parent - - By Felix Kling (Gold) [de] Date 2007-10-09 14:34
mmh, strange... maybe a bug in the randomizer?!
Parent - By Kapaun (****) [de] Date 2007-10-09 15:58
Well, I just repeated the whole process from the scratch - starting with creating new randomized Rybkas, but the results don't seem to change. Guess that means that the results are not reliable (not even inside the intended boundaries) because moves can be left out that should be included...
Parent - - By Fulcrum2000 (****) [nl] Date 2007-10-09 16:25
Maybe your book is still active and has e4 marked red?.
Parent - By Kapaun (****) [de] Date 2007-10-09 16:31 Edited 2007-10-09 16:36
Good thinking, but no. Book is deactivated and in the match setup an empty .ctg book is included, just to be sure. Besides that I don't have any book with e4 redded out... ;-)

By the way: Maybe someone can reproduce the problem? Or can't reproduce, which would be interesting as well?
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-09 19:46
That sounds quite strange. I added this to the list of things to check.

By the way - 80 centipawns is too big a margin for best results. I would recommend somewhere around 10 to 20.

Vas
Parent - - By Kapaun (****) [de] Date 2007-10-09 20:14 Edited 2007-10-09 20:25
By the way - 80 centipawns is too big a margin for best results.

That may be so. I was just having this idea to check if the randomizer perhaps could not only be used to get a general evaluation of the position, but also to determine the best move. But that would mean having bigger margins in order to not miss it - because at low depths (which are necessary to get the needed big amount of games) said move may well hide a bit down the ladder...

By the way - if this really is some kind of bug, it might be connected with using such a big margin - which probably has never been tested. Guess I'm going to try a lower margin tomorrow. Just to see, if that changes something...

Edit: I'm generally having the impression that not all moves are used which are within the margin. If I try the same process with the position after 1.e4 c5 - then only 5 moves are used (Nc3, Nf3, Bc4, Bb5 and Bd3). I even tried what happens if I rename the config file Rybka.config to Rybka.config.txt - but then only one move at all is being used, so I guess the config file is correct.
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-09 20:22
This is logical, but the problem is that the huge margin will be used throughout the entire tree.

It might make sense to have a randomizer version which considers every move at the root. You can also do this manually by randomizing positions after the interesting root moves.

Vas
Parent - - By Banned for Life (Gold) Date 2007-10-09 20:25
It might be even better if you could set the threshold at each ply so that the it got smaller with larger ply (and maybe even down to zero at some point).

Alan
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-09 20:30
Maybe I'm missing something, but this doesn't seem useful. Keep in mind that the margin is only used when a position is revisited. A margin of zero means that the same move is played (in single-processor mode).

Vas
Parent - - By Banned for Life (Gold) Date 2007-10-11 16:04
If the randomizer is being used to help evaluate a particular solution (and life is too short for anything else), it makes sense to allow more branching closer to the root and less branching farther away. This is the approach used by Deep Position Analysis (which uses a fixed branching factor for succeeding plies). The randomizer could perform the same function much more effectively by allowing a variable delta based on ply depth rather than a fixed branching factor. I think this is a better approach than the "try all first moves" and then go to a fixed delta proposed above (although this could be implemented if desired with adjustable deltas per ply).

At some ply depth, going to zero (essentially reverting to shootout mode) makes sense to me. DPA does this at ply 5. I'm not sure if this is optimal, but I do think that at some point you just want to say its a win/draw/loss.

Regards,
Alan
Parent - By Kapaun (****) [de] Date 2007-10-11 17:35
Now, this is really starting to get interesting. I'd love to compare the results of something like that to the results of the DPA feature! :)
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-13 09:07
I understand the proposal but not the logic behind it.

With a fixed non-zero delta, you make sure that at every step of the tree, each valid (ie. within the delta) sub-branch gets an equal number of games. If your delta ever goes to zero, this property (which is IMO desirable) goes away.

Vas
Parent - - By Banned for Life (Gold) Date 2007-10-13 15:40
OK, but this is not the main thrust of the argument. The main point is that the delta should be greatest at the root and decrease from there. The final value could be carried for all remaining plies.

We've all seen positions where the eval has a significant error at or near the root position. It would be useful to allow use of the randomizer without either requiring all plies to have a huge delta, or only the first ply to have this delta. Allowing a delta which necks down as a function of ply depth seems like a reasonable solution.

Regards,
Alan
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-15 14:02
Maybe I'm being really dense - but why should the delta be the greatest at the root? You can have errors anywhere in the tree.

Vas
Parent - - By Kapaun (****) [de] Date 2007-10-15 15:21
Because with an error 6 moves away there is still some opportunity to correct it completely or even partly or to circumvent it in the upcoming moves. An error directly ahead, though, may kill you.
And on the other hand it's just a question of time to have the delta great not during all the moves...
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-17 09:54
Moves closer to the root do get more games. The question is how resources are split amongst the candidate moves at some node. I don't see why this shouldn't be done the same way throughout the tree.

Vas
Parent - - By Kapaun (****) [de] Date 2007-10-17 10:06
Well, if I'm correctly understanding Alan's idea this modified randomizer would not be used mainly to get lots of games and then to draw conclusions out of statistics (like it is now) but rather to analyze a position with higher ply depths. Okay, maybe I'm just transporting my idea into his - if so he should say. :-)
Anyway in this case a delta shape would be better...
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-17 10:29
Even if you're trying to construct a search tree instead of a statistical simulation, I don't see any reason to use different algorithms near the root than in the middle of the tree. The resources are bigger (and the stakes higher) at the root, but the relative prioritization should be the same.

Vas
Parent - - By Kapaun (****) [de] Date 2007-10-17 11:16
I'm not sure if we are not maybe talking at cross-purposes, because it seems obvious to me. Again: If I analyze a position and miss an important move right at the root, this may well be deciding the game. If I miss an important move some moves ahead, this may bias my analysis and my choice of move - but there are chances that I will be able to correct the mistake either completely or at least partly, for instance in the next move.
So given scarce ressources (= calculation power) it seems better to me to have a broad delta at the root, because mistakes up ahead may not be so important yet.
Parent - - By Banned for Life (Gold) Date 2007-10-17 16:59
As you've stated, it seems clear that:

- There are many critical positions with significant evaluation errors where you need to set the delta as high as 0.5 to include the best move (even at a pretty good search depth)
- Life is too short to run the randomizer with 0.5 for all plies
- Losing a best move later in the game is less critical than losing the best move at the root (as is clearly stated by Kapaun above)

Assuming the randomizer works in a way that tries all possibilities at the root before trying lower scoring moves at greater ply depths, the difference between the two approaches will be that having a decreasing delta will finish the search faster at the potential expense of missing a move that would otherwise have been seen at a larger ply depth due to the decreasing delta. I see this as a good (and really a necessary) trade-off if you want to use a high delta in the first few plies and still get results in a reasonable time frame.

Regards,
Alan
Parent - By Kapaun (****) [de] Date 2007-10-17 17:29
Yep, exactly my thoughts...
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-19 12:09
If the delta is the same, moves closer to the root still get more resources (in Monte Carlo, more games).

If you're playing 1000 games, and have 5 root moves within the delta, then each gets 200 games. Somewhere deep in the tree, you may again have 5 moves within the delta, but each only gets 10 games.

Vas
Parent - - By Kapaun (****) [de] Date 2007-10-19 14:26
Yes, that is obvious. But what if the winning move at the root were number 6?
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-20 16:32
If this happens in a Monte Carlo simulation, then your results will be skewed away from the side to move.

If on the other hand you accept a sixth move only at the root, and this move is as bad as its score would suggest, then the results will also be skewed away from the side to move.

Think of it recursively. After each root move, you again are basically doing a Monte Carlo simulation of the subsequent position. Why would you want to use a different algorithm?

Vas
Parent - By Kapaun (****) [de] Date 2007-10-21 13:12
Actually, for practical reasons. To save time. To find the optimum of time usage, depth, benefit and reliability. And because a mistake at the root would be much more severe than further ahead (as has been pointed out several times), we would need a delta shape anyway. The more I think about it: What we are talking about is practically identical to the DPA feature in the Fritz GUI....
Parent - - By turbojuice1122 (Gold) [us] Date 2008-06-03 23:04
Actually, I think the main reasons that we want a randomizer that changes the eval window with depth have to do with the fact that there is already far too much garbage in the results of the current randomizer.  Since it's true that it is more likely for one side to be able to avoid a position that is misevaluated by 0.5 units that is six moves away than one that is two moves away, it is more likely that the results six moves away are going to be completely garbage.  In the initial position, however, the opportunity for misevalation is at its highest, and the randomizer has the potential of remedying that if one can obtain enough non-garbage games to give a good sample size.  This is what one does by reducing the window after a few ply.

A second major issue is that the nature of the game can change after tactical possibilities.  In many positions, there are definite tactical possibilities that result in long-range strategical possibilities that are misevaluated by engines.  In tactical positions, one wants to have a high eval window, but in strategical positions, one needs a relatively low eval window because there are more moves available within such and such window, but if the window is still the same as in the tactical case, the vast majority of positions will be bad one way or the other due to an accumulation of inaccurate moves, and one has wasted a lot of time on those positions when the time would have been better spent looking at future ply to get a deeper tree.
Parent - By Vasik Rajlich (Silver) [hu] Date 2008-06-04 11:10
Point 1 seems somewhat valid. The more games you will play in a position, the more moves you can afford to consider - this seems logical.

It also makes sense to have a bigger window in sharper positions, but this will be hard to do.

Vas
Parent - - By Kapaun (****) [de] Date 2007-10-09 20:35
Yes, I guess that might work. Quite time consuming, though. Probably right now it's still more effective to search in the traditional way for the best move, but with hardware power always growing the day might not be so far away when a random approach would be more successful.
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-11 12:17
I doubt that the randomizer approach will ever be the best way to determine the best move. It could be that some sort of a hybrid approach will make sense at some point.

Vas
Parent - - By Kapaun (****) [de] Date 2007-10-11 12:39
As far as I've read somewhere the "Go" engines are already quite successful using a combined approach.
Parent - - By Felix Kling (Gold) [de] Date 2007-10-11 13:00
In Mainz there was a short discussion between Stefan Mayer Kahlen and Vas on this topic. Stefan Mayer Kahlen seemed to be much more optimistic for the future of this method than Vas. I guess Shredder will have some kind of randomizer soon, too.
I think when we have computers with a really big number of processors, this approach will become more interesting.
Parent - - By Vempele (Silver) [fi] Date 2007-10-11 13:58
Anthony Cozzie tried it on his WCCC 2006 machine, and concluded it wasn't good enough. How much bigger can you get?
Parent - - By Felix Kling (Gold) [de] Date 2007-10-11 14:50
Well, as Vas said, a hybrid solution may be more effective... anyway, this is pure speculation, so let's wait and see what the future will bring us :)
Parent - - By Kapaun (****) [de] Date 2007-10-11 15:25 Edited 2007-10-11 15:34
Interesting. Vas should watch out! By the way, wouldn't the randomizer itself be a hybrid solution if you would chose higher plies? Would cost an awful lot of time, but then, behind a certain point the usual way costs an awful lot of time, too...
Parent - By Vasik Rajlich (Silver) [hu] Date 2007-10-13 09:10
Well, it's hard to be sure. Sure, Monte Carlo works in some games. Howeve, Go is quite different from chess. Besides, it's not even clear that Monte Carlo is really best in Go - it just happens to be the best right now.

Vas
Parent - - By Kapaun (****) [de] Date 2007-10-09 22:17
Okay, we are closing in on the problem: It seems that regardless what I write into the config file (randomize=80 or 20 or 10 or even 5) always the same 14 moves of the starting position are being used. And don't tell me my config file doesn't work, because obviously it does: Each time I change this file (for instance save it in another txt format like Unicode or UTF-8 instead of ANSI), it really does not work, which means that only the very first move (in this case 1.Nc3) is used over and over again.
Now I'm running out of options here. Only two reasons I still can think of are: It's either simply a bug or it's some kind of GUI issue, and a randomized Rybka does not work correctly together with the Fritz GUI.
Parent - - By Sesse (****) [no] Date 2007-10-10 21:41
Err, if you save an ASCII file as UTF-8, that's really only the same as normal ASCII...

Perhaps the byte order mark Notepad (erroneously) inserts confuses the .ini parser, though.

/* Steinar */
Parent - By Kapaun (****) [de] Date 2007-10-11 05:46
Whatever you say, Sesse. But indeed it does not work with UTF-8. Try it yourself...
Parent - - By Vasik Rajlich (Silver) [hu] Date 2007-10-11 12:19
It sounds like a bug. Maybe there is a limit on the number of moves. I bet that it doesn't invalidate the scores for the root position, anyway I will take a closer look when there is time.

Vas
Parent - By Kapaun (****) [de] Date 2007-10-11 12:36
Maybe there is a limit on the number of moves.

Don't know, doesn't feel like that.

Guess if I were you I'd first check if maybe the config file is not (under certain circumstances?) correctly "translated" to the Rybkas.

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill