Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Aquarium / Tree configurations explained
- - By Zruty (*****) [ru] Date 2008-08-14 16:14 Edited 2008-11-19 15:09
Tree configurations explained

Overview.

There is a very good introduction to the Tree view in the Aquarium manual. What I want to do here, is to focus on the tree configuration philosophy that we have implemented in Aquarium.

This article may seem hard to understand. I put all the information here, and the complexity increases as I proceed. So if you don't understand some part of it, don't worry - this will come later. For most difficult tasks we also plan to introduce simpler approaches.

Tree Configurations.

Tree Configuration is an entity that is used everywhere in Aquarium. Tree Configuration consists of several columns that can be of different type. Each column has a tree associated with it, and the column is filled with the data from the corresponding tree:



As you see on the image, there are 4 columns currently displayed in the Tree window, apart from moves themselves and flags. Each of the columns (Eval, Total, CAP, Annotation) is based on a different tree file.

Moves shown in the tree window are all the moves that are present in ANY of the trees, or in the game notation.

The advantage of the tree configuration approach is that you don't need to copy the DATA to display it in different way. Different tree configurations may refer to the same trees. Data is stored in trees, appearance - in the configuration.

You can edit any tree configuration in any way: include or exclude columns, change the column types and trees. We'll discuss it later in detail.

Main Tree.

Every tree configuration has two additional trees - Main tree and Discarded tree.

Main tree is responsible for:
* Move colors
* Move ordering (you can reorder moves my pressing Shift+Up and Shift+Down in Tree window).
* Flags ('flg' column)

Discarded tree is responsible for:
* Discarding moves

Setting Up Tree Configurations.

To alter the tree configuration, click Tree Options button on the Ribbon:



You'll see the Tree Configuration Options dialog:



Here, you can add and remove the confuguration's columns, create and remove the configurations themselves, re-order columns, render them visible/invisible, change the associated trees.
You can change the Main tree, by clicking Advanced button first.

I suggest you always create a copy of the existing configuration when you are going to experiment with it until you know what you are doing :)

Column Types.

When you add a column to a tree configuration, you must select the column type:



The available column types are:
* Number of games - number of times the move have been played
* Success - percentage of points taken by a side to move after playing the move
* Elo - maximum Elo of all who played the move
* Year - last year when the move was played
* Human evaluations - position evaluation in symbolic form (+=, ...)
* Computer evaluations - position evaluation in numeric form (+0.19)
* Percents - not implemented for now
* Categories - not implemented for now
* Reliability - not implemented for now
* Played percents - percents of times the move have been played, with respect to other moves
* Annotation - text and graphical annotation
* Hash Score - contents of Rybka 3's hash. Filled during analysis.
* Hash Depth - contents of Rybka 3's hash. Filled during analysis.

Usage as Opening Books.

Any tree configuration may be used as an opening book. Some of them are not suited for it - the engine will start playing from the very beginning. In the release version, the following configurations are valid opening books:
* NarrowBook
* WideBook
* CTGExample
* HandicapBook
* Annotations

Every time you create a copy of one of the above tree configurations, your new configuration can be used as an opening book.
The first thing you see when viewing such a configuration is a 'Play%' column that displays a probability of playing each of the moves. These probabilities are used when the engine is playing with this book.

Editing opening books - a simple way.

You can edit the opening book percents by marking the moves green, blue, orange and red, and discarding them (5,4,3,2,8 keys). The percents are immediately updated. The rule is:
1) Play only green moves, if possible.
2) If no green moves available, play blue or black moves, of possible.
3) If all moves are discarded, orange or red, don't play any move and pass the position to the engine.

This way you can configure your opening book to play certain moves.

Editing opening books - a versatile way.

Now we have to dig deeper into the Tree Configurations Options dialog:



Click 'Settings', and look at the following Advanced options dialog:



The 'Move colors' button enables you to change the policy by which the moves are selected. You can do this by dragging the colored squares up and down:



As you see on the above image, the current policy is the default one: make green moves first, if not available, make black and blue, never make orange, red and discarded.

As you have figured out, you can, for example, set the priority of 'interesting' (blue) move highest, etc.

Now the hardest part of it - bonuses.

As you have seen on the image, there are several other columns in the tree configuration, not shown in the first dialog. Most of them are of 'Bonus' type:



You can make these columns Visible, and you'll see the numbers in the tree configuration. They are bonuses, and they define the move percentages.

For each move we calculate the bonus in the following way:
* We find the column which 'ID' is the same as the bonus's column 'Column'.
* We take the numerical value X out of this column (for our current move).
* We take the function F which name is 'Function'.
* We calculate the Y = F(X). For example, for the 'identity' function F(X)=X, for 'constant' function F(X)=1, etc.
* We multiply Y by 'Weight': Bonus = Y*Weight.

All of the bonuses for a given move are summed up. The total bonuses are normalized and treated as play probability. Note that move colors take effect before this, so the move color policy hase higher priority then bonuses. But, when the move has non-positive bonus, it is never played, no matter what its color is.

How to know the function names? There's no GUI for this, but they are stored in the Config\tree_configurations.xml file. For those of you who is interested in this stuff, I'll explain even further, since I still omitted some of the opening book features.
Parent - - By Victor Zakharov (*****) [ru] Date 2008-08-14 16:57
Hope that this "Trees: getting started" description will help to use power of Aquarium trees better.

I just want to clarify the "discarded" Flag. Discarded Flag covers move collors i.e. colored moves temporarily lose their colors in opening book. Discarded Flag is adviced to be used for experimenting with books. For example you have three green moves 1.e4, 1.d4 and 1.c4. For some day you decided to play 1.c4 only to understand how good it is for some new engine. Surely you may unmark green moves 1.e4 and 1.d4 . But later you can forget what color was used for these moves, especially if these moves are not the first moves in the game. So another decision was accepted during beta testing - to discard moves. Discarded moves remember their color, but this color is ignored during game play. So you always know that you discard a move for some reason and are going to restore its initial state some time later.
Parent - - By Steffelino (**) [se] Date 2008-08-14 17:03
Oh I am happy about this thread!
My questions is, can I reset the config. to default?
can I change the blue background color  in Aquarius to wood for example?
Parent - - By Zruty (*****) [ru] Date 2008-08-14 19:32

>can I reset the config. to default?


We plan to release the corresponding utility to restore the various values to factory defaults.
Parent - By Steffelino (**) [se] Date 2008-08-14 19:36
ThankĀ“s very much!! Hope soon!
Parent - - By Zarkon (***) [th] Date 2008-08-14 23:47
Thanks for the very clear description and overview.

However, I wonder if more than a handful of people are interested in the more refined settings, such as bonuses. Such apparent complexity can also be a turn off for users, if they feel that the GUI is not targetted for their use.
Parent - - By Zruty (*****) [ru] Date 2008-08-15 08:54

>Such apparent complexity can also be a turn off for users, if they feel that the GUI is not targetted for their use.


I think that move coloring is sufficient enough for most opening book tasks.
Parent - By Felix Kling (Gold) [de] Date 2008-08-15 10:38
I also think so. Since the colours are quite variable, it should be possible to do everything you need.
Parent - By buffos (Silver) [gr] Date 2008-08-15 09:20

> Such apparent complexity can also be a turn off for users, if they feel that the GUI is not targetted for their use.


95.00% of the users will not use it but i guess it does not harm to be there for the other 5%. Besides it under the advanced settings (although its very advanced :) )
But i am sure the 5% that will create some useful configurations may share it with the other 95%.

Ofcourse for a first presentation of Tree configuration, i agree that the part on advanced tab setting should better be omitted because for many users is overhelming and does bad and not good to be there.

Maybe the best place to be, is under a 'Aquarium Tree configurations ADVANCED explained'

I think its best to remove the advanced part from this reading, so users are not "afraid"
Parent - - By cma6 (****) Date 2008-08-18 03:04
Zruty's article taken together with Buffos' videos on the subject of Trees are quite useful.

However, the average user probably would be more than satisfied if he were capable of creating and maintaining his own "Opening Repertory" Tree. There are valuable hints on this subject in the above mentioned resources, but I can see it will take much experimenting to learn how to create one's own Tree.
Parent - - By buffos (Silver) [gr] Date 2008-08-18 05:20
if you want to fast start experimenting do the following

1) Create a blank new configuration
2) Click Advanced and add as  mainbook yourrepertoirename.hsh  (there need not be such file. Just enter the name)
3) Put as discarded book the same
4) You can optionally add columns from other trees if you want
5) Now go to the sandbox (this is where you best create trees)
6) If you want to add moves to your tree simply enable "add moves to main tree" by right clicking inside the tree configuration

What ever you play is recorded (entered) in the tree. If you want to stop adding moves to the tree just rightclick again and untick add moves to main tree

DONT BE AFRAID TO EXPERIMENT!!!

You can always delete you treename.hsh . Use it as a testing box. You cannot learn if you dont make your hands dirty
Parent - - By cma6 (****) Date 2008-08-21 00:29 Edited 2008-08-21 00:53
I went into the Tree Menu and tried to Load a tree (*.dst file) but nothing loaded, so I am stuck at step 1).

I followed p. 74 of the manual on building a tree. "Select games you want to build the tree and then click "Build Tree" button in Tree tab to display "Tree Build Options" dialog.

  After I left default parameters and clicked OK, I got another dialog "What do you want to do now?"

  Selected "Create New Configuration" since there is no configuration. This displays tree configuration wizard.

  However, no tree was created. So we are back to square 1 without worrying about the "Tree philosophy": how to create a tree?

The only tree I can find is the "Deep Analysis tree" which I believe was automatically created from IDeA.

Before one can adjust or perfect a tree, one has to be able to create one.
Parent - - By buffos (Silver) [gr] Date 2008-08-21 06:39 Edited 2008-08-21 06:43

> However, no tree was created


The tree was created because the wizard appears after the tree is created.

Now some clarifications to make sure we communicate.

1) Goto Sandbox. Tree Tab. Select tree configurations from the first button in tree tab (the one that has Narrowbook, widebook, etc...) So if you create a configuration you will find it there under the name you chose
2) I guess when you created a new configuration you where asked for a name. Go to the tree configurations button and select it to make it active.
Parent - By cma6 (****) Date 2008-08-21 18:19
Thank you, Buffos, that was the key information we needed: where and how to find a created tree configuration. Without your clarification, one could not have found that.
Parent - By whiteb [us] Date 2008-09-07 20:24 Edited 2008-09-08 06:26
I have a Question on Trees, books and colors. I just recieved The new open book and have an open that starts off like this:

1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Ba4 Nf6 5. O-O d6 6. Re1 Be7 *

Rybka3 Open Book has move 7. as "c3"  being the better move and colored in green, However the infinite analysis recomends 7.Bxc6, which is colored in red on the opening book. Also all analysis that ids done at that position shows 7.Bxc6 is a stronger move that 7.c3; any recomendations

Bobby

Added by moderator:
Please do not make double post! It has been reported that the same post also appear in a separate post
http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=7249
Parent - - By buffos (Silver) [gr] Date 2008-08-15 16:40
Have just finished demystifying the advanced settings xml file.

I just have to say. The power hidden in there is endless.

Yes its not for everyone, but hey, i use foobar2000 media player and dont know how to make all those crazy configs, but others make it and share with everyone else.

Whoever thought of doing it so versatile is simply a genius.
Parent - By Zruty (*****) [ru] Date 2008-08-15 16:51
Oh.

:)
Parent - - By JohnL (***) Date 2008-08-15 18:37
The available column types are:
* Number of games - number of times the move have been played
* Success - percentage of points taken by a side to move after playing the move
* Elo - maximum Elo of all who played the move
* Year - last year when the move was played
* Human evaluations - position evaluation in symbolic form (+=, ...)
* Computer evaluations - position evaluation in numeric form (+0.19)
* Percents - not implemented for now
* Categories - not implemented for now
* Reliability - not implemented for now
* Played percents - percents of times the move have been played, with respect to other moves
* Annotation - text and graphical annotation
* Hash Score - contents of Rybka 3's hash. Filled during analysis.
* Hash Depth - contents of Rybka 3's hash. Filled during analysis.


I'm missing ELO performance here. Are you planning that too?
Parent - - By buffos (Silver) [gr] Date 2008-08-15 18:47
Basically performance of who in what?

The only thing that comes to my mind is for a given position, have WhiteELO average, BlackELO average , so based on the %success you have the perfomance for white and black
So basically what you need is WhiteELO average & BlackELO average because the performance column can be calculated from existing data.

Is that what you have in mind?
Did i understand right?
Parent - By JohnL (***) Date 2008-08-15 20:20 Edited 2008-08-15 20:22
Yes!

I think that is very basic. The pure win-% often hides the true score.
For example weaker players may prefer the exchange variation in the slav after 1.d4 d5 2.c4 c6  making the win% too low.

Actually, what you really want is the difference of ELO performance and ELO of the players making that move.
Parent - - By cma6 (****) Date 2008-08-18 02:59
"* Success - percentage of points taken by a side to move after playing the move".

Didn't you mean: "* Success - percentage of points taken by a side that has just made the indicated move" ?
Parent - - By JohnL (***) Date 2008-08-18 22:03
I guess so!

I just checked and CA9 seems to miss this feature as well. But ChessBase9 has it... 
Is that why they don't give any answers here...?

Actually, I think it is quite important for the serious user.
Parent - - By Zruty (*****) [ru] Date 2008-08-19 13:58

>I just checked and CA9 seems to miss this feature as well. But ChessBase9 has it... 


Err, what feature?
Parent - By buffos (Silver) [gr] Date 2008-08-19 14:50
I think he meant this

>'m missing ELO performance here. Are you planning that too?

Parent - By Vempele (Silver) [fi] Date 2008-08-18 22:04

> Didn't you mean: "* Success - percentage of points taken by a side that has just made the indicated move" ?


Yes.
Parent - By Zruty (*****) [ru] Date 2008-08-22 09:21

>I'm missing ELO performance here. Are you planning that too?


Yes, when we start changing the tree representation.
Parent - - By dgordon (*) [ca] Date 2013-01-06 05:40
Is Aquarium able to generate Elo performance stats for each opening move now?  I agree that that would be a useful feature.
Parent - By Moz (****) Date 2013-01-06 17:27

> Is Aquarium able to generate Elo performance stats for each opening move now?  I agree that that would be a useful feature.


Sadly, no. Draw percentage was recently added to the tree structure so, in theory, it should easy to add this feature. Unless I'm mistaken, I believe all of the information necessary to calculate Elo performance is now stored in the tree.

Victor has stated that the team has plans to improve Aquarium's opening book functionality in the future. It would make sense for this to a part of that update.
Parent - By Gordon (**) [gb] Date 2008-08-16 15:20
Sorry if I just being stupid, but I still have Tree problem.
I have followed the instructions here (and in the video) and have created a ctg tree - there appeared to be no problems and everything has worked  - the same number of games as in the original Chessbase tree etc. However the Aquarium tree contains fewer moves in each game - it cuts off after about 10 moves. Does something control the number of moves stored and if so, what is it?
Best wishes
Gordon
Parent - - By Banned for Life (Gold) Date 2008-08-16 16:28
Thanks for posting. Before I read this, I had no idea that each column could be associated with a different tree.

Regards,
Alan
Parent - - By ppipper (*****) [es] Date 2008-08-17 16:13
These features of joining columns from different trees seems to be really powerfull.

However, I still miss some columns quite important in my opinion:

- ECO Code: for me, it helps a lot as a criteria to search for in a game's database
- Depth: in the analysis trees I think this should be very important. IMHO not only having the evaluation of a position is important, but also the depth reached with the analysis.

Thanks

Regards
Parent - - By Zruty (*****) [ru] Date 2008-08-17 16:39
We are going to revisit the tree structure. At the moment, we basically can't add any other columns because nothing else can be stored in tree.
Parent - - By Banned for Life (Gold) Date 2008-08-17 16:47
Understood. When you revisit, I hope you will consider not only ECO and analysis depth, as suggested by ppipper, but also the engine performing the analysis along with the ability to store analysis and depth from multiple engines. It doesn't hurt to ask, right? :-)

Regards,
Alan
Parent - - By buffos (Silver) [gr] Date 2008-08-17 17:06

> ability to store analysis and depth from multiple engines


well this is what tree configurator is.
Do you have a tree with analysis from Rybka and a tree with analysis from Hiarcs, you can see them in one tree.
So no need one tree to hold everything. So if you analyze with Hiarcs, name your tree HiarcsAnalysis.. (or smthg like that).
Divide and conquer  :)

ECO can be done i guess, as an annotation column (but manually would take too much time, so i wont volunteer :) )
With a computer utility this could be ready i guess very easily.
So you actually need just one tree , with just one column (Annotations)

For Depth , well no tools for that yet :)
Parent - - By ppipper (*****) [es] Date 2008-08-17 17:59
Of course I dont know database structure where trees are stored behind the scenes, but when you analyze the game (regardless of the feature - infinite analysis, IDeA, blunder checking, etc) it is supposed that the engine is providing all of these data: evaluation, depth, KNs, time, legal positions, etc.

If this is correct (I am not totally sure) the "only" problem is to store these data in the tree?

Regards
Parent - - By buffos (Silver) [gr] Date 2008-08-17 18:13
I guess depth is an interesting figure and worth the space, but legal positions in a tree structure? Time? KN?
This could be stored as comments in game annotation, but not in a tree
For example you analyzed position A for 10 hours and got to depth 25 and i analyzed the same position in another machine and got the same depth and evaluation after 10 minutes.
depth and evaluation are important and invariant results
Parent - - By ppipper (*****) [es] Date 2008-08-18 11:07
yes, I agree with you. KNs, time and legal positions might not worth the space in tree structure. But depth is important.

Another issue would be the possibility of creating a chart combining all this info (eval + depth). Currently, chart's feature looks promising but I think it sholud be revisited:
- to be able to get a chart everytime you get the whole game analyzed, regardless of the analysis type (infinite, blunders, IdeA, etc). Now you just get the chart in game analysis
- to be able to zoom / print specific part of the chart

Regards
Parent - - By buffos (Silver) [gr] Date 2008-08-18 11:12

> - to be able to get a chart everytime you get the whole game analyzed, regardless of the analysis type (infinite, blunders, IdeA, etc). Now you just get the chart in game analysis


You can do that.
Just add the computer evaluations of that tree to the current tree configuration and follow the same procedure. I have made an extensive tutorial on charts on this thread. Its not video, its text and pictures, but i covers a lot of stuff.

Infinite analysis can be always viewed in the chart. I guess many questions will be answered in the tutorial (and hopefully many more created , which means creativity)

I would also like to print or have a jpg of the chart. I like that tool very much
Parent - - By ppipper (*****) [es] Date 2008-08-18 11:22
wow impressive!

Thx very much and sorry if I have made some questions thay might be already answered in that tutorial. I'll came back once I have read it. :-)

Regards
Parent - By buffos (Silver) [gr] Date 2008-08-18 11:24
well i always ask questions. generally i like questions. (especially those without answers :)  )
Parent - By Banned for Life (Gold) Date 2008-08-17 18:33
Thanks for the correction. That sounds reasonable. Is there any thought to putting together a diff tool that would find big discrepancies between equivalent analysis from multiple engines?

Regards,
Alan
Parent - By Geomusic (*****) Date 2009-12-14 16:27
Is it possible to back convert from tree to pgn? for making opening books?
Parent - - By oudheusa (*****) [nl] Date 2008-09-06 14:25
Can you plse post a description of the various tree's that are delivered with Aquarium in the Atree directory including the fields in each tree and the source of the data (eg for the eval tree).

Thanks.
Parent - - By ppipper (*****) [es] Date 2008-09-10 22:51
buffos,

is there any fomula or bonus function to display in a *DA tree's column, how many moves ahead are in that tree for a certain position?

Thanks!

Regards
Parent - By buffos (Silver) [gr] Date 2008-09-11 05:45
as far as i know no.

Functions to read forward would be cool.
- - By RazgrizRWA (*) [jo] Date 2010-08-17 05:37
Thnx but I don't think I understan

I have the rybka 4 opening book in a hsh and elm files and I want to know if i can make rybka use this book

I just don't know how, any help is greatly appreciated
Parent - By buffos (Silver) [gr] Date 2010-08-17 05:58
read this http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=6296

also there is a lot of online help and online manual

> if i can make rybka use this book


ofcourse you can. Just take some time (i guess you are new to the interface) to read manual + look the options available (book for aquarium == tree configuration == multiple trees combines == multiple (.hsh and .elm) files combines in one table look

Just take some time to understand how configurations work (the first link has a lot of information , depending on your familiarity with computers you can do a lot)
Up Topic Rybka Support & Discussion / Aquarium / Tree configurations explained

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill