Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / How to encode endgame tablebases
- - By Mr. beo (*) Date 2013-10-18 10:47
Hello

Does anybody know how to encode endgame tablebase to get list of FEN positions? Is there a tool for that? For example, we have 4-piece Nalimov (or other) krkr tablebase. We need to get text file with something like this:
Draw|8 / 8 / 5K2 / 3k4 / 8 / 8 / 6r1 / 7R w - - 0 1
White wins|8 / 8 / 5K2 / 3k2r1 / 8 / 8 / 8 / 7R w - - 0 1
e.t.c.

P.S. I placed additional spaces in FEN format to avoid trasformation to picture.
Parent - - By syzygy (***) Date 2013-10-19 17:07
Although I generally don't like to ask this question: why would you want to do this?

A tablebase as simple as KQKR has about 3.5 million positions. What do you want to do with a file containing 3.5 million of these lines?

Note that existing tablebases never come in a format like this, because it takes up way too much space and is very inefficient (practically useless) for looking up a particular position.
Parent - - By Mr. beo (*) Date 2013-10-19 18:39
I wanted to use this in FinalGen special version:
http://www.mtu-media.com/phpBB3/viewtopic.php?f=3&t=33
where you can concede positions by writing in text file something like this:
W|8 /8/8/PkP5/1P4p1/5pKp/8/8 b - - 0 1
B|8 /8/PkP5/1P4p1/5pKp/8/8/8 w - - 0 1
The idea was to use Nalimov or other endgame tablebases in FinalGen to avoid plenty of indefinitness in it's solutions like "Unsolved" or "White wins or Draw".
Parent - - By syzygy (***) Date 2013-10-19 23:02
Ok, that makes sense. I don't know if FinalGen is able to cope with so much information, though (maybe it is). But I would have expected FinalGen to generate such tablebases itself (at least the smaller ones).
Parent - By Mr. beo (*) Date 2013-10-20 09:42
Unfortunately, FinalGen cannot handle with some simple 4-pieces positions. One of them is krkp, where white can't stop black from promoting a pawn, but white can exchange rook for that pawn, so it must be a draw, but FinalGen cannot see it and marks position as "White wins or Draw". There are plenty of such krkp positions which lead to draw (i think thousands of positions for a pawn fixed at one line). So if you have initial position with, for example, 8 pieces, some moves could be marked as "White wins or Draw" instead of clear "Draw".
Of course it would be better to have some uci engine that can use FinalGen tablebases with the help of dlls written by FinalGen author:
http://www.mtu-media.com/finalgen/clickdescargainterf.php
That is because engine could follow some thousands of lines marked as "White wins or Draw" and understand that it's clear draw. And other billions of positions would be solved by FinalGen, so engine would get complete solution to the endgame very fast.
Unfortunately, I dont know how to make, for example, Stockfish to use any endgame tablebases, because I am not chess programmer.
- - By h.g.muller (****) Date 2013-10-18 11:21 Edited 2013-10-18 11:24
Tablebases are usually compressed, and it is not trivial to decompress them. It would still be very easy to write such a tool, though, using the probing code that comes with them. Basically the program would look ike

for(EVERY_POSSIBLE_POSITION) {
  result = ProbeTablebase(position);
  if(result = WIN) printf("Win wins"); else
  if(result = DRAW) printf("Draw "); else
  if(result = LOSS) printf("Black wins ");
  PrintFEN(position);
  printf("\n");
}

That would be all. How the position would have to be represented depends on the probing code. It could just be a number, for instance. Then the for loop could be a simple loop with an index running from 0 to a billion (for 5 men). But the PrintFEN would then have to decode that number to the location of the various pieces on the board.

The file would get pretty big, though. Like 30GB for a single 5-men.
Parent - - By Mr. beo (*) Date 2013-10-18 12:09
Your program looks very nice, but where can I get functions ProbeTablebase and PrintFEN? Are they available for Nalimov or for another endgame tablebase?
Parent - By Vegan (****) Date 2013-10-18 13:19
hope you have a supercomputer, EGTB are extremely complex and what you want to do will require a tremendous amount of code
Parent - - By h.g.muller (****) Date 2013-10-19 15:08 Edited 2013-10-19 15:11
Tablebases usually come with functions to probe them. You would have to look at the documentation of the particular brand of tablebase to know how to probe them (i.e. what function to call, and how to specify the position you want to know the result for).

I am afraid that you would have to write PrintFEN yourself. It would be a quite trivial exercise, but how it exactly would have to look depend on how the encoding format the tablebase requires for the positions. As I have never used Edwards, Nalimov, Gaviota or Syzygy tablebases, but always generate my own, I cannot be of much help there. I imagine that the various format would allow you to access a tablebase by supplying a position number, which might just be the concatenation of the 6-bit square numbers (e.g. 4096*wKingSqr + 64*wQueenSqr + bKingSqr for KQK). It could also be that they compute that themselves, and you only have to supply individual square numbers (e.g. Probe3Man(wKingSqr, wQueenSqr, bKingSqr);).

It should be pretty simple, though. There are many open-source programs around that can print FENs from a board represented as an array, and you could use those after setting up the board according to the format they expect for the position you want to probe.
Parent - - By Mr. beo (*) Date 2013-10-19 18:32
Thank you
I've found function for probing Nalimov tablebases, but it seems that i can't use it without permission from Eugene Nalimov...
Parent - By h.g.muller (****) Date 2013-10-19 19:16
I suppose private use would be always OK, and that you just cannot include it in a program you intend to distribute. But you hit upon one of the major drawbacks of Nalimov tablebases, that you need permission from someone who doesn't answer his e-mail. Which is why most people switched to using more modern and performant formats.
Parent - - By syzygy (***) Date 2013-10-19 23:12
It would not be too difficult to adapt my generator to output such files:
https://github.com/syzygy1/tb

Instead of decoding a tablebase file, you would generate the table (which is a matter of seconds for 5-piece tables), loop through it (in tbgen.c after calling iterate()) and output the information you need. There is already a print_fen() function in stats.c. For pawnful tables you need tbgenp.c and statsp.c. The meaning of the values of table_w[idx] and table_b[idx] can be found in rtbgen.c / rtbgenp.c in combination with stats.c / statsp.c where the information is collected and the total number of wins, draws etc. is counted.

If you're not a reasonably experienced programmer this will be pretty hard, though.
Parent - By Mr. beo (*) Date 2013-10-20 09:54
At glance it looks very hard for me. I work with c++ code almost every day but I am newbie in chess programming...
Up Topic The Rybka Lounge / Computer Chess / How to encode endgame tablebases

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill