Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / 7-man tablebases
- - By ptobler (**) [au] Date 2009-09-25 04:28
Hi - a question for Vas. If we had 7-man tablebase files, and added them to the folder the others are in, would Rybka use them or is it programmed to only use up to 6-man tablebases?
Parent - By Sesse (****) [ch] Date 2009-09-25 11:31
Rybka supports only up to 6-man.

/* Steinar */
Parent - - By Uri Blass (*****) [il] Date 2009-09-25 11:45
why 6-man or 7-man and not 6-piece or 7-piece?

Queen is certainly a female so by definition 6-man or 7-man tablebases cannot include KQP vs KQP positions.

Uri
Parent - By Debaser (***) Date 2009-09-25 12:12
Not a native english speaker ;)

But man (also chess-man) means piece not only male human being. It is used to for draughts and backgammon too

Ancient quotes in the OED:

c1460 (?c1400) Tale of Beryn 1821 Comyth nere! Ye shul se..How he shall be matid with what man me list

1474 CAXTON Chesse I. iii, The table of the chesse borde and the chesse meyne.]

I suppose we should say 6 or 7-men not man
Parent - By InspectorGadget (****) [za] Date 2009-09-25 19:12

> why 6-man or 7-man and not 6-piece or 7-piece?
>
> Queen is certainly a female so by definition 6-man or 7-man tablebases cannot include KQP vs KQP positions.


Oh Master Blass you are killing me now :-D LOL
Parent - By Peter Grayson (****) [gb] Date 2009-09-25 19:14
The word "man" can be genderless for example, mankind, manning, to man, where it can cover men, women and chidren. Historically the chess pieces have been referred to as Chessmen therefore the use of "man" here is appropriate and correct.

Sometimes political correctness goes over the top and gets it wrong!

PeterG
Parent - - By buffos (Silver) [gr] Date 2009-09-26 16:11

> Queen is certainly a female


not sure...."she" has never gone through an DNA gender test
Parent - By Banned for Life (Gold) Date 2009-09-26 16:55
Napoleon once claimed that Queen Louise was the only real man in Prussia.
Parent - - By Felix Kling (Gold) [de] Date 2009-09-25 12:24
There are no 7 men tablebases yet, so Rybka can't support it :)

Also note that they would require huge hard disks with several terrabytes.
Parent - - By papacamilla (*) [de] Date 2009-09-25 15:11
Probably 7 men tablebases do exist! -> http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=8477

Why double r in terrabytes? This used to be a joke about computer amateurs in Berlin magazine "TIP" or maybe just part of Latin memory!?

Terabytes of disk space are quite cheap nowadays - so if they existed ... ??
Parent - - By Felix Kling (Gold) [de] Date 2009-09-25 15:16
terrare humanum est ;)

I guess noone here has several 100 TB disk space
Parent - - By Nelson Hernandez (Gold) [us] Date 2009-09-25 20:37
If the full Nalimov 7-man set did exist, and someone acquired it, and bought the necessary HD storage space, and had the patience to load the index tables (many hours, perhaps days), and it was compatible with Rybka... (a lot of ifs here, and I probably haven't covered them all) ...it would be pretty cool to wipe out people with the full 6-man tables, wouldn't it?  What do we suppose the net ELO gain would be, about +10?  I have to think that the progressive ELO gain gets higher the more men the EGTBs can handle. 

But maybe not.  Imagine a person with 32-man tables playing someone with 31-man tables.  The whole strategy for the former would be to avoid a capture that could lead directly to a draw or loss.  That would be quite a trick--there probably is not such a move!
Parent - - By ptobler (**) [au] Date 2009-09-26 02:13
I think the net ELO gain would be a fair bit more than +10 - you would have many times more tablebase calls than you do with the six man - and they would occur a fair bit earlier
Parent - By Gaмßito (****) [cr] Date 2009-09-26 02:26
Yes, that's very simple. More tablebases-man, major elo gain you'll have.

Regards,
Gaмßito.
Parent - - By Sesse (****) [ch] Date 2009-09-27 10:14
I guess it depends on all the common factors: Are they used in the search or not? If so, are they distance-to-mate tablebases or bitbases? And what's the access time? And of course, the dreaded fifty-move-rule, which you'd assume gets more and more important for tablebases as the situations get more complex.

One would imagine that if seven-piece tablebases did indeed become a reality, it would hopefully be in a format where you wouldn't have to load indexes for days first :-)

Honestly I'm a bit surprised nobody has really started defining a format for 7-man bitbases yet and tried pushing it public, but perhaps the gain is simply too small. When did the last 6-man bases become public? 2003?

/* Steinar */
Parent - - By Nelson Hernandez (Gold) [us] Date 2009-09-27 12:19
Funny you should ask.  The last six-man tbs became public after I personally took the initiative, looked up Nalimov's phone number in the telephone directory, and called him at home.  This call, my receipt of the files and their release happened over the course of August 2006.
Parent - By Sesse (****) [ch] Date 2009-09-27 17:52
Wow. Interesting that so much should rest on one man -- I've heard rumors the Nalimov code is indecipherable for anyone but Nalimov himself, but never read it.

/* Steinar */
Parent - By Mark (****) [us] Date 2009-09-27 13:09
The only place I've seen that discussed the various possible formats for 7-man tablebases was at the endgame tablebase forum http://kirill-kryukov.com/chess/discussion-board/
Parent - - By Lukas Cimiotti (Bronze) [de] Date 2009-09-25 18:00
A small selection of 7 man TBs would be cool, for example KRPPKRP would be helpful.
Parent - - By Uly (Gold) [mx] Date 2009-09-25 19:08
But before tables with pawns are generated, all pawnless tables need to be generated first.
Parent - - By Felix Kling (Gold) [de] Date 2009-09-25 19:14
oh yes, the ones with queens may take some time :)
Parent - - By Banned for Life (Gold) Date 2009-09-25 20:23
Yes, and don't forget Bishop under-promotion!
Parent - By Nelson Hernandez (Gold) [us] Date 2009-09-25 20:28
Damn straight.
Parent - By Master Om (Bronze) [in] Date 2009-09-26 04:08
Yes . Suj uses some 7-men tablebases. It took him 6 months to generate it. Why should Rybka be used to use  it if it can be used in GUI. There is option in fritz GUI.
Parent - - By ptobler (**) [au] Date 2009-09-26 01:45 Edited 2009-09-26 02:29
It goes without saying that it is implied with appropriate naming conventions followed for the filenames . Looking forward to memory increasing - and to Yakov Konoval and Marc Bourzutschky finishing their 7-man tablebases (they estimate they will be finished in 2015).
Parent - - By ptobler (**) [au] Date 2009-09-26 02:24
Is it possible that in Rybka 4 we might have programmed in the ability to utilise a selection of 7-man tablebases - the selection that you have included in the appropriate folder?
Parent - - By Geomusic (*****) Date 2009-09-26 07:52 Edited 2009-09-26 07:56
I wonder why we cant generate a 7 man with the info we already have on a similar 6 man position and just build from there adding a piece using an nth degree polynomial equation.
Parent - By ernest (****) [fr] Date 2009-09-26 15:19

> adding a piece using an nth degree polynomial equation.


Math 101? :lol:
Parent - - By titanium cranium (***) [us] Date 2010-07-18 13:10
where can I get a copy of the the 6-man table bases?
Parent - By Uly (Gold) [mx] Date 2010-07-18 23:48
http://tablebases.sesse.net/

Only make one connection at a time or you'd be banned from downloading.
- - By Gordon (**) [gb] Date 2009-09-26 16:15
I find it rather difficult to see any connection between tablebases and polynomials!
Parent - - By Geomusic (*****) Date 2009-09-27 09:14 Edited 2009-09-27 09:57
Hmm you might be right.... maybe  I do need to take Math 101 over again. I must be confused...

I think you need to look up http://en.wikipedia.org/wiki/Generating_polynomial

and

http://en.wikipedia.org/wiki/Rook_polynomial]]

just add the pieces to the the tablebase array using the above function for each added piece and  in this example, a rook. difficult much?
Parent - - By Gordon (**) [gb] Date 2009-09-27 15:28 Edited 2009-09-27 15:32
Neither of these articles has any remote connection with tablebase generation.

Those is simply no connection between the 6-man tablebases and 7-man tablebases and there is no possibility of generating one from the other.

Remember also that the 3-4-5-man bases occupy about 6 GB (I can't remember the exact figure) and the complete 6-man bases are over 1 TB - the 7-man bases going to take a lot of generating and they are not going to be small! The generation process is quite simple and it has been described in many articles - it is just very very slow, even on a very fast machine.

Also remember that if you want the tablebase KPPP v KPP you first need the tablebases in which all the pawns have all possible promotions, for example you require KQQR v KBN etc. You also need all the 6-man tablebases, since a single capture results in a 6-man ending!
Parent - By Geomusic (*****) Date 2009-09-27 23:54 Edited 2009-09-28 00:04
maybe I'm talking of starting from scratch
- - By Eagleclaww (***) [us] Date 2009-09-28 19:43
Out of curiosity, I looked up how much hard drive space the 6 man tablebases take up (plus 3/4/5 man tablebases), and its well over a 1TB of space. I might seriously think about downloading all the non pawn 6 man tablebases.

The first one I'd get is the KQQKQQ, because it was used in the Deep Blue/Kasparov match in the 90's. LOL. Not sure, but that might have been the only 6 man tablebase used in the match. Dunno why, other than the programmers wanted to make sure that if it came up, the machine wouldn't blunder.

Nowadays, chess engines are much more elegent, and don't need to rely on brute force so much. Of course, using more processors/cores would add to its overall strength.
Parent - - By Nelson Hernandez (Gold) [us] Date 2009-09-28 20:47
In your case I would go for the KQQQKP tablebase.  (Only because they don't have 5-1 tablebases out, in which case I would urge you to get KQQQQK.)
Parent - - By Fulcrum2000 (****) [nl] Date 2009-09-28 21:08

> Only because they don't have 5-1 tablebases out, in which case I would urge you to get KQQQQK.


LOL Even Rybka might be able to win that endgame without tablebase assistance :lol:
Parent - - By Sesse (****) [ch] Date 2009-09-28 22:32
Hypothetical question: If you could have as many queens as you'd like, and the other side had only his king (so KQ* vs. K), and if you disregard the 50-move rule and threefold repetitions how many would you need to make it _impossible_ to draw the game? (In other words, you would never, even if you wanted to, put one of the queens unprotected and next to the enemy king.)

/* Steinar */
Parent - By Vempele (Silver) [fi] Date 2009-09-29 06:47 Edited 2009-09-29 07:01

> (In other words, you would never, even if you wanted to, put one of the queens unprotected and next to the enemy king.)


Stalemate is always possible (or you'll start getting impossible positions --- I mean, positions where you can't even reconstruct the last pair of moves).

Or positions where the last move is a promotion-mate. I suppose that would make it impossible to draw. :)
- - By Victor Zakharov (*****) [ru] Date 2009-10-02 15:57
After reading this thread I’ve made a conclusion that most people can’t imagine how difficult is the process of creating the tablebases. So here are my 2 pence to this topic.

It is true that an experienced person can write a 7-men tablebase generator in one-two days. The only problem with such a program is that it either will work for ages (using disk all the time) or will need about 3 TB RAM (and then it will be several times quicker). 3 TB RAM is 1000 times more than a modern PC has. Fortunately, it is possible to implement slicing, i.e. splitting the tables to the parts that do not much depend on each other.

The major tricks in 7m generation are connected with optimizations in speed, RAM and final table size on the disk. There are trivial optimizations like considering symmetries, there are average optimizations like compression, slicing tables in the limited RAM, and there are huge optimizations for significant reducing number of operations and dimensions. These significant optimizations are the key for the solution of the problem in reasonable time.

Usually different optimization targets contradict each other. For example for reducing dimensions it is logical to exclude impossible positions. So the program will need less memory and disk size. It is easy to discover that two kings can’t be in touch with each other. But it is not easy to exclude the positions where on the white move the white queen attacks the black king. Discovering king capturing and proper indexing slows down the program several times and adds years to the final date.

So developers frequently have not-pretty choice between reducing dimensions and raising calculation time. I believe that there were many developers who started their own generation project (the initial idea of how to generate is quite simple). But most of them quickly understood that there is no universal trick to avoid huge complexity of chess endings for modern computers.

We should not forget that sizes invoke major technical problems with debugging (imagine program failure after a month of calculations), hardware stability, verification, backups and so on. The developer should devote significant part of his life for the solution of the Problem. For my estimation, Nalimov worked 15 years for his 5 and then 6-men tables. Not everyone is ready to make such big efforts for such a long period.

And surely a cost of hardware is significant. So the entering fee is expensive.

Some known large-dimension problems were solved using distributed calculations. Unfortunately 7-men generation process can’t use the technique of distributed calculations with many networked computers. Having some 7-men position you may need to know evaluation of any position in the current ending (and in the minor endings). The theoretical number of 7-piece positions with the pawns is about 2,500,000,000,000  ( = 64*63*62*61*60*59*48). So you need 5TB to manage them. Now you can understand that the programmers need think thoroughly about every decision they make in their program.

BTW, Nalimov opened his code for operating with 6-men tablebases. But he didn’t make public the generation program. For my opinion there was a lot of different tricks in his code. And these tricks were not the best part of the code. Possibly he even didn’t have universal program but generated special programs for different classes of endings.  (Sorry, this is just some thoughts, not the facts.)

So after reading of all this you may understand why the progress of 7-men generation is not well visible. Only one brave and successful group is well-known: Yakov Konoval and Mark Bourzutschky. But there is no reliable information what progress is achieved and what is estimated time for finish. The only thing that is clear from announced generation time that their algorithm is better than Nalimov’s one.

And finally let’s imagine that 7-men tablebases has been generated. The lower estimation for their size in compressed form is 100 TB. I believe that one useful ending like KRPPKRP will demand hundreds of gigabytes on disk. Is anybody ready to download such data and to keep this on the disks? Only a few persons in the world may have hardware that will allow them to use 7-men in engine search. So the only way of using the tables is on-line access via web-services.

It is too early to wait 7-men tables for everyday life. It is too early even to speak about standards for TB7-access and to ask about engine support of them.
Parent - By Sesse (****) [ch] Date 2009-10-02 16:14
Thanks for the insights!

/* Steinar */
Parent - By Highendman (****) Date 2009-10-02 16:27
Many thanks - very clear and interesting.
Parent - - By BankShots (***) Date 2010-07-13 19:31 Edited 2010-07-13 19:35
Not to try to be too absurd or something, but it might be possible!  It is all a little "space age", but here it goes:
1.  In this modern era there is massively parallel computing!  The data could be retrieved/"mined" and stored on "web-service".
2.  The data could be massively indexed (and perhaps put into only small (<5MB) files).
3.  People with broadband connections speed, when get close enough into endgames (eg. within 1-2 material of Endgame (probably preferably Bit) Bases), could then query for evaluations or send for the solutions files themselves in the background and store only temporarily on users drive.  In games with long enough time controls this might work!
:grin:
Parallel Computing links interest:
note:  quasi-computing/large-scale distributed computing/"embarassingly" parellel computing:  http://en.wikipedia.org/wiki/Supercomputer
http://en.wikipedia.org/wiki/Folding@home

Theoretically at least, these could solve ALL your logistical problems (wrt Hardware and Time)!! :grin:
Parent - - By Victor Zakharov (*****) [ru] Date 2010-07-16 13:40
1) The idea of chess data interchange between computers is quite healthy. It is under consideration. But a lot of thinking is essential before the start of the implementation.
2) I am afraid that this will not work for endgames. Dimensions essential for endgame analysis are very high.
Parent - By BankShots (***) Date 2010-07-16 14:35

> 2) I am afraid that this will not work for endgames. Dimensions essential for endgame analysis are very high.


Supposedly is known what likelihood of winning is for that material and could be reported to engine from far horizon for positional evaluation from knowning known endgame on Quasi-"Super"Computer and information could be then transferred with very low dimensions.  Perhaps. :roll:  (Obviously a big project.)

Perhaps option in Aquarium (and could perhaps try to team-up with Chessbase as well) to allow background work on Endgame Tablebases.  I have "655 GB free of 931 GB" on my own Harddisk.  I may install some games later, but you could use eg. upto 200GB (that's 1/5th of Terabyte right there! :wink:) (if I wasn't installing games, I might have let you use 500GB!! 0.5TB :surprised:) of mine I suppose toward your hardware requirements on solving Endgame Tablebases.  JUST when using engine, we get cpu @ 100% for analysis when playing/analyzing for our own positions.  Background = for @ night or just leaving computer running.  With enough participants, I suppose high-end tablebases could be done very quickly.  Maybe participants get premium membership @ playchess.com or something?! :smile:
Up Topic Rybka Support & Discussion / Rybka Discussion / 7-man tablebases

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill