Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / The ultimate RECAP of chess formats, databases, and database
- - By budfit Date 2012-05-13 09:35
I am writing this post as a hybrid between a systems programmer / Modula-2 /, the creator of chess databases / Opening Master /, chess player / ICCF SIM / and user of all programs currently available on the market. Let’s start from "ground zero", because many people still confuse these. Let’s see the different concepts and different perspectives on the same topic as an interesting new development for the future.

Let’s define the basic concepts:

a) chess game
b) chess database / DB /. e.g. Chessbase, Hugebase, Opening Master.
c) chess database program. eg. ChessBase, ChessAssistant, SCID.
d) chess program - engine. Core program without environment. Eg. Houdini, Rybka, Critter
e ) chess environment / Graphics user interface-GUI / in which the chess program starts to play, or analysis e.g. Fritz, Aquarium, Chess King

a) Chess game consists of information, who, when, where and other parameters to identify the game. Then the body…i.e. individual moves. Games began to spread in PGN format, then CBF, CDP, CBH. It is worth considering why and so what? The problem is that the first format is inefficient, but Open Source! Others are more effective but commercial (i.e. not public). All formats were invented some time ago and had nothing to do with system programming. The languages ​​in which it had been programmed didn’t have instructions to the level of pure bit but rather byte. And this is the difference in saving space when storing games. I will demonstrate it on one example. The result can be regarded as one of four states: win, loss, draw, and unfinished game. The 4 states exhausted just setting two bits in one byte. The 6 remaining bits are free to store other information. If we thus proceed further, we come to about a 50% savings compared to today's most effective format of CDP from Chess Assistant. The difference is then seen on the whole DB. Any conversion operations are then close to 0 seconds of machine time.

b / Chess database (chess DB) is actually a permanent “data mining” on the Internet and the result is a DB in specific size and quality, or specialized DB. Important is frequent update and user access to the DB. Returning to the chess game, it is therefore not the actual size of a DB that matters. It would seem in today capacity Hard Disks / HDD / that it is! Well, It is not! The future is in the Cloud is where all data is being transferred to various possible devices and this has nothing to do with the HDD, but only the size of the file which being transferred and needed. The smaller the better. With increasing transmission speed, which will theoretically go with LTE technology to 100 Mb/s, it would seem that even so the DB will move swiftly. The opposite is true! Today's DB are significantly larger. Opening Master has 8 million games in the DB and its transmission is virtually impossible on-line in any format. Therefore, the overall solution must be client / server and selection from the DB must be on the server. Well, what if the choice must pass 20 thousand games to your device. Then again, you remember the bits. The chess databases were mostly considered as COMPUTER programs. Therefore, the user was relatively independent of chess except for the updates. Furthermore, a variety of specifications of computers influenced the work with any DB. The weaker configuration of your system, the slower is the process. But what is not online is not user friendly anymore. In order to be somehow online the only solution is to have super hardware with lots of CPUs, multithreading, large RAM, SSD drives and super fast network, etc. .. Working with data in client / server mode is a better solution. Nobody has come up with solution yet.

c) The DB chess program – first you need to be aware of when the programs were written, which new solutions were brought by new versions. Well, mostly it was later to finish GUI graphics, a few extra features and it is everything. The hardware surpassed software by thousands miles while DB programs used nowadays do not use multiprocessing, parallel programming, nothing from 21st Century. Chess Bases and Chess Assistant do not want to radically upgrade their programs, they would have to write them again and this hurts. Whoever comes up with new software technologies and solutions All in One, Cloud, Client / server solution multiplatforms OS in the same program will be the new king of chess market. When we think of chess database it can be mentioned the DBs for non-chess programs programs. Why developing a new software if it is ready on the market and working. Just add a new format. For your own database software it is perhaps only avoiding unnecessary overhead, which is not negligible. The question perhaps is how things will be processed at once/parallel? It will be maybe effective more. It is possible that the new global chess DB will not be able to run without standard DB program. Chess data are common to other data processed by any given relational database. It only matters choosing the right proportion between each other. Do not use tank for killing mosquito (Oracle size) but on the other side millions of games in one database can not be processed online through MS Access either. Today's chess DB programs are thus in state 15 years ago and upgraded similarly to Vista experience. In addition ChessAssintant has nowadays enough bugs and DOS error like messages which nobody can understand. The database world is generally quite developed with hundreds of DB programs. It is the only a choice. You don’t need to reinvent the wheel.

d / The world of chess engines, in recent years improved a lot compared to DB programs. The UCI / Universal Chess Interface / solution is ideal because you can simply write the core program and run it in a standard environment. It relieved the programmer from producing non-standard GUI and thus can focus on the CORE of the problem and the assessment / evaluation function / feature positions multi-variability and speed of calculation. A big improvement area is still the endgame, where also quality engines are lost. In addition you need to get to Tablebase, which is already in the 3-4-5-6 men endgames with deterministic result without loss of any material. The multithreading was initially thought to bring a big increase in speed, but it's not. Multiple CPU does not equal multiple of increase of speed. Again, only a clear solution is using parallel programming techniques. Do not rely solely on that program written in C + + which is itself fast. It is not enough. Most of this section is described also on the website:

There is one more interesting way to turbo boost development of engines. The path defined by Hydra is a long journey, a combination hardware / software in one array of processors. But it is a topic in itself. We envision that in 10 years we begin to explore these solutions, because multiprocessor hardware solutions will grow vigorously.

e / chess environment (or Graphical User Interface = GUI) is also a theme similar to the DB program. The most famous is the GUI of the Fritz from ChessBase. Its main function is to use game engines and organize human - engine or engine-engine, engine tournaments. What is interesting is the automatic annotation of completed games. Environment options are not so difficult, so that ordinary users to handle them well. The DB programs are a much more tricky especially those from ChessAssistant. It is suitable only for highly advanced users. I can imagine the All in One solution with an appropriate number of features of the environment and DB Fritz and ChessBase 11th. Again, there are no new software or hardware solutions in GUIs! It is worth mentioning only interactive analysis of CA Aquarium.


From the above we can conclude that new hardware will influence and affect the entire chess area in very near future (it already does), but the software so far lags very much behind its possibilities. The time will come when new chess company comes which offers this solutions and will send all others to old school. They will no longer have any added value nor will stay competitive. Each however, determines its fate itself.

Alexander Horvath, SIM ICCF
creator of Openingmaster(dot).com
May 12, 2012
Parent - By Nelson Hernandez (Gold) Date 2012-05-13 12:22
Your conclusion sounds almost defiant.  Are you throwing down a gauntlet?
Parent - - By ppipper (*****) Date 2012-05-15 08:02
I currently have MySql storing several million games online for private purposes.

Very interesting post
Parent - - By budfit Date 2012-05-15 09:38
thanks ppipper. How does your SQL works, would you imagine it working on cloud?
Parent - By ppipper (*****) Date 2012-05-15 10:09
it works pretty well. It is currently working on a physical machine which I can access via internet, it would not be too much deal to open this MySQL server to cloud users.

The workflow to maintain my database is a bit tricky, but it works!
Up Topic The Rybka Lounge / Computer Chess / The ultimate RECAP of chess formats, databases, and database

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill