Not logged inRybka Chess Community Forum
- - By Jessica (**) Date 2007-12-26 20:57
if i have over 4gb of ram and windows xp x64.  Playing a 10 min game how much hash should i give rybka for max performance i was giving before between 1gb - 2gb.
Parent - - By Vempele (Silver) Date 2007-12-26 21:39
If you care about performance, get Linux and Microwine and configure Rybka to use huge pages (I think one of Sesse's posts has the instructions). You'll also be able to use more hash than in Windows (she won't crash if you give her 4 GB).

Or 2 GB if you're going to stick with windows.
Parent - - By ChessStar (*) Date 2007-12-26 23:40
I too have 4gb on windows xp x64 so how much is safe to use in 10 min game anyone know?
Parent - By Uly (Gold) Date 2007-12-27 00:45

> Or 2 GB if you're going to stick with windows


That remains true.

Anyway, you could give them 128 MB and you would be fine, the rest of the RAM basically doesn't do anything on a game so fast as 10 + 0...
Parent - - By Gaмßito (****) Date 2007-12-27 03:02
If you are talking about 10+0 (without no increment time), then no, it is too much ram the one that you are using. A little more than 80% of that ram won't be filled never under that time.

In my opinion, 256mb as maximum is quite good for you at this time control.

Regards,
Gaмßito.
Parent - - By Permanent Brain (*****) Date 2007-12-27 03:38
About fill rates: Crafty uses 12 bytes per hash position. Assuming that this is a typical value:

Example Node rate (just using a reasonable, typical number; not engine specific):

300 kN/s = 300.000 nps

300.000/s * 12 bytes = 3.6 million bytes per second (!) That is ~3.4 MB/s

(of course, the node rate depends on the computer speed too, and the particular engine)

Keep in mind that we do not know Rybka's exact internal node rate, and if all (or how many %) go into the hash tables. Nevertheles, the example calculation indicates that on a "typical comp with typical engine", a 256 MB hash table can be full after a few minutes of calculation already, and entries will be replaced/overwritten.

At the same time, I suspect that - unlike in (simple?) theory - there ARE access speed issues with very large hash tables. Unfortunately I didn't get an answer recently in CCC about that and I only roughly remember what I read about that, before. But I can imagine that memory access on the OS and hardware levels is "complicated" in a PC, for example if I consider that a large h.t. may be distributed over several RAM chips. But I am no programmer and have only rough ideas of all that stuff.
Parent - - By ChessStar (*) Date 2007-12-27 04:25
so what happens if you give too much hash?
Parent - By Uly (Gold) Date 2007-12-27 19:34

> so what happens if you give too much hash?


With too much hash, the operating system and other programs may ask for more Ram, so to give it to them, the engine's hash is sent to the hard disk. This is called memory swapping and hurts the engine's performance tremendously.

But as long as you keep enough RAM free to the operating system and you don't run anything else, you'll be fine, and you should give as much RAM as possible to the hash.
Parent - - By BB (****) Date 2007-12-27 06:23
the example calculation indicates that on a "typical comp with typical engine", a 256 MB hash table can be full after a few minutes of calculation already, and entries will be replaced/overwritten.

This is about right. I use 16 bytes per hash position (this is the size of a cache line or something I think, and thus might be more typical), but the node-count might only count non-quiescence nodes (as many programmes do not hash these). Of course, replacing/overwriting depends on the implementation - for instance, Fruit uses clusters (of size 4 I think) to soften the nuisance of over-writing. In most cases, the birthday paradox will imply that some overwriting occurs quite early, but it will not be major concern until the hash table is (say) at least half full. In 5m+5s games with 256MB hash for my programme on a 3.4Ghz P4, almost always 99%+ of the hash is full at the game's end, while this would be typically be about 80% at 2m+2s. Whether or not having a full hash is really a problem (it need not be a big deal if the only things replaced are stale/unimportant entries) is a different question.
Parent - - By Permanent Brain (*****) Date 2007-12-27 09:26
Thanks... so, would you say that for general calculations, the typical figure to use (if unknown) is 16 bytes per position?

About the filling/overwriting: I can only guess, but aren't many old entries useless anyway... for example, from calculated positions where both had queens still, while in the game the queens are gone. Same for pawns or actually anything which has been captured while it was still there in the hashed positions. No chance that those positions will appear again in the game. Therefore I think, the whole idea of having a hash table so large that nothing is overwritten in the entire game, is wrong anyway.

(Although it should probably be large enough to keep at least all hashed positions from 3 or 5 consecutive moves...?)
Parent - By BB (****) Date 2007-12-27 10:07
Thanks... so, would you say that for general calculations, the typical figure to use (if unknown) is 16 bytes per position?

I don't know. I've heard of people having multiple hashes, some of which (especially pawn structure/weaknesses) seem to be quite bulky. The versions of Crafty I have (20.9/21.6) use 128 bits for hash entries (see hash.c), as does Fruit (see entry_t in trans.cpp). As I mentioned, there might be a cache reason for the popularity of this number - Fruit uses clusters of size 4 (which then gives the 64 bytes that Vempele indicates  - Fruit wraps the clusters, which could conceivably cause misalignment, though I think most memory subsystems do not have a problem with this). I'd guess the 128-bit datapath is more important than the 64-byte cache line, but I've never given it much thought.

About the filling/overwriting: I can only guess, but aren't many old entries useless anyway.

Yes, this is certainly true. Usually an "age" is given to each entry - if there is a choice in replacement (as in Fruit), then the oldest is discarded first, and if everything has equal age, then the shallowest is ditched.

(Although it should probably be large enough to keep at least all hashed positions from 3 or 5 consecutive moves...?)

One of the UCI output fields is "hashfull", which will tell you how much of the hash has been used for a given search (at least, this is what Fruit reports).  On my computer, with 256MB, this number goes up at about 0.6% per second, which gives about 2-3 minutes to fill completely. However, there is not complete uniformity in hash fills, and the problem is more when two useful bits both want to go to the same hash spot. I could probably derive something more rigourous, but your guess of 3-5 moves is likely close to the expected usefulness of a hash entry. If true, then 256MB would be reasonable for 30sec/move in my case, but for quads you'd have to scale appropriately.

It is unclear to me what the problem could be with too large of a hash size, but I'd agree that this would more likely be  a problem with the operating system or memory subsystem than anything theoretical.
Parent - By Vempele (Silver) Date 2007-12-27 09:36

> I use 16 bytes per hash position (this is the size of a cache line or something I think, and thus might be more typical)


A quarter of it. 9 bytes is another possibility (with one byte of padding every 7 entries).
Parent - - By Vasik Rajlich (Silver) Date 2007-12-27 13:07
There is no such thing as too much hash, as long as the speed (ie. nodes-per-second) doesn't drop.

Vas
Parent - - By BB (****) Date 2007-12-28 05:48
There is no such thing as too much hash

There is an obscure effect that can occur  - say we have two positions A and B for which we do (consecutive) hash lookups. With a small amount of hash, A and B  are (very slightly) more likely to live in the same cache line of memory, and thus we don't get a cache miss with the smaller hash size. I can't imagine this has any measurable effect in practise.
Parent - By Vasik Rajlich (Silver) Date 2007-12-30 11:47
Sure. I think it's implied that we're talking about the sum of the effects :)

If you disable the transposition table completely, in some positions Rybka will end up playing a better move.

Vas

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill