Last week someone pointed out that 2.3.2mp is freely available. I downloaded the executable. I've been sort of perusing the disassembled asm source to develop an overall understanding. First thing I located, immediately, and posted last friday, was the bishop shift arrays. Here's a better picture of them from objdump which gives the memory addresses as hex, but it also shows the values as little-endian hex so it takes a bit of looking to follow the numbers.
This array from Crafty 19.x is one of the two bishop shift arrays:
const unsigned char bishop_shift_rr45[64] = {
0, 0, 4, 7, 11, 16, 22, 29,
0, 4, 7, 11, 16, 22, 29, 37,
4, 7, 11, 16, 22, 29, 37, 44,
7, 11, 16, 22, 29, 37, 44, 50,
11, 16, 22, 29, 37, 44, 50, 55,
16, 22, 29, 37, 44, 50, 55, 59,
22, 29, 37, 44, 50, 55, 59, 0,
29, 37, 44, 50, 55, 59, 0, 0
};
In Rybka 1.0 beta and 2.3.2 they are stored as ints. Here's the actual memory addresses and contents:
581ed0 00000000 00000000 04000000 07000000 ................
581ee0 0b000000 10000000 16000000 1d000000 ................
581ef0 00000000 04000000 07000000 0b000000 ................
581f00 10000000 16000000 1d000000 25000000 ............%...
581f10 04000000 07000000 0b000000 10000000 ................
581f20 16000000 1d000000 25000000 2c000000 ........%...,...
581f30 07000000 0b000000 10000000 16000000 ................
581f40 1d000000 25000000 2c000000 32000000 ....%...,...2...
581f50 0b000000 10000000 16000000 1d000000 ................
581f60 25000000 2c000000 32000000 37000000 %...,...2...7...
581f70 10000000 16000000 1d000000 25000000 ............%...
581f80 2c000000 32000000 37000000 3b000000 ,...2...7...;...
581f90 16000000 1d000000 25000000 2c000000 ........%...,...
581fa0 32000000 37000000 3b000000 00000000 2...7...;.......
581fb0 1d000000 25000000 2c000000 32000000 ....%...,...2...
581fc0 37000000 3b000000 00000000 00000000 7...;...........
If you look at the first byte of each 4 byte doubleword, and convert from hex to base 10, they match exactly.
Here's the second bishop array from Crafty:
const unsigned char bishop_shift_rl45[64] = {
29, 22, 16, 11, 7, 4, 0, 0,
37, 29, 22, 16, 11, 7, 4, 0,
44, 37, 29, 22, 16, 11, 7, 4,
50, 44, 37, 29, 22, 16, 11, 7,
55, 50, 44, 37, 29, 22, 16, 11,
59, 55, 50, 44, 37, 29, 22, 16,
0, 59, 55, 50, 44, 37, 29, 22,
0, 0, 59, 55, 50, 44, 37, 29
};
And here's the corresponding rybka memory addresses and data:
581fd0 1d000000 16000000 10000000 0b000000 ................
581fe0 07000000 04000000 00000000 00000000 ................
581ff0 25000000 1d000000 16000000 10000000 %...............
582000 0b000000 07000000 04000000 00000000 ................
582010 2c000000 25000000 1d000000 16000000 ,...%...........
582020 10000000 0b000000 07000000 04000000 ................
582030 32000000 2c000000 25000000 1d000000 2...,...%.......
582040 16000000 10000000 0b000000 07000000 ................
582050 37000000 32000000 2c000000 25000000 7...2...,...%...
582060 1d000000 16000000 10000000 0b000000 ................
582070 3b000000 37000000 32000000 2c000000 ;...7...2...,...
582080 25000000 1d000000 16000000 10000000 %...............
582090 00000000 3b000000 37000000 32000000 ....;...7...2...
5820a0 2c000000 25000000 1d000000 16000000 ,...%...........
5820b0 00000000 00000000 3b000000 37000000 ........;...7...
5820c0 32000000 2c000000 25000000 1d000000 2...,...%.......
Same drill, repeated. First byte converted from hex to base-10 and you get perfect matches.
I am slowly mapping arrays that exist in Crafty to those that exist in Rybka 2.3.2. Here's 4 more. I am just going to give hex memory addresses and the corresponding names in the Crafty 19.x source code:
0x181ed0 (+0x400000) = int bishop_shift_rr45[64]
0x181fd0 (+0x400000) = int bishop_shift_rl45[64]
0x1822d0 (+0x400000) = uint64_t bishop_attacks_rr45[64][64]
0x18a2d0 (+0x400000) = uint64_t bishop_attacks_rl45[64][64]
0x1922d0 (+0x400000) = uint64_t rook_attacks_r0[64][64]
0x19a2d0 (+0x400000) = uint64_t rook_attacks_r90[64][64]
To get the actual memory addresses for the 6 variables on the right (crafty names) you have to add "0x400000" to the hex values on the left. Compilers like to do this stuff so that every memory address is not 64 bits long making instructions very long. If you notice, the first one (after adding the two values) is 0x581ed0, which is the same as the first addresses in this post. The first value is the common hex value used in instructions, something like "mov 0x181ed0(%rax, %rdx, 8), %r12" or something similar (depending on whether it is an int or uint64_t type array. Where %rax will have that 0x400000 beginning of data segment address so that the offset is smaller and doesn't need 64 bits. If you want to see all of the locations where bishop moves are generated, you can search for either of the bishop shift offsets (0x181ed0 or 0x181fd0) in the objdump -D disassembly of the binary and you will find every place that is done. For bishops, a total of 250 different places in the code. Probably a result of C++ template expansion or whatever, as that seems excessive.
In any case, I will stop here for the moment. My approach was to find something identifiable, and then use that to backtrack. I now have all the bishop attack generation points. I am going to wade through each one looking for the stuff done in functions like AttacksTo() and Swap() to see if they are identifiable as such and if they match Crafty.
I have so far found exactly ONE difference in the bitboard implementations. For bishops, Rybka 2.3.2 and Crafty match perfectly. For Rooks, Crafty computes the shift amount since for the normal board, if you just scrub off the rightmost 3 bits of the square, and add 1, you get the number of bits to shift the ranks to the right to get the correct rank (minus two end bits, remember) in the rightmost 6 bits. For the rotated 90 bitboard, you shift the square left 3 bits, and with 0111000b, and add 1, to get the shift amount. Vas decided to use rook shift arrays just as I did for the bishops. I had tried that but found the computation was cheaper than the memory/cache lookup to fetch a 1 byte (Vas uses 4 for rooks and bishops) shift amount. So one difference, everything else matches exactly. And note that the shift amounts match precisely anyway, but in Crafty the rook shifts are computed as (square & 0x38) + 1 (for non-rotated occupied bit board) while Rybka does a table lookup to get exactly the same number.
I have had to put this down for a bit to get a lot of grading done, so I thought I would give an accurate update so far, since this IS the version that competed in an ICGA event. Looking here gets rid of all the clutter and noise about "those older versions didn't compete, so who cares?"
As I map addresses to crafty variable names, I am editing my asm source for Rybka to replace the addresses by names to make the code a little easier to read. If someone wants to look, I can make it available on my web page easily enough, although it is BIG. 75mb big for the objdump output. I can even tell you where his rook shift arrays are if someone is interested. :)
> Last week someone pointed out that 2.3.2mp is freely available. I downloaded the source.
You downloaded the source, or the exe ?

:)

As for me, I think that the entire system should be overhauled and replaced with wider-scale Medicare, which has been shown in many studies to be far cheaper for everyone involved and retains nearly the same profits for the hospitals and doctors (it's amazing what happens when you get rid of the middlemen!).
> Remember that the Supreme Court ruled that the act itself was unconstitutional, and could only be made constitutional if it was cast in the form of a tax
Whoa! That is a stretch!
The Supreme Court ruled the Affordable Care Act is constitutional.
It was Robert's who made the deciding vote.
The Republicans based their premise that the act was unconstitutional in that Congresses was posing a tax on Americans. The justices ruled that the individual mandate is a tax that Congress can impose on Americans.
I think we need to give the Affordable Care Act time to see if it will work. It is about time this country had competitive health care. It hasn't even take hold yet and you have stat's???? Come now!
> The Supreme Court ruled the Affordable Care Act is constitutional.
You seem to have trouble with your memory. The Supreme Court ruled that the act, as it was passed in Congress and enacted by Obama, was unconstitutional. However, it was ruled that if the penalty for not abiding by the individual mandate could be considered to be a tax, then it is constitutional.
Don't you remember the day? I remember it very clearly. When the decision was read to the reporters, it was stated that the individual mandate was unconstitutional due to penalizing Americans. The stock market immediately flew up 100 points. Then, they continued reading, and said that, however, the "penalty" could be seen as a "tax", and in that sense, it could be made to be constitutional. The stock market then immediately dove around 150 points or so. I remember this well because I was trading options for VIX, UPRO, and SDS during those exact moments. In commentary afterward, Justice Roberts said that while the act itself was unconstitutional, a precedent set in 1890 implied that if the Supreme Court justices could find a way to actually make it constitutional, then they should. Obama had said all along that "this is not a tax". The Supreme Court ruled that it could be made constitutional only if it could be thought of as a tax, and used in the tax code. The original act passed by Congress had no such provision.
On June 28, 2012, the United States Supreme Court upheld the constitutionality of most of the ACA in the case National Federation of Independent Business v. Sebelius. However, the Court held that states cannot be forced to participate in the ACA's Medicaid expansion under penalty of losing their current Medicaid funding.
...
That is the long and the short of it.

You guys slay me!
> Wait, with the current approach, there IS no tomorrow for most.
Well, you just put your bitboard argument into perspective! And whatever else it is you're arguing about re: rybka 2.3.2!

Of course, no such thing will happen. What will happen is that the economy will eventually (and maybe very soon) go tits-up and then we'll have a major fork in the road: tyrannical dictatorship or liberty. Unfortunately, the establishment favors the former and practically nobody favors the latter because they don't understand what it means, other than "no more free stuff", and almost everyone who is getting free stuff is against liberty.

$9B Off-shore financing loophole for banks – Sec. 322
Tax credits for foreign subsidiaries – Sec. 323 $1.5 billion
Bonus Depreciation, R&D Tax Credit
Subsidies for Goldman Sachs Headquarters – Sec. 328
Railroads - Sec. 306
Disney (even Disney!) - Sec. 317
NASCAR - Sec 312- $43 million over two years
No end to this list! $205B of tax breaks for corporations.
> You do realise that the biggest winners of the Health Care Act are the big pharmaceutical and health care companies?
And the abortion industry.
Second point is that the bishop shift arrays (we already discussed this) contain forced values, for the natural and obvious first time one does bitboards by diagonal stacking, you and I already agreed there are two possible arrays only.
So, depending on the state of development of anyone's bitboard code, the likelihood that there is a Crafty version that matches is close to 100%. I would imagine there's a Crafty version to match just about all relatively early developments of every single chess program that uses bitboards.
If Vas wrote Rybka using 0x88, his move generator (which is what bitboards do in effect in this context) would be "semantically" identical to at least one and probably more than one of Crafty's 0x88 move generators. The move offsets are identical by force, and the algorithm is forced too. Bitboards are just a fancy move generator, more complex to think about, but that complexity doesn't alter the basic issue - chess pieces move according to strict rules, offsets for moving on diagonals are fixed by the 64 board squares, offsets for bitboard shift patterns are likewise fixed.
These "comparisons" of yours need to come with health warnings else people may get misled.
That is a 4-5-6 month window, roughly. Nobody has mentioned any OTHER versions of Crafty that might have been copied.
Second, there are variations in the bishop shift arrays. if you rotate right/left 45 degrees, there are two sets of 64 values. But then you get to choose between 64x256 (which was in crafty versions prior to 19.7beta) or the 64x64 approach which was in 19.7 beta, and 19.9 and beyond, and which was NOT mentioned in the rotated bitboard paper I published (it referenced the 64x256 approach because it was written well before 19.7beta and beyond). Then there was the stupid corner square idea I tried, which basically said "don't bother shifting the 4 short diagonals, (ones with 1 or two squares only) because the occupancy contents are useless anyway." So there are some choices. Not a lot, but choices. Bottom line, only the mid-19.x versions are candidates for copying in Rybka. They came out RIGHT at the right time. Just in time to create the 1.4-1.6.1 versions in early 2004 (19.7beta had source files dated either Jan 04 or Dec 03, I don't remember and don't have them handy). Exactly in time for the march/april 1.4-1.6.1 versions.
Remember, I am not done. I am slowly working my way toward other recognizable bit board functions in Crafty that would likely be present in Rybka 2.3.2, IF there was any significant copying. Just hold your water for a while, C++ code is a royal pain to RE because code gets replicated so many times by the compiler. For example, bishop attacks are generated 250 times in the 2.3.2 binary, but only 9 times in Crafty-19.7beta plus twice for mobility (I hope no one else suggests that I convert to c++, btw).
I'm not trying to mislead at all. I have already pointed out one minor difference in that I used chars for the shift arrays, while Vas used ints for unknown reasons, but STILL accesses them as chars btw, using movsx type instructions. The other difference in the first post in this thread is that he uses a rook_shift array while I use a very simple computation to produce the exact same shift numbers.
I'm still looking. Takes time to unwind all this procedure call crap.
My intent is to post things that are beyond argument, however. Things ANYONE can look at and reach a rational conclusion without all this hyperbole and vitriol.
Why should essentially the same process, bitboards (which are just move generators by lookup, basically) be any different?
Bet you also that Rybka bitboard move generator code with bitboards is semantically identical to a Crafty bitboard move generator version. But I am rather inclined to doubt you are going to try and persuade any programmers that code was copied .... you'ld be laughed off the programmers forum. Same situation bitboard code and constants.
As to move generation, you are aware there are LOTS of ways to vary that? What do you generate first? What do you do to order them? I may get around to looking at that, maybe not. I am not going to disassemble the entire 75megs of asm code.
But for heaven's sake, stop the "pre-whining".
The first solid example that is incontestable is the fruit hash code match. Then the rather minor bitboard matching code. My looking has not stopped, yet...
The stuff about Rybka 1 beta and the other betas may point to character flaws in Vas, but really have nothing to do with the ICGA contest, so far as I can see.
The Rybka 2.3.2 program, on the other hand, is very salient to the contest and I think it is probable that there were very few significant differences between the version that was punished and 2.3.2.
Can you find any post where you actually released the Crafty beta in question?
I am wondering why I don't have it and I cannot find any sign of it in old postings or using the wayback machine.
Bob is right about something:
Publishing his article makes the idea free to use (which is not really different than publishing code, because the idea is still free).
But the code implementation still belongs to Bob.
I do not know how much use of Bob's code would constitute fair use, if a citation were included, but probably not very much because commercial offerings clearly get less leeway than free ones or academic exercises.
Note that I am STILL not particularly worried/bothered by the rotated bitboard code. It appears that it was copied like the rest of the stuff to make the early Rybka's, but it really is not that important for reasons already given. I only investigated because a few here keep claiming the code is original. While it still might be, I think the probability is low. But there are some other things that one might be tempted to copy. Swap() is one such piece of code. It is an original piece of code, written from scratch, and rewritten a couple of times as the bitboard code changed (ie when magics came along, they produce the entire sliding piece attack bitmap, where rotated does each diagonal, or rank or file separately. So it makes a difference in how you might handle x-ray attacks in Swap(). I am going to look for that code, but the c++ crap is mind-numbing. If you look at procedure headers, where the machine saves %rsp in %rbp, then mechanically adjusts $rsp to reserve enough memory for the local variables, there are roughly 2,000 of those. No chess program has 2000 procedures. Not even 200. Unless you use C++, templates, and function overloading. So tracking through those is unpleasant and will take some time. Which will happen after I finish grading my architecture exams, and software development exams, given at the end of last week and this morning.
Since you are here, do you have any explanation for the oddball movsz accesses to the int arrays for bishop/rook shifts??? I've not spent a lot of time on this, but I did at least ask myself "Why would I EVER take a direct index into an array of ints, adjust it by a factor of 4 (rather than using direct scaling in the memory reference) and then do a movsxbl (linux type opcode) to access the value? The arrays of shifts are definitely 256 bytes long, 4 bytes per value. Can't imagine it being a struct with the shift amount being in the first byte and the other 3 bytes being padding.
It's done everywhere in 2.3.2a (and I guess in 1.0 beta, I am not looking at that right now). Seems odd, to say the least, perhaps some sort of compiler quirk?
If we ever meet face-to-face, you'd better have a bullet-proof vest on.
Your post caused me to go look at the old ip/robo source again, just to refresh my memory. And as always, it makes my head explode. Who dimensions an array using "0100" (which is hex 40 which is 64 base 10? I had forgotten just how ugly that code was. "thanks" for reminding me.
BTW the swap() looks pretty familiar once one gets past the horrible way that the decompiler unwound the if/then/elses... I'm going to try to look at it more closely.
Another question... have you noticed much use of the "direction" array? Always been in Crafty (and even Cray Blitz). Idea is "direction[sq1][sq2]" gives an "increment to repeatedly apply to get from sq1 to sq2, a zero if there is no increment (only considers sliders). I used it in my Swap() back in the 19.x era, but when magic came along I decided to use another approach for xray attacks to pick up pieces supporting pieces in a a "battery". on the same rank/file/diagonal. I didn't find one in Fruit, which also seems to use a recursive approach that I tried back in 1995 but decided it was not as efficient as what I was using in Cray Blitz and went back to that approach, which is similar to what I have done ever since.
I will try to look some more after recovering from the ip/robo source...
edit: in re-reading the above, it is somewhat disjointed. I'll simply blame that on reading the robolito/ippolit code...
Richard: A warning.
If we ever meet face-to-face, you'd better have a bullet-proof vest on.
Your post caused me to go look at the old ip/robo source again, just to refresh my memory. And as always, it makes my head explode. Who dimensions an array using "0100" (which is hex 40 which is 64 base 10? I had forgotten just how ugly that code was. "thanks" for reminding me.
BTW the swap() looks pretty familiar once one gets past the horrible way that the decompiler unwound the if/then/elses... I'm going to try to look at it more closely.
Another question... have you noticed much use of the "direction" array? Always been in Crafty (and even Cray Blitz). Idea is "direction[sq1][sq2]" gives an "increment to repeatedly apply to get from sq1 to sq2, a zero if there is no increment (only considers sliders). I used it in my Swap() back in the 19.x era, but when magic came along I decided to use another approach for xray attacks to pick up pieces supporting pieces in a a "battery". on the same rank/file/diagonal. I didn't find one in Fruit, which also seems to use a recursive approach that I tried back in 1995 but decided it was not as efficient as what I was using in Cray Blitz and went back to that approach, which is similar to what I have done ever since.
I will try to look some more after recovering from the ip/robo source...
edit: in re-reading the above, it is somewhat disjointed. I'll simply blame that on reading the robolito/ippolit code...
que?
Have you LOOKED at the robolito code? I'd be happy to email the 17K lines of crap to you...
> Another question... have you noticed much use of the "direction" array? Always been in Crafty (and even Cray Blitz). Idea is "direction[sq1][sq2]" gives an "increment to repeatedly apply to get from sq1 to sq2, a zero if there is no increment (only considers sliders).
In Rybka this array has a slightly different meaning. It does not contain integer increments, rather just an enumerated direction type: 4 = unreachable, 3 = vertical, 2 = horizontal, 1 = diagonal, 0 = antidiagonal.
Edit:
This direction value is then used in swap() as an index for two local arrays (int shift_amounts[4], U64 occupancy_rotations[4]).
>So just like ippolit...
Meaning "a" sign Ippolit = Rybka ?
Richard, when will we see Critter 1.8 ?

What's your plan?
http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=469321
then when gentleman Richard moves in look how Houdart (grabbing the first 3000 elo from Vas) as a predator attacks Richard.
http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=469641#pid469641
Bottom line, there might me no plan at all. It's one those occasions I hope I am wrong.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill