1
2
As Vas has made it clear (painfully,to me) that,everything else being equal,32-bit processing can't compete with 64-bit.
In fact,he told me this personally when I 1st bought Rybka last Spring.
Many members of this site have successfully switched to 64 -bits but through no lack of effort I've been unable to do so.
When I joined Rybka,I had an HP AMD Athlon 64x2 4400+ dual-core-2.2GHz XP Professional desktop so I bought the
WinXP Pro x64 CD but failed to install it because,as HP explained to me, the motherboard wouldn't support it.
3 months ago I bought the Gateway FX530XT desktop quad-core QX6700 with WinXP Professional and,again,the
motherboard wouldn't support 64-bit.
Has anyone here obtained off-the-shelf 64-bit processing or have all the successes been achieved by way of building
your own computers? I'm referring now to native 64-bit as opposed to effecting it by way of the WinXP x64 CD or
through Vista.
Even though my Gateway quad-core came with a free upgrade to Vista,and at least one of the Vista Premium packages
supports 64-bit,the motherboard won't permit it.
In fact,he told me this personally when I 1st bought Rybka last Spring.
Many members of this site have successfully switched to 64 -bits but through no lack of effort I've been unable to do so.
When I joined Rybka,I had an HP AMD Athlon 64x2 4400+ dual-core-2.2GHz XP Professional desktop so I bought the
WinXP Pro x64 CD but failed to install it because,as HP explained to me, the motherboard wouldn't support it.
3 months ago I bought the Gateway FX530XT desktop quad-core QX6700 with WinXP Professional and,again,the
motherboard wouldn't support 64-bit.
Has anyone here obtained off-the-shelf 64-bit processing or have all the successes been achieved by way of building
your own computers? I'm referring now to native 64-bit as opposed to effecting it by way of the WinXP x64 CD or
through Vista.
Even though my Gateway quad-core came with a free upgrade to Vista,and at least one of the Vista Premium packages
supports 64-bit,the motherboard won't permit it.
![[hu]](/mwf/v12/flags/hu.png "Hungary")
![[us]](/mwf/v12/flags/us.png "United States")
This is a major reason why I'm pushing for a linux version of Rybka. As far as 64 bit computing is concerned, linux has all the hardware support, and Vista isn't changing that any time soon. Not only that, but WinXP x64 can't compete performance wise to linux, and it's getting WORSE with Vista. Using wine(windows emulator for linux) with Rybka is tricky; I'm STILL working on figuring that out. Vas, please consider making a linux version of Rybka for version 3. With Vista coming out and all, 6 months-1 year from now you'll likely have a fairly large consumer base for it.
I haven't really thought about it, but chess engines should be the easiest type of applications to port between operating systems.
probably the only part which has to be OS specific is the Input and Output handling of the engine (with a few things that can actually be the same in here too).
There is no GUI in the engine itself, and the non IO, non GUI parts of the engine are purely computational, so there should be no pressing reason to use the system's APIs. However, these things CAN definitely be written to be more dependent upon a specific OS, and it may be more convenient for fast development, especially in the early days of the engine.
I think it's too much to ask Vas to maintain and develop different versions for the 2 OSs at such an early stage of development, when there is still so much development to do (I know you're thinking that it's already the best engine, but the code is pretty immature, only a couple of years, without too many programmers working on it (understatement...)).
Also, if there are any plans to add a GUI in the future, to build extra features upon the engine capabilities, porting becomes a royal pain.
Although, I have to agree that performance wise, 64 bit Windows just cannot compete with 64 bit Linux, even with XP, and it will definitely be worse with Vista/Longhorn.
probably the only part which has to be OS specific is the Input and Output handling of the engine (with a few things that can actually be the same in here too).
There is no GUI in the engine itself, and the non IO, non GUI parts of the engine are purely computational, so there should be no pressing reason to use the system's APIs. However, these things CAN definitely be written to be more dependent upon a specific OS, and it may be more convenient for fast development, especially in the early days of the engine.
I think it's too much to ask Vas to maintain and develop different versions for the 2 OSs at such an early stage of development, when there is still so much development to do (I know you're thinking that it's already the best engine, but the code is pretty immature, only a couple of years, without too many programmers working on it (understatement...)).
Also, if there are any plans to add a GUI in the future, to build extra features upon the engine capabilities, porting becomes a royal pain.
Although, I have to agree that performance wise, 64 bit Windows just cannot compete with 64 bit Linux, even with XP, and it will definitely be worse with Vista/Longhorn.
I'd argue that the BEST time to port any program is in the early days of development. If a GUI was ported, that would be a real plus, and would probably be a real hit, but that doesn't need to happen since it can be plugged into any other GUI. Ideally, he'd just release the Rybka 1.0 Beta source code and the Linux community would jump all over it, but I know that's too much to ask from Vas, would most likely only hurt him in the long run with the competition. Yea, it can be a pain to port anything, but Vas would at least be sure he'd have a large market 1 year from now. Imagine the uproar of demand a year from now if Linux is in much higher demand, and people who want Rybka learn that it only runs on Windows, knowing that Linux would perform far better with it.
I believe that Vas simply Can't do it. Not because of lack of ability, but because he does not have the time.
He'll need the help of more programmers. it's practically impossible, when you're pushing the schedule for so many improvements (which are more important, and would have a larger impact on Rybka's strength than Linux performance would), porting becomes unrealistic.
Also, porting is practically always impossible in commercial products in the very beginning. For lots of reasons, but I've seen people trying to port many projects before they had them nearly complete (Rybka is not there yet, as strong as it is). The result was always bad.
You seem to be talking from an open source perspective, and indeed, only open source projects can (and should, IMO) worry about porting in the early stages. When you have a community to help you, there are bound to be at least one or 2 (or 40) people who want that project to run on their OS and are willing to keep watching the main development efforts...
The BEST time to port an open source project is, indeed, in the early days.
I have yet to even hear of a single closed source commercial project that tried this approach and survived
(and I've known many, with superb programmers on their teams... the showstoppers were never programming issues. It takes too many resources that an open source project just HAS right from the start, and a closed source commercial project does not).
Rybka, however, is not open source, for obvious reasons... And I hope Vas keeps earning well on the fish.
As for people WANTING Rybka on Linux and complaining it only runs on windows, that sentence has so many assumptions behind it:
1) The expectations that Linux will be in a much higher demand is a bit unrealistic. The demand WILL probably increase. But it will take time for Linux demand to REALLY increase for home computers (EVEN if they have specific reasons for wanting it. So far, only hi tech workers / lovers (like me) seem to have it on their home computers. Sometimes some of their friends who were (politely) bludgeoned to do so...).
2) Assuming the demand for Rybka for Linux would increase because of a massive performance difference. You have to be careful here,
there will be a performance boost speed wise, but the effects on Rybka's ratings may be very small (and porting will be necessary just to test it...).
3) A smart person once said: "Everything is relative". As long as Rybka is stronger than the other engines, most people won't care. Sure, they'll want it to be stronger, but as long as it doesn't hurt, the demand won't shift too much in Linux's favor.
4) Rybka's search/tactics/heuristics are all still in their early stages of development. The heuristics themselves seem to exhibit many unwanted behaviors. Improving these would most probably create a much larger increase in performance than porting to Linux.
That being said, of course it would be grand if Vas just hires another programmer to be responsible for the porting...
But who am I to say when is the right time (or, probably, when are the earnings from Rybka large enough to hire more help).
He'll need the help of more programmers. it's practically impossible, when you're pushing the schedule for so many improvements (which are more important, and would have a larger impact on Rybka's strength than Linux performance would), porting becomes unrealistic.
Also, porting is practically always impossible in commercial products in the very beginning. For lots of reasons, but I've seen people trying to port many projects before they had them nearly complete (Rybka is not there yet, as strong as it is). The result was always bad.
You seem to be talking from an open source perspective, and indeed, only open source projects can (and should, IMO) worry about porting in the early stages. When you have a community to help you, there are bound to be at least one or 2 (or 40) people who want that project to run on their OS and are willing to keep watching the main development efforts...
The BEST time to port an open source project is, indeed, in the early days.
I have yet to even hear of a single closed source commercial project that tried this approach and survived
(and I've known many, with superb programmers on their teams... the showstoppers were never programming issues. It takes too many resources that an open source project just HAS right from the start, and a closed source commercial project does not).
Rybka, however, is not open source, for obvious reasons... And I hope Vas keeps earning well on the fish.
As for people WANTING Rybka on Linux and complaining it only runs on windows, that sentence has so many assumptions behind it:
1) The expectations that Linux will be in a much higher demand is a bit unrealistic. The demand WILL probably increase. But it will take time for Linux demand to REALLY increase for home computers (EVEN if they have specific reasons for wanting it. So far, only hi tech workers / lovers (like me) seem to have it on their home computers. Sometimes some of their friends who were (politely) bludgeoned to do so...).
2) Assuming the demand for Rybka for Linux would increase because of a massive performance difference. You have to be careful here,
there will be a performance boost speed wise, but the effects on Rybka's ratings may be very small (and porting will be necessary just to test it...).
3) A smart person once said: "Everything is relative". As long as Rybka is stronger than the other engines, most people won't care. Sure, they'll want it to be stronger, but as long as it doesn't hurt, the demand won't shift too much in Linux's favor.
4) Rybka's search/tactics/heuristics are all still in their early stages of development. The heuristics themselves seem to exhibit many unwanted behaviors. Improving these would most probably create a much larger increase in performance than porting to Linux.
That being said, of course it would be grand if Vas just hires another programmer to be responsible for the porting...
But who am I to say when is the right time (or, probably, when are the earnings from Rybka large enough to hire more help).
Ok, points well taken. I personally though would be more than happy to wait a few more months for 3.0 if there was a Linux port. Vas would probably be beneftted in the long run by taking the extra time to do this, I would think.
![[ph]](/mwf/v12/flags/ph.png "Philippines")
Wow.That's so unfortunate.I have a really cheap ASUS motherboard (P5VD2 for under 80 bucks) running on my Stock Core 2 Duo and so far so good,feels smooth on my 64-bit OS (but this one sucks at OCs lol)...That's strange,I thought these modern-day dual core motherboards are all 64-bit capable,a bit too greedy on their part.
Hi,
I reckon 99.9% well maybe a bit less :) of computer users dont have a clue how to install\setup Linux and many wont have even heard of it lolol .... What performance increase if any would there be? I know there is a Linux version of Shredder\Deep Shredder maybe asking SMK about this would help .... Performance with Vista Ultimate x64 is fine compared to XP64 etc ...
Regards
I reckon 99.9% well maybe a bit less :) of computer users dont have a clue how to install\setup Linux and many wont have even heard of it lolol .... What performance increase if any would there be? I know there is a Linux version of Shredder\Deep Shredder maybe asking SMK about this would help .... Performance with Vista Ultimate x64 is fine compared to XP64 etc ...
Regards
64 bit Linux seems to be quite a bit faster than 64 bit windows.
Also, performance for raw computing power applications is will only degrade with Vista in the long run, since MS have a policy to count on processors and computer hardware in general to get stronger. Which is actually a correct assumption, which is fine for most users' needs, and may actually be good for them (they care more about the look and feel and the features than the rest... Performance should just remain fine).
Chess engines prefer supersonic performance, and in that, the gap between Linux and Windows will only increase.
(Actually, performance gap in IO is even worse, but irrelevant for chess engines).
But even if the trends are ignored, right now Linux64 bit is dozens of % faster than Windows 64bit.
What's more, Linux environments can be tuned to work even faster for certain types of programs. It does require quite a bit of knowledge though. Most people don't have that.
Still, not sure how many people would care that much about the few added rating points enough to leave their comfy Windows... (I like Linux, but if you're used to working on Windows, or you do most of your work on Windows, switching can be a pain, and most people don't like to have the burden of 2 operating systems installed with multi boot (or run at the same time, if anyone is following the Virtualization scene...) on the same computer.
Still, many people in the Linux community will love to see a Rybka version for it.
I'm just realistic as to if / when that would happen. Maybe never, maybe in a long time... For absolutely understandable and acceptable reasons.
If, however, such a version ever comes out, Vas can be sure one of his first customers / testers have just written THIS post:)
Also, performance for raw computing power applications is will only degrade with Vista in the long run, since MS have a policy to count on processors and computer hardware in general to get stronger. Which is actually a correct assumption, which is fine for most users' needs, and may actually be good for them (they care more about the look and feel and the features than the rest... Performance should just remain fine).
Chess engines prefer supersonic performance, and in that, the gap between Linux and Windows will only increase.
(Actually, performance gap in IO is even worse, but irrelevant for chess engines).
But even if the trends are ignored, right now Linux64 bit is dozens of % faster than Windows 64bit.
What's more, Linux environments can be tuned to work even faster for certain types of programs. It does require quite a bit of knowledge though. Most people don't have that.
Still, not sure how many people would care that much about the few added rating points enough to leave their comfy Windows... (I like Linux, but if you're used to working on Windows, or you do most of your work on Windows, switching can be a pain, and most people don't like to have the burden of 2 operating systems installed with multi boot (or run at the same time, if anyone is following the Virtualization scene...) on the same computer.
Still, many people in the Linux community will love to see a Rybka version for it.
I'm just realistic as to if / when that would happen. Maybe never, maybe in a long time... For absolutely understandable and acceptable reasons.
If, however, such a version ever comes out, Vas can be sure one of his first customers / testers have just written THIS post:)
Hi,
Any chance of any actual figures? as in the end this is what counts if you dont want to give Mr Bill Gates your money :) ... Dual booting is prob the best way for people to go as then you have the best of both worlds.
Regards
Any chance of any actual figures? as in the end this is what counts if you dont want to give Mr Bill Gates your money :) ... Dual booting is prob the best way for people to go as then you have the best of both worlds.
Regards
Well, as for dual booting: Agreed!
Best solution, at least for the near future.
Best solution, at least for the near future.
As for the actual figures, it depends on the exact application.
That's the problem with those things.
Since there is no Rybka for Linux, I cannot give the figures for that, and the variance between different applications is enormous.
With applications with chess engine like behavior (including some engines) I've noticed anything between 5-40%(!!! once. Only once... average at about 25) increase in raw speed / calculations.
This was without tuning the Linux environment for the App.
You want exact numbers of the situation today with specific apps?
I think I'll just repeat the test with modern kernels and both XP64 and maybe even vista 64 in the future...
and a few chess engines. See what's the situation with the up to date systems.
That's the problem with those things.
Since there is no Rybka for Linux, I cannot give the figures for that, and the variance between different applications is enormous.
With applications with chess engine like behavior (including some engines) I've noticed anything between 5-40%(!!! once. Only once... average at about 25) increase in raw speed / calculations.
This was without tuning the Linux environment for the App.
You want exact numbers of the situation today with specific apps?
I think I'll just repeat the test with modern kernels and both XP64 and maybe even vista 64 in the future...
and a few chess engines. See what's the situation with the up to date systems.
Hi,
I know it is hard to get exact figures etc, that is why I suggested trying to find out how Deep Shredder x64 on Linux http://www.shredderchess.com/chess-software/linux.html compares to Deep Shredder x64 on Windows which would at least give a comparison
Regards
I know it is hard to get exact figures etc, that is why I suggested trying to find out how Deep Shredder x64 on Linux http://www.shredderchess.com/chess-software/linux.html compares to Deep Shredder x64 on Windows which would at least give a comparison
Regards
![[fi]](/mwf/v12/flags/fi.png "Finland")
I have to admit that I am quite surprised that 64-bit Linux it that much faster. I find it hard to believe.
There are several reasons. 64-bit computing is not always faster than 32-bit because 64-bit instructions are slower like 64-bit mul is slower than 32-bit mul.
http://www.agner.org/optimize
5 vs. 7.
I have written some performance critical scientifical applications and sometimes 64-bit version is slower than 32-bit version. Pointers are longer so you have less cache.
But we know that 64-bit rybka is a lot faster than 32-bit. Probably because there are 64-bit additions that are a lot faster when you can use 64-bit add.
I have always thought that OS overhead is not so big.
One way to test differences is the most cpu intensive application I know = prime95. I think that it takes most out of cpu and it is hand tuned assembler.
You can get windows and linux versions from here and test if there is any speed difference:
I am surprised if there is any significant difference. Calculation code is same in mersenne95 and mprime.
(Menu Options / Benchmark)
http://www.mersenne.org/freesoft.htm
Regards,
hamlet
There are several reasons. 64-bit computing is not always faster than 32-bit because 64-bit instructions are slower like 64-bit mul is slower than 32-bit mul.
http://www.agner.org/optimize
5 vs. 7.
I have written some performance critical scientifical applications and sometimes 64-bit version is slower than 32-bit version. Pointers are longer so you have less cache.
But we know that 64-bit rybka is a lot faster than 32-bit. Probably because there are 64-bit additions that are a lot faster when you can use 64-bit add.
I have always thought that OS overhead is not so big.
One way to test differences is the most cpu intensive application I know = prime95. I think that it takes most out of cpu and it is hand tuned assembler.
You can get windows and linux versions from here and test if there is any speed difference:
I am surprised if there is any significant difference. Calculation code is same in mersenne95 and mprime.
(Menu Options / Benchmark)
http://www.mersenne.org/freesoft.htm
Regards,
hamlet
![[ee]](/mwf/v12/flags/ee.png "Estonia")
I also doubt, that chess engine on Linux system has serious advantages over the one that runs on Windows system, if they are both full 64-bit systems.
Compared to 32-bit systems a full 64-bit system gives hudge advantage for chess engines that use bitboard method (see http://en.wikipedia.org/wiki/Bitboard ) for representation of positions. The advantage is that a single 64-bit word representing one 64-bit bitboard is loaded instantly in 64-bit systems, but on 32-bit systems it takes more CPU cycles to load and calculate. I am not an expert of chess engines, but my studies of source code of some engines don't let me believe, that the code might be considerably more optimized or run faster on Linux systems. If someone tells otherwise, I'd like to hear, how the advantage is generated for Linux systems.
Compared to 32-bit systems a full 64-bit system gives hudge advantage for chess engines that use bitboard method (see http://en.wikipedia.org/wiki/Bitboard ) for representation of positions. The advantage is that a single 64-bit word representing one 64-bit bitboard is loaded instantly in 64-bit systems, but on 32-bit systems it takes more CPU cycles to load and calculate. I am not an expert of chess engines, but my studies of source code of some engines don't let me believe, that the code might be considerably more optimized or run faster on Linux systems. If someone tells otherwise, I'd like to hear, how the advantage is generated for Linux systems.
Loads should not be problem with modern cpu's because of out of order execution + prefetching. Cpu will read whole cache line from main memory to cache and this helps speed up loads.
Thanks for the link!
LOL. I read the article and well, Yes, Now I understand why chess engines are naturally 64-bit stuff. Quite trivial thing. I have to admit that I am newbie in chess engine world.
Regards,
Hamlet
Thanks for the link!
LOL. I read the article and well, Yes, Now I understand why chess engines are naturally 64-bit stuff. Quite trivial thing. I have to admit that I am newbie in chess engine world.
Regards,
Hamlet
Hi,
I have emailed Dr Robert Hyatt (Crafty) and SMK (Shredder) as both have a Linux version of there engines asking for there experience\comments on the Linux 64 v Windows 64 question ... I will post any replies that I receive :)
Regards
I have emailed Dr Robert Hyatt (Crafty) and SMK (Shredder) as both have a Linux version of there engines asking for there experience\comments on the Linux 64 v Windows 64 question ... I will post any replies that I receive :)
Regards
First thing: if the CPU is 64 bit, many operations which are usually slower are actually the same speed.
The example of additions which you gave is just one of many many things which are actually faster in 64 bit. besides, even when something takes slower, it can still be faster overall if you actually need and use all 64 bits (twice the amount of data.... only if it will be worse than twice the speed, it'll be slower in that case).
Memory can also be read a lot faster, but again, depends on the application.
the majority of applications are a lot faster on 64 (when the cpu, OS, and application supports it. If even one is missing 64 support, it'll be slower).
Granted, there are exceptions.
Just remember that you've been writing scientific applications, so I'd imagine you're doing quite a fair use of the FPU.
I think your opinion stems from there... most applications use it very little.
As to the special case of chess: it's not really only the additions (ok, those too...), there are quite a few reasons for it, and I'd imagine I can't even begin to think of them all:
I have no idea what Vas has used in his program (for example, bitboards are 64 bit...), but even just the simple memory scans (in 64 bit steps), additions and most integer operations are by and large just faster... And when done millions of times each second...
second, OS overhead with windows is enough to be felt. Linux is not exactly innocent there too... but less.
Also, linux scheduler is considerably better, and even when you're running only one process after the system starts,
all the system services/modules/window managers etc. take a little time, and linux seem to be doing a better job (not 64 bit related... of course).
Second, 64 bit windows is still in its infant stages... and it just exhibits worse behavior. I'd imagine Microsoft would want to fix it in the future,
and close the gap somewhat. But probably not all of it.
but when? I mean, even in 32 bit there's about a 2-4% increase on Linux, didn't seem to bother them (Well, it was very small...)
Finally, Linux can be run without X (its graphical environment), taking most of the overhead down.
The windows' environment overhead now is really not that large if you're doing doing absolutely nothing else and have set all preferences to "best performance", but it does use a LOT of memory.
Vista only made it worse. This means so much less memory for Hashing for out poor chess engine.
(That's why the only true reliable test is another chess engine...).
I've already made such tests as you describe before, but it has been a while. Maybe it's time to do them again.
Thanks for the incentive.
The example of additions which you gave is just one of many many things which are actually faster in 64 bit. besides, even when something takes slower, it can still be faster overall if you actually need and use all 64 bits (twice the amount of data.... only if it will be worse than twice the speed, it'll be slower in that case).
Memory can also be read a lot faster, but again, depends on the application.
the majority of applications are a lot faster on 64 (when the cpu, OS, and application supports it. If even one is missing 64 support, it'll be slower).
Granted, there are exceptions.
Just remember that you've been writing scientific applications, so I'd imagine you're doing quite a fair use of the FPU.
I think your opinion stems from there... most applications use it very little.
As to the special case of chess: it's not really only the additions (ok, those too...), there are quite a few reasons for it, and I'd imagine I can't even begin to think of them all:
I have no idea what Vas has used in his program (for example, bitboards are 64 bit...), but even just the simple memory scans (in 64 bit steps), additions and most integer operations are by and large just faster... And when done millions of times each second...
second, OS overhead with windows is enough to be felt. Linux is not exactly innocent there too... but less.
Also, linux scheduler is considerably better, and even when you're running only one process after the system starts,
all the system services/modules/window managers etc. take a little time, and linux seem to be doing a better job (not 64 bit related... of course).
Second, 64 bit windows is still in its infant stages... and it just exhibits worse behavior. I'd imagine Microsoft would want to fix it in the future,
and close the gap somewhat. But probably not all of it.
but when? I mean, even in 32 bit there's about a 2-4% increase on Linux, didn't seem to bother them (Well, it was very small...)
Finally, Linux can be run without X (its graphical environment), taking most of the overhead down.
The windows' environment overhead now is really not that large if you're doing doing absolutely nothing else and have set all preferences to "best performance", but it does use a LOT of memory.
Vista only made it worse. This means so much less memory for Hashing for out poor chess engine.
(That's why the only true reliable test is another chess engine...).
I've already made such tests as you describe before, but it has been a while. Maybe it's time to do them again.
Thanks for the incentive.
OK. I know my asm. It really depends on what you are doing if 64-bit is faster. If you are doing things with 64-bit data like big integers then 64-bit is must or when we are speaking chess engines 8x8 chess board-> 64-bit bitboards and and/or/xor stuff or other bitmap things for bitboards. Then it is obvious that 64-bit world is faster. But if we are doing floats then 64-bit Alu does not help much. And latency of some instructions depends on size of argument (like mul) and sometimes 64-bit argument is slower. So if your data is 32-bit and you are using 64-bit instructions then you might end up with slower code. But anyway when we are talking about chess engines 64-bit is faster because if you want to do something to 64-bit data you have to do 2 32-bit operations or one 64-bit operation. I agree.
latency of add is 1 and it does not depend on size of argument. Adding is faster only if you are adding numbers that are larger than 2^32-1 and I have to admit that this is not happening very often in normal applications. Cryptography is exception.
Yes. That came to my mind after my post. And I have to admit that i have had always quite a lot memory (currently 4GB and planning to upgrade to 8GB) so I have avoided memory aspect of OS. Yes, if os uses less memory then we have more memory to hash and that can speed up chess engine.
I am curious. Tell me what is stored in hash tables?
(I don't know and you seem to know) Is position evaluation expensive so evaluation value of certain position is stored to memory so it can be looked up when we meet same position again or what?
mersenne95 might be bad comparison for chess engines because it doesn't use much memory.
Regards,
Hamlet
>I have no idea what Vas has used in his program (for example, bitboards are 64 bit...), but even just the simple memory scans (in 64 bit steps), additions and >most integer operations are by and large just faster... And when done millions of times each second...
latency of add is 1 and it does not depend on size of argument. Adding is faster only if you are adding numbers that are larger than 2^32-1 and I have to admit that this is not happening very often in normal applications. Cryptography is exception.
>Also, linux scheduler is considerably better, and even when you're running only one process after the system starts,
Yes. That came to my mind after my post. And I have to admit that i have had always quite a lot memory (currently 4GB and planning to upgrade to 8GB) so I have avoided memory aspect of OS. Yes, if os uses less memory then we have more memory to hash and that can speed up chess engine.
I am curious. Tell me what is stored in hash tables?
(I don't know and you seem to know) Is position evaluation expensive so evaluation value of certain position is stored to memory so it can be looked up when we meet same position again or what?
mersenne95 might be bad comparison for chess engines because it doesn't use much memory.
Regards,
Hamlet
Yes, it's for storing the evaluation function's result on different positions.
The full position evaluation function is the slowest piece of code in a chess engine.
So you store the value of positions reached during the search so that if you reach the same positions again, you just KNOW the value.
But the thing to understand is, you do NOT do this just to save a single positional evaluation... It saves you the trouble of searching the ENTIRE subtree of the search space (in the process using who knows how many evaluations, cutoff tests, and whatnots).
In Rybka, the evaluation function seems to be a LOT slower than other engines (just compare the Nodes/sec) with Fritz, Shredder and co.
They're calculating a few times faster.
But Rybka compensates (and then some...) by much more aggressive pruning than the other engines.
Ok, after this digression, I'll have to say what happens if you don't have enough memory for a hash size you choose:
The OS will start using the swap space/file, which will have the opposite result of the one desired.
Still, IMO, 4GB seems to render this entire issue almost meaningless... Unless you have a truly supersonic moulticore/cpu system which can fill a 1GB hash relatively fast. Usually 512Mb hash tables seem to yield excellent performance for me.
(Remember, Microsoft recommends 2GB for vista.....).
Well, if you have the resources to get whatever system you want, you don't have to be plagued by these issues.
The full position evaluation function is the slowest piece of code in a chess engine.
So you store the value of positions reached during the search so that if you reach the same positions again, you just KNOW the value.
But the thing to understand is, you do NOT do this just to save a single positional evaluation... It saves you the trouble of searching the ENTIRE subtree of the search space (in the process using who knows how many evaluations, cutoff tests, and whatnots).
In Rybka, the evaluation function seems to be a LOT slower than other engines (just compare the Nodes/sec) with Fritz, Shredder and co.
They're calculating a few times faster.
But Rybka compensates (and then some...) by much more aggressive pruning than the other engines.
Ok, after this digression, I'll have to say what happens if you don't have enough memory for a hash size you choose:
The OS will start using the swap space/file, which will have the opposite result of the one desired.
Still, IMO, 4GB seems to render this entire issue almost meaningless... Unless you have a truly supersonic moulticore/cpu system which can fill a 1GB hash relatively fast. Usually 512Mb hash tables seem to yield excellent performance for me.
(Remember, Microsoft recommends 2GB for vista.....).
Well, if you have the resources to get whatever system you want, you don't have to be plagued by these issues.
Hi, I received the below email from Dr Hyatt ...
From an operating system perspective, there is no difference at all
since Crafty doesn't use any operating system services to speak of
during the search. Typically windows is faster for Crafty because MSVC
is a better compiler than gcc. But the 64 bit version of Intel's C
compiler is probably about the same as MSVC so that advantage is only
present if you use linux/gcc.
Otherwise, linux is more responsive, and seems to run better with
xboard/crafty than windows with winboard/crafty...
Robert M. Hyatt, Ph.D. Computer and Information Sciences
hyatt@uab.edu University of Alabama at Birmingham
(205) 934-2213 136A Campbell Hall
(205) 934-5473 FAX Birmingham, AL 35294-1170
From an operating system perspective, there is no difference at all
since Crafty doesn't use any operating system services to speak of
during the search. Typically windows is faster for Crafty because MSVC
is a better compiler than gcc. But the 64 bit version of Intel's C
compiler is probably about the same as MSVC so that advantage is only
present if you use linux/gcc.
Otherwise, linux is more responsive, and seems to run better with
xboard/crafty than windows with winboard/crafty...
Robert M. Hyatt, Ph.D. Computer and Information Sciences
hyatt@uab.edu University of Alabama at Birmingham
(205) 934-2213 136A Campbell Hall
(205) 934-5473 FAX Birmingham, AL 35294-1170
Well, the same would be said on pretty much any interface you use with Linux Vs. Windows.
You don't need to use operating system specific services in order to have a difference in performance on different operating systems.
Well, to be fair, Dr. Hyatt DID say "From an operating system perspective"...
Hyatt's claim that MSVC is a better compiler than gcc is far from being true. It is, unfortunately, not completely false either. It depends on exactly what you are compiling, and frankly, with Chess engines, I do not think this is the case from my tests.
One of the reasons 64bit linux is a bit better than Windows is simply that the operating system itself is written slightly better. The handling of OTHER processes / services on the system (which are not the running engine), the scheduler itself is MUCH better (that's one of the reasons for Hyatt's mention of "better responsiveness" on Linux. Though in reality, there are other reasons too).
I can imagine Microsoft will strive to change that, and I'll imagine they'll do an OK job, but as always, that will be at their own pace, which can take a Looooong time (not really their fault, that's the problem with any large company).
In addition, Linux can be tuned to work even better with Chess engines, while windows will remain (even at "tune for best performance" setting) an extremely heavy system, incurring a slight (though slight indeed) overhead on performance, and a HUGE overhead on memory
Which brings me to my last point, which is a performance problem specific to Chess Engines:
The huge memory consumption of the OS causes a lot less memory to be available to the Chess engine hash table.
This means that you either have to use a smaller hash table, risk having low memory and the OS using your swap space / file instead (ouch. sooo slooow), or buy 4 GB of memory or more specifically if you plan on using Vista + Chess engines.
Vista will still incur a small overhead, however, it may indeed be small.
One interesting point made by Hyatt: I was talking only about the engine itself, but Hyatt's reference to the interfaces themselves running better on Linux due to better responsiveness is also an issue (even if a small one).
There are many more points, issues, and differences, but I don't want to turn this into a congressional debate (at least not on this forum at this stage).
In the end, it is a resource management issue, but of the programmers more than the OS...
If Vas decide's that investing time on a releasing a Linux version someday, It may become a more interesting topic.
Until then, Rybka's performance on Windows Vs. Linux 64 is a bit hard to compare.....
You don't need to use operating system specific services in order to have a difference in performance on different operating systems.
Well, to be fair, Dr. Hyatt DID say "From an operating system perspective"...
Hyatt's claim that MSVC is a better compiler than gcc is far from being true. It is, unfortunately, not completely false either. It depends on exactly what you are compiling, and frankly, with Chess engines, I do not think this is the case from my tests.
One of the reasons 64bit linux is a bit better than Windows is simply that the operating system itself is written slightly better. The handling of OTHER processes / services on the system (which are not the running engine), the scheduler itself is MUCH better (that's one of the reasons for Hyatt's mention of "better responsiveness" on Linux. Though in reality, there are other reasons too).
I can imagine Microsoft will strive to change that, and I'll imagine they'll do an OK job, but as always, that will be at their own pace, which can take a Looooong time (not really their fault, that's the problem with any large company).
In addition, Linux can be tuned to work even better with Chess engines, while windows will remain (even at "tune for best performance" setting) an extremely heavy system, incurring a slight (though slight indeed) overhead on performance, and a HUGE overhead on memory
Which brings me to my last point, which is a performance problem specific to Chess Engines:
The huge memory consumption of the OS causes a lot less memory to be available to the Chess engine hash table.
This means that you either have to use a smaller hash table, risk having low memory and the OS using your swap space / file instead (ouch. sooo slooow), or buy 4 GB of memory or more specifically if you plan on using Vista + Chess engines.
Vista will still incur a small overhead, however, it may indeed be small.
One interesting point made by Hyatt: I was talking only about the engine itself, but Hyatt's reference to the interfaces themselves running better on Linux due to better responsiveness is also an issue (even if a small one).
There are many more points, issues, and differences, but I don't want to turn this into a congressional debate (at least not on this forum at this stage).
In the end, it is a resource management issue, but of the programmers more than the OS...
If Vas decide's that investing time on a releasing a Linux version someday, It may become a more interesting topic.
Until then, Rybka's performance on Windows Vs. Linux 64 is a bit hard to compare.....
There won't be any real difference in performance.
The reason to port is to pick up some extra users. I just need to see where it will fit into the schedule. These decisions will be made after 2.3.2 is finished.
Vas
The reason to port is to pick up some extra users. I just need to see where it will fit into the schedule. These decisions will be made after 2.3.2 is finished.
Vas
Sorry Vas, but I have trouble believing there won't be a performance difference between 64 bit linux and say, Vista. The drain on system resources with Vista is too great to not be an issue. Anyways, I'm glad you're considering porting Rybka to Linux, and I guarentee you I'll be a return customer if you do:)
Hi,
Just received an email from SMK which is shown below:
Hello,
Shredder mainly uses the CPU, so there should be no big difference between
Linux and Windows. However as Shredder is written in C the compiler plays an
important role. Here the Microsoft compilers are best for Shredder which is
why the Windows versions of Shredder are a bit faster.
Best regards
Stefan Meyer-Kahlen
Just received an email from SMK which is shown below:
Hello,
Shredder mainly uses the CPU, so there should be no big difference between
Linux and Windows. However as Shredder is written in C the compiler plays an
important role. Here the Microsoft compilers are best for Shredder which is
why the Windows versions of Shredder are a bit faster.
Best regards
Stefan Meyer-Kahlen
As far as memory and hash tables are concerned, I think most would agree that Linux has the edge, not only from less overhead, but also because it has the ability to use different GUIs(such as Fluxbox, which needs a minimal amount of resources), or even no GUI at all. Microsoft's policy(as I believe was mentioned earlier) is to make Windows more resource intensive as computers get more powerful. With this mindset, the gap will only get bigger..
Then the CPU issue. Microsoft originally designed their code in mind for the average home user to use, away from the internet or even networking in general. In short, it wasn't an operating system designed to push the hardware to the limits. Unix, on the other hand, was designed from the ground up for performance, to use with mainframes, specifically. Dummy terminals, etc all spawned out of this idea.
Out of the box, Linux might not have a huge advantage at the moment. But Linux definately has more possibilities to leave Microsoft in the dust on this, and Windows is moving more towards the extreme of their original philosphy with Vista.
Then the CPU issue. Microsoft originally designed their code in mind for the average home user to use, away from the internet or even networking in general. In short, it wasn't an operating system designed to push the hardware to the limits. Unix, on the other hand, was designed from the ground up for performance, to use with mainframes, specifically. Dummy terminals, etc all spawned out of this idea.
Out of the box, Linux might not have a huge advantage at the moment. But Linux definately has more possibilities to leave Microsoft in the dust on this, and Windows is moving more towards the extreme of their original philosphy with Vista.
I partially agree with this (leave it "in the dust" probably won't happen... but making it better is very probably).
I dare not make any predictions here.
Time will tell...
Time will tell...
I dare not make any predictions here.
Time will tell...
Time will tell...
It's true that Windows goes in this direction, but this really only applies to graphical applications.
At my last job, our unreleased to-be-flagship product (written in C#) had a 1-minute startup time! And weeks of investigations into the problem didn't help. Much to the amusement of the Mac guys in the office :)
Vas
At my last job, our unreleased to-be-flagship product (written in C#) had a 1-minute startup time! And weeks of investigations into the problem didn't help. Much to the amusement of the Mac guys in the office :)
Vas
Heh C#. That's a pretty slow language in of itself. Wouldn't you consider the GUI a graphical application, though? Anyways, I personally wouldn't touch C#. Was thinking about dabbling a bit in Visual Basic, but since I use a Linux box almost exclusively, it seems kind of pointless. I'll stick to C++ and Java for the moment, maybe take on a little bit of Assembly.
Java is the same way. As hardware gets better, the VM will get bigger. Really, there's nothing wrong with this.
Vas
Vas
Of course Java is the same way.. Microsoft basically made C# to copy Java. Java is great for some applications, such as small ones and ones that don't rely on speed, but for other apps I'd cringe when someone tries to write something big, and that needs speed, with Java/C#. Vista would be one of those apps, I think it's a horrible idea that an operating system is based off C#. No wonder it needs 1 gig of ram minimum. How ridiculous.
Actually, it's all very logical.
For most apps, speed doesn't matter. What matters is the quality of the abstraction layer available to the programmer.
Developing a GUI is much faster in C# or Java than for example in MFC.
Vas
For most apps, speed doesn't matter. What matters is the quality of the abstraction layer available to the programmer.
Developing a GUI is much faster in C# or Java than for example in MFC.
Vas
Oh I agree with you. But I'd think for a GUI specifically, speed WOULD matter. It's obviously much faster to develop programs in an interpreted language like C# or Java, so it would make sense to do such if you can do everything you need to do with it with a high quality. But I can't help but think the RAM requirements for Vista might possibly even be cut in half if it was done in C/C++.
Sure, that's true. You have to draw the line somewhere - assembly on one extreme, .NET on the other. :)
As hardware prices contiue to fall, programmer time becomes more valuable than CPU cycles. I think this trend is going to continue. In 10 years, today's .NET will be considered 'coding to the metal'.
Vas
As hardware prices contiue to fall, programmer time becomes more valuable than CPU cycles. I think this trend is going to continue. In 10 years, today's .NET will be considered 'coding to the metal'.
Vas
Having started this thread,I feel responsible to report the outcome on my end.
SUCCESS!!!
Via dual booting I now have 32-bit AND 64-bit OSs.For the 64-bit I used the WinXPPro x64 CD trial version.
However,even with the hands-on help of a computer technician/programmer professional,the conversion took
7(SEVEN!)hours.Hell,just reformatting the hard drive took one hour!
Knowing everything I now know,it's possible to imagine accomplishing the same result in perhaps an hour or so
less time but,in any event,working this out on my Gateway FX530XT(which has no external floppy disc drive) is a
very big deal.
This may be a hardware-specific matter because Gateway,itself,not only provided no 64-bit support but,also,seemed
to have done everything possible on its flagship computer to block 64-bit computing.
Convekta was kind enough to immediately respond to my request by providing me with a new download link for the Rybka
2.3.1 Beta 64-bit for multi-processors version which I downloaded and installed through my Deep Fritz 10 GUI...so,all is now
well on this end.
I thank all of you posters for any guidance you provided and recognize that the whole Linux discussion that ensued was an
opportunity for the more highly computer literate amongst you to have what was for me,in essence,your own private conversation
..........in Greek!
Best regards,David
SUCCESS!!!
Via dual booting I now have 32-bit AND 64-bit OSs.For the 64-bit I used the WinXPPro x64 CD trial version.
However,even with the hands-on help of a computer technician/programmer professional,the conversion took
7(SEVEN!)hours.Hell,just reformatting the hard drive took one hour!
Knowing everything I now know,it's possible to imagine accomplishing the same result in perhaps an hour or so
less time but,in any event,working this out on my Gateway FX530XT(which has no external floppy disc drive) is a
very big deal.
This may be a hardware-specific matter because Gateway,itself,not only provided no 64-bit support but,also,seemed
to have done everything possible on its flagship computer to block 64-bit computing.
Convekta was kind enough to immediately respond to my request by providing me with a new download link for the Rybka
2.3.1 Beta 64-bit for multi-processors version which I downloaded and installed through my Deep Fritz 10 GUI...so,all is now
well on this end.
I thank all of you posters for any guidance you provided and recognize that the whole Linux discussion that ensued was an
opportunity for the more highly computer literate amongst you to have what was for me,in essence,your own private conversation
..........in Greek!
Best regards,David
Congrats. I'm quite proud to have started a firestorm of discussion about Linux and how it compares to Linux, as well as hardware comparisions. There's alot of good stuff here.
I don't claim to know all these terms, etc either, but I do know a thing or two about programming, and so just simply reading about this stuff helps me learn, which is really my whole aim(other than to convince Vas to port Rybka, of course). I do hope to see 64 bit Linux as an alternative to Windows, because there HAS to be one.
Yea, these hardware manufacturers really aren't comfortable supporting 64 bit operating systems because of all the problems that 64 bit Windows has. This is the nice thing about Linux: the open-sourceness allows this stuff to be made, and be made well, without relying on the hardware manufacturers(for the most part, but you'll still see certain hardware drivers requested, or the specifications requested from the manufacturers so the Linux community can get the best performance possible out of the hardware, but this isn't needed for the majority of the hardware).
I don't claim to know all these terms, etc either, but I do know a thing or two about programming, and so just simply reading about this stuff helps me learn, which is really my whole aim(other than to convince Vas to port Rybka, of course). I do hope to see 64 bit Linux as an alternative to Windows, because there HAS to be one.
Yea, these hardware manufacturers really aren't comfortable supporting 64 bit operating systems because of all the problems that 64 bit Windows has. This is the nice thing about Linux: the open-sourceness allows this stuff to be made, and be made well, without relying on the hardware manufacturers(for the most part, but you'll still see certain hardware drivers requested, or the specifications requested from the manufacturers so the Linux community can get the best performance possible out of the hardware, but this isn't needed for the majority of the hardware).
Excellent, I think you'll like the performance.
And yes, in my experience, manufacturers often don't provide 64 bit drivers. I really don't understand this - how can you sell a core 2 duo (for example) and expect people to run it in 32 bits? It's like buying a Porche and driving it around in second gear.
Vas
And yes, in my experience, manufacturers often don't provide 64 bit drivers. I really don't understand this - how can you sell a core 2 duo (for example) and expect people to run it in 32 bits? It's like buying a Porche and driving it around in second gear.
Vas
Via dual booting I now have 32-bit AND 64-bit OSs.For the 64-bit I used the WinXPPro x64 CD trial version.
David,
I just copied what you did but I can't get my internet connection...32bit connects but 64bit doesn't.
Sony is also giving me the run around on my new dual core laptop with 64bit help!
I would enjoy talking on the phone (my dime!) with you on this problem of mine.
Thanks
David,
I just copied what you did but I can't get my internet connection...32bit connects but 64bit doesn't.
Sony is also giving me the run around on my new dual core laptop with 64bit help!
I would enjoy talking on the phone (my dime!) with you on this problem of mine.
Thanks
Yea, manufacturers are afraid to offer any kind of 64 bit support whatsoever.. so you're pretty much on your own from that point. When I had tried to install WinXP x64 on my laptop a while back and got a BSOD upon startup, I called HP and they said "We don't support XP x64 and if you installed it on your laptop your warranty is null and void blah blah blah blah blah blah blah." It could be that your WinXP x64 just doesn't support/have drivers for the NIC that you have. That's why I'm running 64 bit Linux.. all my hardware works and I've been having no problems:) Of course, if you want 64 bit just to run Rybka, Linux probably won't work for you:( Come on, Vas!
You're probably correct. I suppose technically that would make today's low level programming languages even MORE valuable in the future. If programming languages increase at the same pace and computers do, there'll be a HUGE gap in performance between today's assembly/C/C++ as opposed to what kind of languages we think might be made in the future. This is actually pretty interesting, since programmers will be able to choose a wide variety of possible tradeoffs between performance and programmer time. I'm guessing the performance between programs written in low level and high level programming languages will continue to drift away. The problem with that theory is, it depends on how much heavier the high level languages actually get compared to the increases in the hardware. With Microsoft leading the charge though, I guess I'm kind of skeptical of that though:)
This trend will continue for GUIs and web apps. There is no real reason to fight against it, unless you are a connoisseur of ancient hardware :)
For algorithms, drivers, etc, low-level stuff like C++ will always be around.
Vas
For algorithms, drivers, etc, low-level stuff like C++ will always be around.
Vas
I did not follow all the discussion above. I think you had some discussion about what OS gives you the best performance. The main issues here are:
+Linux system usage can be customized. if you really want best performance from your machine, you can compile your custom kerne.
You can build your OS from sources and optimize it to play chess. For instance you can set up many cpu and memory parameters to tune up the memory usage and SMP efficiency (of course, the optimal value of the parameters depend on what you have to do; in this case it is just "playing chess"). Of course, you would choose this kind of kernel at start up only if you want to play chess; you can ran a standard kernel for standard use by default.
+You can find many minimal windows managers in linux. fluxbox is a popular one, although flwm and jwm are even more efficient. you can save a lot of memory and (a little) cpu time using them. eventually you can disable X, and see what rybka thinks straight from the terminal. More generally, you can really keep your linux system extremely minimal and efficient.
-The standard (free) C++ compilers in linux are "generic-purpose", and optimization is not always the main focus. On the other hand, rybka has not only been developed thinking to windows, but there are many performance-focused C++ compilers in windows. Anyway, there exist also commercial linux compilers with extreme performance, possibly the best one. Check this out
http://www.intel.com/cd/software/products/asmo-na/eng/compilers/277618.htm
With these two + and one - in mind, if a rybka linux version were out, I would suggest to the extreme freestylers to switch to linux, and to spend some time to tune up their system for chess. I would suggest to all the others to switch to linux, but not to do that just to gain a 2% speed up in kN count.
+Linux system usage can be customized. if you really want best performance from your machine, you can compile your custom kerne.
You can build your OS from sources and optimize it to play chess. For instance you can set up many cpu and memory parameters to tune up the memory usage and SMP efficiency (of course, the optimal value of the parameters depend on what you have to do; in this case it is just "playing chess"). Of course, you would choose this kind of kernel at start up only if you want to play chess; you can ran a standard kernel for standard use by default.
+You can find many minimal windows managers in linux. fluxbox is a popular one, although flwm and jwm are even more efficient. you can save a lot of memory and (a little) cpu time using them. eventually you can disable X, and see what rybka thinks straight from the terminal. More generally, you can really keep your linux system extremely minimal and efficient.
-The standard (free) C++ compilers in linux are "generic-purpose", and optimization is not always the main focus. On the other hand, rybka has not only been developed thinking to windows, but there are many performance-focused C++ compilers in windows. Anyway, there exist also commercial linux compilers with extreme performance, possibly the best one. Check this out
http://www.intel.com/cd/software/products/asmo-na/eng/compilers/277618.htm
With these two + and one - in mind, if a rybka linux version were out, I would suggest to the extreme freestylers to switch to linux, and to spend some time to tune up their system for chess. I would suggest to all the others to switch to linux, but not to do that just to gain a 2% speed up in kN count.
Not only that, but Linux simply does high performance computing better than Windows. There's better hardware compatibility with 64 bit Linux than there is with Windows as well. The nice thing about Linux is that you have the whole range of options to speed up a potential Linux-native Rybka, without having to go to some unsupported or buggy/slow OS to do it.
Please note that all of these issues are not going to impact performance by more than 1 or 2%. When Rybka runs, the OS sits and watches and behaves itself :)
Vas
Vas
The perfomance impact factor depends a lot on your hardware. On an old machine, it can be quite noticeble. On some good and new hardware, you should be below 1%.
Yes, I suppose. Anyway, guys with old machines probably don't care about a 5% slowdown.
Vas
Vas
Indeed, from my point of view, nobody should really care about a 5% slowdown. But extreme freestylers.
Oh, I beg to differ. It'll impact my machine by about 60% if I'm able to finally dump the 32 bit Rybka and go to the 64 bit version :) That's the difference in my case. My laptop doesn't like WinXP x64.
You drew with us in the freestyle with 32 bits? Ouch .. :)
Vas
Vas
1
2
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill