In fact,he told me this personally when I 1st bought Rybka last Spring.
Many members of this site have successfully switched to 64 -bits but through no lack of effort I've been unable to do so.
When I joined Rybka,I had an HP AMD Athlon 64x2 4400+ dual-core-2.2GHz XP Professional desktop so I bought the
WinXP Pro x64 CD but failed to install it because,as HP explained to me, the motherboard wouldn't support it.
3 months ago I bought the Gateway FX530XT desktop quad-core QX6700 with WinXP Professional and,again,the
motherboard wouldn't support 64-bit.
Has anyone here obtained off-the-shelf 64-bit processing or have all the successes been achieved by way of building
your own computers? I'm referring now to native 64-bit as opposed to effecting it by way of the WinXP x64 CD or
Even though my Gateway quad-core came with a free upgrade to Vista,and at least one of the Vista Premium packages
supports 64-bit,the motherboard won't permit it.
probably the only part which has to be OS specific is the Input and Output handling of the engine (with a few things that can actually be the same in here too).
There is no GUI in the engine itself, and the non IO, non GUI parts of the engine are purely computational, so there should be no pressing reason to use the system's APIs. However, these things CAN definitely be written to be more dependent upon a specific OS, and it may be more convenient for fast development, especially in the early days of the engine.
I think it's too much to ask Vas to maintain and develop different versions for the 2 OSs at such an early stage of development, when there is still so much development to do (I know you're thinking that it's already the best engine, but the code is pretty immature, only a couple of years, without too many programmers working on it (understatement...)).
Also, if there are any plans to add a GUI in the future, to build extra features upon the engine capabilities, porting becomes a royal pain.
Although, I have to agree that performance wise, 64 bit Windows just cannot compete with 64 bit Linux, even with XP, and it will definitely be worse with Vista/Longhorn.
He'll need the help of more programmers. it's practically impossible, when you're pushing the schedule for so many improvements (which are more important, and would have a larger impact on Rybka's strength than Linux performance would), porting becomes unrealistic.
Also, porting is practically always impossible in commercial products in the very beginning. For lots of reasons, but I've seen people trying to port many projects before they had them nearly complete (Rybka is not there yet, as strong as it is). The result was always bad.
You seem to be talking from an open source perspective, and indeed, only open source projects can (and should, IMO) worry about porting in the early stages. When you have a community to help you, there are bound to be at least one or 2 (or 40) people who want that project to run on their OS and are willing to keep watching the main development efforts...
The BEST time to port an open source project is, indeed, in the early days.
I have yet to even hear of a single closed source commercial project that tried this approach and survived
(and I've known many, with superb programmers on their teams... the showstoppers were never programming issues. It takes too many resources that an open source project just HAS right from the start, and a closed source commercial project does not).
Rybka, however, is not open source, for obvious reasons... And I hope Vas keeps earning well on the fish.
As for people WANTING Rybka on Linux and complaining it only runs on windows, that sentence has so many assumptions behind it:
1) The expectations that Linux will be in a much higher demand is a bit unrealistic. The demand WILL probably increase. But it will take time for Linux demand to REALLY increase for home computers (EVEN if they have specific reasons for wanting it. So far, only hi tech workers / lovers (like me) seem to have it on their home computers. Sometimes some of their friends who were (politely) bludgeoned to do so...).
2) Assuming the demand for Rybka for Linux would increase because of a massive performance difference. You have to be careful here,
there will be a performance boost speed wise, but the effects on Rybka's ratings may be very small (and porting will be necessary just to test it...).
3) A smart person once said: "Everything is relative". As long as Rybka is stronger than the other engines, most people won't care. Sure, they'll want it to be stronger, but as long as it doesn't hurt, the demand won't shift too much in Linux's favor.
4) Rybka's search/tactics/heuristics are all still in their early stages of development. The heuristics themselves seem to exhibit many unwanted behaviors. Improving these would most probably create a much larger increase in performance than porting to Linux.
That being said, of course it would be grand if Vas just hires another programmer to be responsible for the porting...
But who am I to say when is the right time (or, probably, when are the earnings from Rybka large enough to hire more help).
I reckon 99.9% well maybe a bit less :) of computer users dont have a clue how to install\setup Linux and many wont have even heard of it lolol .... What performance increase if any would there be? I know there is a Linux version of Shredder\Deep Shredder maybe asking SMK about this would help .... Performance with Vista Ultimate x64 is fine compared to XP64 etc ...
Also, performance for raw computing power applications is will only degrade with Vista in the long run, since MS have a policy to count on processors and computer hardware in general to get stronger. Which is actually a correct assumption, which is fine for most users' needs, and may actually be good for them (they care more about the look and feel and the features than the rest... Performance should just remain fine).
Chess engines prefer supersonic performance, and in that, the gap between Linux and Windows will only increase.
(Actually, performance gap in IO is even worse, but irrelevant for chess engines).
But even if the trends are ignored, right now Linux64 bit is dozens of % faster than Windows 64bit.
What's more, Linux environments can be tuned to work even faster for certain types of programs. It does require quite a bit of knowledge though. Most people don't have that.
Still, not sure how many people would care that much about the few added rating points enough to leave their comfy Windows... (I like Linux, but if you're used to working on Windows, or you do most of your work on Windows, switching can be a pain, and most people don't like to have the burden of 2 operating systems installed with multi boot (or run at the same time, if anyone is following the Virtualization scene...) on the same computer.
Still, many people in the Linux community will love to see a Rybka version for it.
I'm just realistic as to if / when that would happen. Maybe never, maybe in a long time... For absolutely understandable and acceptable reasons.
If, however, such a version ever comes out, Vas can be sure one of his first customers / testers have just written THIS post:)
Any chance of any actual figures? as in the end this is what counts if you dont want to give Mr Bill Gates your money :) ... Dual booting is prob the best way for people to go as then you have the best of both worlds.
Best solution, at least for the near future.
That's the problem with those things.
Since there is no Rybka for Linux, I cannot give the figures for that, and the variance between different applications is enormous.
With applications with chess engine like behavior (including some engines) I've noticed anything between 5-40%(!!! once. Only once... average at about 25) increase in raw speed / calculations.
This was without tuning the Linux environment for the App.
You want exact numbers of the situation today with specific apps?
I think I'll just repeat the test with modern kernels and both XP64 and maybe even vista 64 in the future...
and a few chess engines. See what's the situation with the up to date systems.
I know it is hard to get exact figures etc, that is why I suggested trying to find out how Deep Shredder x64 on Linux http://www.shredderchess.com/chess-software/linux.html compares to Deep Shredder x64 on Windows which would at least give a comparison
There are several reasons. 64-bit computing is not always faster than 32-bit because 64-bit instructions are slower like 64-bit mul is slower than 32-bit mul.
5 vs. 7.
I have written some performance critical scientifical applications and sometimes 64-bit version is slower than 32-bit version. Pointers are longer so you have less cache.
But we know that 64-bit rybka is a lot faster than 32-bit. Probably because there are 64-bit additions that are a lot faster when you can use 64-bit add.
I have always thought that OS overhead is not so big.
One way to test differences is the most cpu intensive application I know = prime95. I think that it takes most out of cpu and it is hand tuned assembler.
You can get windows and linux versions from here and test if there is any speed difference:
I am surprised if there is any significant difference. Calculation code is same in mersenne95 and mprime.
(Menu Options / Benchmark)
Compared to 32-bit systems a full 64-bit system gives hudge advantage for chess engines that use bitboard method (see http://en.wikipedia.org/wiki/Bitboard ) for representation of positions. The advantage is that a single 64-bit word representing one 64-bit bitboard is loaded instantly in 64-bit systems, but on 32-bit systems it takes more CPU cycles to load and calculate. I am not an expert of chess engines, but my studies of source code of some engines don't let me believe, that the code might be considerably more optimized or run faster on Linux systems. If someone tells otherwise, I'd like to hear, how the advantage is generated for Linux systems.
Thanks for the link!
LOL. I read the article and well, Yes, Now I understand why chess engines are naturally 64-bit stuff. Quite trivial thing. I have to admit that I am newbie in chess engine world.
I have emailed Dr Robert Hyatt (Crafty) and SMK (Shredder) as both have a Linux version of there engines asking for there experience\comments on the Linux 64 v Windows 64 question ... I will post any replies that I receive :)
The example of additions which you gave is just one of many many things which are actually faster in 64 bit. besides, even when something takes slower, it can still be faster overall if you actually need and use all 64 bits (twice the amount of data.... only if it will be worse than twice the speed, it'll be slower in that case).
Memory can also be read a lot faster, but again, depends on the application.
the majority of applications are a lot faster on 64 (when the cpu, OS, and application supports it. If even one is missing 64 support, it'll be slower).
Granted, there are exceptions.
Just remember that you've been writing scientific applications, so I'd imagine you're doing quite a fair use of the FPU.
I think your opinion stems from there... most applications use it very little.
As to the special case of chess: it's not really only the additions (ok, those too...), there are quite a few reasons for it, and I'd imagine I can't even begin to think of them all:
I have no idea what Vas has used in his program (for example, bitboards are 64 bit...), but even just the simple memory scans (in 64 bit steps), additions and most integer operations are by and large just faster... And when done millions of times each second...
second, OS overhead with windows is enough to be felt. Linux is not exactly innocent there too... but less.
Also, linux scheduler is considerably better, and even when you're running only one process after the system starts,
all the system services/modules/window managers etc. take a little time, and linux seem to be doing a better job (not 64 bit related... of course).
Second, 64 bit windows is still in its infant stages... and it just exhibits worse behavior. I'd imagine Microsoft would want to fix it in the future,
and close the gap somewhat. But probably not all of it.
but when? I mean, even in 32 bit there's about a 2-4% increase on Linux, didn't seem to bother them (Well, it was very small...)
Finally, Linux can be run without X (its graphical environment), taking most of the overhead down.
The windows' environment overhead now is really not that large if you're doing doing absolutely nothing else and have set all preferences to "best performance", but it does use a LOT of memory.
Vista only made it worse. This means so much less memory for Hashing for out poor chess engine.
(That's why the only true reliable test is another chess engine...).
I've already made such tests as you describe before, but it has been a while. Maybe it's time to do them again.
Thanks for the incentive.
>I have no idea what Vas has used in his program (for example, bitboards are 64 bit...), but even just the simple memory scans (in 64 bit steps), additions and >most integer operations are by and large just faster... And when done millions of times each second...
latency of add is 1 and it does not depend on size of argument. Adding is faster only if you are adding numbers that are larger than 2^32-1 and I have to admit that this is not happening very often in normal applications. Cryptography is exception.
>Also, linux scheduler is considerably better, and even when you're running only one process after the system starts,
Yes. That came to my mind after my post. And I have to admit that i have had always quite a lot memory (currently 4GB and planning to upgrade to 8GB) so I have avoided memory aspect of OS. Yes, if os uses less memory then we have more memory to hash and that can speed up chess engine.
I am curious. Tell me what is stored in hash tables?
(I don't know and you seem to know) Is position evaluation expensive so evaluation value of certain position is stored to memory so it can be looked up when we meet same position again or what?
mersenne95 might be bad comparison for chess engines because it doesn't use much memory.
The full position evaluation function is the slowest piece of code in a chess engine.
So you store the value of positions reached during the search so that if you reach the same positions again, you just KNOW the value.
But the thing to understand is, you do NOT do this just to save a single positional evaluation... It saves you the trouble of searching the ENTIRE subtree of the search space (in the process using who knows how many evaluations, cutoff tests, and whatnots).
In Rybka, the evaluation function seems to be a LOT slower than other engines (just compare the Nodes/sec) with Fritz, Shredder and co.
They're calculating a few times faster.
But Rybka compensates (and then some...) by much more aggressive pruning than the other engines.
Ok, after this digression, I'll have to say what happens if you don't have enough memory for a hash size you choose:
The OS will start using the swap space/file, which will have the opposite result of the one desired.
Still, IMO, 4GB seems to render this entire issue almost meaningless... Unless you have a truly supersonic moulticore/cpu system which can fill a 1GB hash relatively fast. Usually 512Mb hash tables seem to yield excellent performance for me.
(Remember, Microsoft recommends 2GB for vista.....).
Well, if you have the resources to get whatever system you want, you don't have to be plagued by these issues.
From an operating system perspective, there is no difference at all
since Crafty doesn't use any operating system services to speak of
during the search. Typically windows is faster for Crafty because MSVC
is a better compiler than gcc. But the 64 bit version of Intel's C
compiler is probably about the same as MSVC so that advantage is only
present if you use linux/gcc.
Otherwise, linux is more responsive, and seems to run better with
xboard/crafty than windows with winboard/crafty...
Robert M. Hyatt, Ph.D. Computer and Information Sciences
firstname.lastname@example.org University of Alabama at Birmingham
(205) 934-2213 136A Campbell Hall
(205) 934-5473 FAX Birmingham, AL 35294-1170
You don't need to use operating system specific services in order to have a difference in performance on different operating systems.
Well, to be fair, Dr. Hyatt DID say "From an operating system perspective"...
Hyatt's claim that MSVC is a better compiler than gcc is far from being true. It is, unfortunately, not completely false either. It depends on exactly what you are compiling, and frankly, with Chess engines, I do not think this is the case from my tests.
One of the reasons 64bit linux is a bit better than Windows is simply that the operating system itself is written slightly better. The handling of OTHER processes / services on the system (which are not the running engine), the scheduler itself is MUCH better (that's one of the reasons for Hyatt's mention of "better responsiveness" on Linux. Though in reality, there are other reasons too).
I can imagine Microsoft will strive to change that, and I'll imagine they'll do an OK job, but as always, that will be at their own pace, which can take a Looooong time (not really their fault, that's the problem with any large company).
In addition, Linux can be tuned to work even better with Chess engines, while windows will remain (even at "tune for best performance" setting) an extremely heavy system, incurring a slight (though slight indeed) overhead on performance, and a HUGE overhead on memory
Which brings me to my last point, which is a performance problem specific to Chess Engines:
The huge memory consumption of the OS causes a lot less memory to be available to the Chess engine hash table.
This means that you either have to use a smaller hash table, risk having low memory and the OS using your swap space / file instead (ouch. sooo slooow), or buy 4 GB of memory or more specifically if you plan on using Vista + Chess engines.
Vista will still incur a small overhead, however, it may indeed be small.
One interesting point made by Hyatt: I was talking only about the engine itself, but Hyatt's reference to the interfaces themselves running better on Linux due to better responsiveness is also an issue (even if a small one).
There are many more points, issues, and differences, but I don't want to turn this into a congressional debate (at least not on this forum at this stage).
In the end, it is a resource management issue, but of the programmers more than the OS...
If Vas decide's that investing time on a releasing a Linux version someday, It may become a more interesting topic.
Until then, Rybka's performance on Windows Vs. Linux 64 is a bit hard to compare.....
The reason to port is to pick up some extra users. I just need to see where it will fit into the schedule. These decisions will be made after 2.3.2 is finished.
Just received an email from SMK which is shown below:
Shredder mainly uses the CPU, so there should be no big difference between
Linux and Windows. However as Shredder is written in C the compiler plays an
important role. Here the Microsoft compilers are best for Shredder which is
why the Windows versions of Shredder are a bit faster.
Then the CPU issue. Microsoft originally designed their code in mind for the average home user to use, away from the internet or even networking in general. In short, it wasn't an operating system designed to push the hardware to the limits. Unix, on the other hand, was designed from the ground up for performance, to use with mainframes, specifically. Dummy terminals, etc all spawned out of this idea.
Out of the box, Linux might not have a huge advantage at the moment. But Linux definately has more possibilities to leave Microsoft in the dust on this, and Windows is moving more towards the extreme of their original philosphy with Vista.
I dare not make any predictions here.
Time will tell...
Time will tell...
At my last job, our unreleased to-be-flagship product (written in C#) had a 1-minute startup time! And weeks of investigations into the problem didn't help. Much to the amusement of the Mac guys in the office :)
For most apps, speed doesn't matter. What matters is the quality of the abstraction layer available to the programmer.
Developing a GUI is much faster in C# or Java than for example in MFC.
As hardware prices contiue to fall, programmer time becomes more valuable than CPU cycles. I think this trend is going to continue. In 10 years, today's .NET will be considered 'coding to the metal'.
Via dual booting I now have 32-bit AND 64-bit OSs.For the 64-bit I used the WinXPPro x64 CD trial version.
However,even with the hands-on help of a computer technician/programmer professional,the conversion took
7(SEVEN!)hours.Hell,just reformatting the hard drive took one hour!
Knowing everything I now know,it's possible to imagine accomplishing the same result in perhaps an hour or so
less time but,in any event,working this out on my Gateway FX530XT(which has no external floppy disc drive) is a
very big deal.
This may be a hardware-specific matter because Gateway,itself,not only provided no 64-bit support but,also,seemed
to have done everything possible on its flagship computer to block 64-bit computing.
Convekta was kind enough to immediately respond to my request by providing me with a new download link for the Rybka
2.3.1 Beta 64-bit for multi-processors version which I downloaded and installed through my Deep Fritz 10 GUI...so,all is now
well on this end.
I thank all of you posters for any guidance you provided and recognize that the whole Linux discussion that ensued was an
opportunity for the more highly computer literate amongst you to have what was for me,in essence,your own private conversation
I don't claim to know all these terms, etc either, but I do know a thing or two about programming, and so just simply reading about this stuff helps me learn, which is really my whole aim(other than to convince Vas to port Rybka, of course). I do hope to see 64 bit Linux as an alternative to Windows, because there HAS to be one.
Yea, these hardware manufacturers really aren't comfortable supporting 64 bit operating systems because of all the problems that 64 bit Windows has. This is the nice thing about Linux: the open-sourceness allows this stuff to be made, and be made well, without relying on the hardware manufacturers(for the most part, but you'll still see certain hardware drivers requested, or the specifications requested from the manufacturers so the Linux community can get the best performance possible out of the hardware, but this isn't needed for the majority of the hardware).
And yes, in my experience, manufacturers often don't provide 64 bit drivers. I really don't understand this - how can you sell a core 2 duo (for example) and expect people to run it in 32 bits? It's like buying a Porche and driving it around in second gear.
I just copied what you did but I can't get my internet connection...32bit connects but 64bit doesn't.
Sony is also giving me the run around on my new dual core laptop with 64bit help!
I would enjoy talking on the phone (my dime!) with you on this problem of mine.
For algorithms, drivers, etc, low-level stuff like C++ will always be around.
+Linux system usage can be customized. if you really want best performance from your machine, you can compile your custom kerne.
You can build your OS from sources and optimize it to play chess. For instance you can set up many cpu and memory parameters to tune up the memory usage and SMP efficiency (of course, the optimal value of the parameters depend on what you have to do; in this case it is just "playing chess"). Of course, you would choose this kind of kernel at start up only if you want to play chess; you can ran a standard kernel for standard use by default.
+You can find many minimal windows managers in linux. fluxbox is a popular one, although flwm and jwm are even more efficient. you can save a lot of memory and (a little) cpu time using them. eventually you can disable X, and see what rybka thinks straight from the terminal. More generally, you can really keep your linux system extremely minimal and efficient.
-The standard (free) C++ compilers in linux are "generic-purpose", and optimization is not always the main focus. On the other hand, rybka has not only been developed thinking to windows, but there are many performance-focused C++ compilers in windows. Anyway, there exist also commercial linux compilers with extreme performance, possibly the best one. Check this out
With these two + and one - in mind, if a rybka linux version were out, I would suggest to the extreme freestylers to switch to linux, and to spend some time to tune up their system for chess. I would suggest to all the others to switch to linux, but not to do that just to gain a 2% speed up in kN count.
Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill