Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / Hyperthreading debate reopened?
1 2 Previous Next  
- - By Zat (*) Date 2016-12-03 18:35
Another user informs me that Komodo associate Larry Kaufman advises contempt setting of "7" and Hyperthreading "ON"

Any idea if there has been a shift in thinking about hyperthreading in the community?

Might this change be viable using say SF8 on dual CPU's?

Thanks in advance,

Zat
Parent - - By APassionforCriminalJustice (***) Date 2016-12-04 03:34 Edited 2016-12-04 03:44 Upvotes 1
Hyperthreading has improved massively since when it first began. Hopefully Lukas can speak more to this because he understands the intricacies that describes why hyperthreading works better for engines now. Absolutely hyperthreading should be left on. With hyperthreading you are allowing your CPU(s) to work harder (meaning A LOT more positions searched). It's essentially an over-clock that comes with the cost of doubling of threads. Here is what Houdart wrote in his manual which has been updated. What should be noted is that he USED to recommend that hyperthreading be disabled:

"Hyper-threading
If your computer supports hyper-threading it is usually recommended not using more threads than physical cores. The additional hyper-threads will yield about 25% to 30% extra node speed, but the inefficiency of the parallel alpha-beta search with the higher number of threads will partially offset this speed gain. This means that the extra hyper-threads will produce only a small increase in Elo – probably at most 10 Elo.

To illustrate this, in a 12 vs 24 threads test match on a 12-core computer (Intel Xeon processor) the outcome after 1500 games was +7 ±10 Elo in favor of the 24-thread engine. Even with 1500 games played the measured Elo difference was still inside the error margin!

If your CPU can be overclocked it may actually be wiser not to use hyper-threading. By not using the hyper-threading you will reduce the thermal load of the CPU which will allow you to reach a higher overclocking frequency."

So you won't gain a tremendous amount of ELO with hyperthreading - but you should gain something. If you can over-clock your CPU then perhaps you should do that. But the idea that hyperthreading is "bad" is trash. I've done hundred and hundreds of games with my 36 core\72-thread rig; hyperthreading has always been great. I personally want as much of the power that I can get from my system. To me, disabling hyperthreading is just plain stupid; I would be doing nothing but crippling my system. I own Xeons so over-clocking my rig is out of the question.

Think of it this way. Hyperthreading is simply doubling of threads = more nodes per second (more positions searched because your CPU(s) can simply do more with more threads). However when you double the threads you INCREASE the inefficiency of the search. So it's simply a question of offsetting that inefficiency. Stockfish, according to Ipman, is yielding basically 30 percent more node speed which is massive. Houdart believes that search inefficiencies with doubling of threads is about 20 percent. Hence you should gain about 10 ELO - or something close to that.

Interestingly Vasik in this forum back in 2011 or so stated that in the near future hyperthreading would produce a small ELO gain eventually; he was correct. That man was smart.
Parent - - By Sesse (****) Date 2016-12-04 08:49
Leaving aside the yes/no question, there are some inaccuracies in your post:

First, hyperthreading is not equal to overclocking in any way. It increases instructions-per-cycle, not amount of cycles.

Second, when you double the number of threads, you don't increase the efficiency of the search, for any reasonable definition of efficiency—you decrease it. You spend more time synchronizing and searching wasted positions.

However, with SMP scaling improving (e.g. Stockfish switching to “lazy SMP”, which I find amusing that works at all :-) ), there might very well be a real strength gain now, but it's likely to depend on your thread count.
Parent - - By Zat (*) Date 2016-12-04 12:28
and if thread counts were in the range of 36/72 or 44/88? Would there be reasonable gains?
Parent - By Sesse (****) Date 2016-12-04 14:53
Impossible to say, you'll need to measure (by actual gameplay, not nps counts). It would certainly depend on the engine.

/* Steinar */
Parent - - By APassionforCriminalJustice (***) Date 2016-12-04 20:10 Edited 2016-12-04 20:14
Hi Sesse,

I said that hyperthreading increases the inefficiency of the search. And I don't care that hyperthreading doesn't meet your definition - or the definition of "over-clocking." The bottom line is that hyperthreading allows for the CPU(s) to do more work. If that means only more instructions per cycle then so be it. It's no different than large memory pages which is also a no-brainer if you want to try and get some more performance out of your system.

People can make their own choices of course. :)

Cheers,

I loved your work with respect to the WCC-live coverage.
Parent - By Sesse (****) Date 2016-12-04 22:34
Sorry, I missed the “in–” part.

And HT is very different from huge pages; using huge pages (assuming you can allocate them) is never a loss, whereas HT can be.

/* Steinar */
Parent - - By leavenfish (***) Date 2016-12-05 17:23
[url=]A Passion[/url] said...." That man was smart. "

Probably why he no longer wastes his time with chess engines as a 'commercial enterprise'. :wink:
Parent - By APassionforCriminalJustice (***) Date 2016-12-06 22:21
Haha, well what about Fritz? :)
Parent - - By Carl Bicknell (*****) Date 2016-12-04 21:59 Edited 2016-12-04 22:03
Based upon new testing of lazySMP here

http://www.talkchess.com/forum/viewtopic.php?t=62146

you should be able to work out the answer if you know what the gain in nps is when you switch on HT. (usually it's 0.6 x nps on each thread with twice as many threads in total now)

With HT switched OFF, and clock speed remaining constant, the speed gain of adding more cores is like this:

1 = 1
2 = 1.91
4 = 3.52
8 = 6.08
16 = 9.55
32 = 13.36
64 = 16.69
128 = 19.06

I will try and work out what that means for HT when I get some time. But even if it does produce a small gain on normal machines (<16 cores) I'd still switch it off, everything just works more predictably with it off.

Incidentally, as you can see the graph flattens (even with the superior Lazy SMP) so I would be surprised if going from 44 to 88 threads is a good thing by switching on HT.
Parent - By Zat (*) Date 2016-12-05 01:02
Thats what I had thought.  Thanks for an informative reply.  HT stays off for now.
Parent - - By CumnorChessClub (***) Date 2016-12-05 10:48
You can find tests here for HT on and off on a big Computer 72 Cores/144 Threads https://docs.google.com/spreadsheets/d/156Iztrz4erBxTntb6A9-4hCNAtLyazdBBh8zGy0RApE/edit#gid=0 for HT=on scroll down to number 352 (left hand side column)
Parent - - By Labyrinth (*****) Date 2016-12-05 11:16
Whose 8890 is that
Parent - - By RFK (Gold) Date 2016-12-07 05:22
This hobby is getting insanely obsessively techno neurotic!
Parent - - By Labyrinth (*****) Date 2016-12-07 15:11
At 100,000 kn/s it would only take 242 days, 18 hours, 48 minutes, and 30.04 seconds to completely analyse up to ply 11 from the starting position! White's sixth move! At 32 bytes per position one could store them all in just 67.12 petabytes! (The next step would be over 19 years though, and a couple exabytes..)
Parent - By RFK (Gold) Date 2016-12-08 03:49
I'm off on a new build -probably have it completed in few weeks. Nothing spectacular by comparison to the techno geeks here but will get me out of the laptop quagmire.
Parent - - By Sesse (****) Date 2016-12-08 08:42
This needless pruning of great opening lines such as e3,f6,Ke2,Kf7,Kd3,Ke6,Ke4,Nh6,Nc3,Kd6,Nce2 needs to stop!

/* Steinar */
Parent - By Antares (****) Date 2016-12-08 08:56
If you would add a persistant hash-feature to your analysis-baby, reloading it with every new game-init (and saving results when the game actually starts), we would find out about starting position more and more... instead of recalulating it again and again! (what a waste!)

(* Antares *)
Parent - - By Sesse (****) Date 2016-12-05 15:02
It always pains me when people buy a 72-core and then run Windows on it :-/

/* Steinar */
Parent - By Antares (****) Date 2016-12-05 15:15 Edited 2016-12-05 15:23
Because chess-wise it doesn't make too many difference! :-)

/* The Very Great Antares */
Parent - - By Lukas Cimiotti (Bronze) Date 2016-12-05 15:12
I'm sorry to say, but most of these calculations look totally wrong to me.
Parent - - By Carl Bicknell (*****) Date 2016-12-05 15:35
Feel free to reply to Kai...he's on a role.

Interestingly, if his model is right Lefler gives infinite cores as 22x faster than one which is rather depressing.
Parent - - By CumnorChessClub (***) Date 2016-12-05 16:36
Parent - By Carl Bicknell (*****) Date 2016-12-05 17:19
I thought he was referring to the calculations done by Kai on LazySMP.
Parent - - By Lukas Cimiotti (Bronze) Date 2016-12-05 17:23
This is referring to nps - and it makes clear the computer they use is not very efficient.
Parent - - By APassionforCriminalJustice (***) Date 2016-12-06 22:08
Could you elaborate? Why is their PC not very efficient?
Parent - - By Lukas Cimiotti (Bronze) Date 2016-12-07 07:00
If you try your own computer (HT and turbo off), you'll find a direct proportional increase of nps with cores - i.e. for 36 cores you'll get 36 times the value of one core. For their computer this is not the case. They have a 4 socket computer. My experience is: these computers are not really efficient. I've got a 4x Xeon E5-4650 computer (32 cores running at 2.9 GHz with turbo enabled). Its performance is very bad for Komodo, Stockfish and several other engines. Best results I get when using Houdini 5 and asmFish. But these results are still below expectations compared to a 2 socket computer. I assume this is a NUMA problem.

A different problem in their testing is: they didn't consider the Windows problem of making processor groups in case there are more than 64 cores. Houdini 5 and asmFish can handle this problem, Stockfish 8 can't.

The tests of Andreas Strangmüller look OK, but I don't know if he disabled Turbo. And there might be some other side effects caused by too fast timings for many cores or other tasks running.
Parent - - By APassionforCriminalJustice (***) Date 2016-12-07 19:44 Edited 2016-12-07 19:46
Ah I see. Yes I've heard that four-socket isn't that great for most chess engines. Stockfish development is now supposed to support more than one processor group but it's not exactly working correctly. About four threads out of 72 stay idle, and Stockfish doesn't seem to bind to each thread. Have you tried Cfish? It also has NUMA, and it works very well. Ronald keeps it up to date too.
Parent - - By RFK (Gold) Date 2016-12-08 03:23 Edited 2016-12-08 03:34
Interesting! They are now selling Intel Xeon E5 2686 V3  on ebay at $569.05! I might buy them and tweet The Donald with a pic just to piss him off! It would be pretty easy to put them in an inexpensive server mobo and lay it on top of a block of wood. I have an number of video cards. Don't even need audio! Couple sticks of ram! I'm good to play Techno insanity! :lol:
Parent - - By Zat (*) Date 2016-12-08 10:39
Those are engineering samples which will run markedly slower than say, the TCEC 9 machine.
Parent - - By RFK (Gold) Date 2016-12-08 15:41
Zat! :eek:You just dashed all  my hopes for joining the trendy Techno geek cult of insanity! :sad:

I even got my block of wood ready for the platform!
Parent - - By APassionforCriminalJustice (***) Date 2016-12-08 21:36 Edited 2016-12-08 21:38
I wouldn't listen to what Zat is saying. These should be qualification samples, not engineering samples. Moreover, there were two machines used in TCEC's tournament. The first one consisted of two 2630 v4s. Those suck (sarcastically speaking - but they are effectively low-tier Xeons), and the 2686 v3s are simply much better. The second machine consisted of two 2699 v4s. That's of course a no-brainer. So Zat's point, I don't get it. Lol. As per benchmarks, I got 3900+ in Cinebench, and 32, 250+ in Passmark CPU score. The chess performance is also GREAT. So if you want performance that will make you horny for a low cost - then there you have it.
Parent - - By Zat (*) Date 2016-12-08 22:33
Ok if you want to get personal.

I wouldn't take the advise of an individual who's only contribution to computer chess is *somehow* buying a computer.  Then getting kicked off of ICC for cheating and subsequently being the rudest individual ever encountered on that site (many people concur, including management after your 4 letter word spamming email to them).

Furthermore all of your "advise" is NOT based anywhere in reality or fact and you are repeatedly called out for this, yet this does not deter you.  Whatever social work you do there in toronto, stick to that, or go back to the warehouse stocking job.  This isn't your arena- it's like a baseball fan in stands thinking he knows how to coach the team.

AND finally if you actually view the screenshot those "QS" which you say are NOT engineering samples, clearly are coded "ES".  Now just sit down and watch the game please.
Parent - - By APassionforCriminalJustice (***) Date 2016-12-09 01:12
My advice helped both cma6, and Sedat, so shut your stupid little face.
Parent - By Zat (*) Date 2016-12-09 10:31 Edited 2016-12-09 11:25
Now that's the real Adam I wanted to show everyone, I knew you wouldn't disappoint.  :lol:
Parent - - By RFK (Gold) Date 2016-12-09 02:06
That cpu-z screen shot gives the show away. Now given that these two chips very well might be "engineering samples" which makes sense given their low price.  How do "engineering samples" enter into the market to begin with? I should think intel would melt these chips down as they might not quite meet up to intel specs! Or am I just guessing the answer here to a more black market situation?
Parent - - By Zat (*) Date 2016-12-09 10:29
The truth is that they are never supposed to be sold, but just try to impose law in China.  This is the *price* Intel pays for their cheapo testing centers in China.
Parent - - By RFK (Gold) Date 2016-12-09 14:36
Apparently, I went to intel's home page and read the write up. I'm surprised that ebay is allowing sales to take place on these items.  There can be some resistance on the part of the seller to return payment if there is a problem with the chip!
Parent - - By Lukas Cimiotti (Bronze) Date 2016-12-10 17:51 Edited 2016-12-10 17:55
There is a basic problem with Xeon E5 v2 CPUs. They can be used for upgrading older So. 2011-0 servers. So you won't get them at a reasonable price. You should prefer v3 or v4 Xeons - they have got better performance per clock speed. Avoid ES. They may have bugs, often have compatibility problems and turbo often doesn't work properly . QS are usually OK, but not perfectly legal. I prefer OEM CPUs, e.g. Xeon E5-2696 v3 / v4. The last pair of E5-2696 v3 I bought cost me ~€1800, a very reasonable price for 36 cores.

p.s. this should have been a reply to your post below - but my laptop made some strange things :wink:
Parent - - By RFK (Gold) Date 2016-12-10 21:58
I really am hesitant going through ebay. Reading reviews buyer end up with ES claiming bait and switch. Most if not many are coming from china and seem to be ES.
There seems to be no getting around the fact that if I want something reliable and decent I'm going to have to pay for it.
Best I can do is set a goal for the level Xeon E5 V3/ 4 Cpu's that are in my budget range and go for it. I think either two 8 cores or 10 cores would work  fine for me. I'm good with something less than earth shattering. :smile: I just have to no where to look!
Parent - - By Zat (*) Date 2016-12-10 22:13
Ok well, on a budget how about something like this then:
http://www.ebay.com/itm/HP-Z620-Workstation-2X-Dual-Xeon-E5-2690-2-9Ghz-Octo-8-Core-64GB-1Tb-Win-7-P-C1-/391647861755?hash=item5b30081bfb:g:CnsAAOSwa~BYNmFf

its not earth shattering but neither is the price.  You'd have 16 cores to work with and 2.9 Ghz isn't bad w/64GB ram.
Parent - By RFK (Gold) Date 2016-12-10 22:49 Edited 2016-12-10 22:52
One probably - I could buy those new for $ 400 each a pop! It is the old Sandy Bridge cpu's. Throw in a decent board and I already have the peripherals. I could build it cheaper.
Parent - - By RFK (Gold) Date 2016-12-09 15:06
What about buying a "refurbished" chip? Newegg is selling

Intel® Xeon® Processor E5-2680 v2 (25M Cache, 2.80 GHz)  Diversity IT  refurbished the chip or intel?

http://www.newegg.com/Product/Product.aspx?Item=9SIABRA50W2717&nm_mc=AFC-C8Junction-MKPL&cm_mmc=AFC-C8Junction-MKPL-_-Processors - Servers - Processor Servers-_-Intel-_-9SIABRA50W2717&cm_sp=&AID=11517614&PID=3962334&SID=

I could do two of these by summer!
Parent - By Zat (*) Date 2016-12-09 21:26
Shouldn't be any issues with refurbs.  Just make sure you stress test is asap and it has some form of return policy if it fails stress test.  Prime-95 is a reasonable
stress test method.
Parent - - By Lukas Cimiotti (Bronze) Date 2016-12-10 09:05
Please don't buy them at that price. There are much better offers. You should prefer Xeon E5 26** v3 (Haswell) or v4 (Broadwell).
Parent - By Zat (*) Date 2016-12-10 11:18
Oh I never even looked at the price, lukas is correct. Skip THAT purchase
Parent - By RFK (Gold) Date 2016-12-10 16:31
Appreciate the feedback! Thanks  for helping me narrow down the search.
Parent - - By Sesse (****) Date 2016-12-08 20:13
I believe they perform at their rated clock frequency (which is slower, but not necessarily that much slower), but highly likely to have bugs of various sorts.

/* Steinar */
Parent - By APassionforCriminalJustice (***) Date 2016-12-08 21:26
I'm not going to bother double checking but those 2686 v3s pictured above should be qualification samples. Mine actually are read by my system as the 2686 v3s. That does not qualify them as mere engineering samples. Highly likely to have bugs is dependent on their stepping.
Parent - - By RFK (Gold) Date 2016-12-08 22:21

> but highly likely to have bugs of various sorts.


Bad enough mobo will have bugs ... throw in  a buggy cpu? That would be a horse with out legs going no where! :eek:
Up Topic The Rybka Lounge / Computer Chess / Hyperthreading debate reopened?
1 2 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill