Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Move similarity between Hiarcs and Houdini
1 2 3 4 Previous Next  
- - By Venator (Silver) Date 2013-09-06 09:25
When playing through some Hiarcs games played in the WCCC 2013, with Houdini 1.5 running in the background, I noticed that many moves chosen by Hiarcs were also played by Houdini. Therefore I asked a friend, who has Houdini 2.0, to do a more in depth test. I also asked him to skip opening moves and simple ending moves, as there the move choices could be influenced by tablebases.

Here is the first game, Shredder-Hiarcs from round 4. Of 34 Hiarcs moves examined, Houdini 2.0 would pick 30 the same (88% similarity). When 6 forced captures are dismissed, we have 24 out of 28 moves the same, still an impressive 86% similarity.

[Site "Yokohama, Japan"]
[Date "2013.08.14"]
[Round "4.2"]
[White "Shredder"]
[Black "Hiarcs"]
[Result "0-1"]
[ECO "B90"]
[PlyCount "98"]
[EventDate "2013.??.??"]

1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. Be3 e5 7. Nb3 Be6 8. f3 h5 9. Qd2 Nbd7 10. O-O-O Be7 11. Kb1 b5 12. Qf2 O-O 13. g4 hxg4 {(Same as H2.0)} 14. h3 gxf3 {(Same as H2.0)} 15. Qxf3 g6 {(Same as H2.0)} 16. Rg1 Kg7 {(Same as H2.0)} 17. Nd5 Rh8 {(Same as H2.0)} 18. Bg5 Bxd5 {(Same as H2.0)} 19. exd5 Nxd5 {(H2.0 prefers 19... Nh7)} 20. Rxd5 Bxg5 {(Same as H2.0)} 21. Rxd6 Be7 {(Same as H2.0)} 22. Rc6 Nf6 {(H2.0 prefers 22... e4)} 23. Bd3 Qe8 {(Same as H2.0)} 24. Rc7 Rd8 {(Same as H2.0)} 25. Ra7 Rd5 {(Same as H2.0)} 26. Rf1 Rh4 {(Same as H2.0)} 27. Qg2 Rd6 {(H2.0 prefers 27... Rf4)} 28. Nc5 Nd5 {(First H2.0 prefers 28... Rf4, but at d25 switches to 28... Nd5)} 29. Qe2 Rf6 {(Same as H2.0)} 30. Rxf6 Bxf6 {(Same as H2.0)} 31. Be4 Nf4 {(Same as H2.0)} 32. Qf1 Qc8 {(Same as H2.0)} 33. Nd7 Be7 {(H2.0 first likes 33... Be7, but then switches to 33... Rxh3)} 34. a3 Bc5 {(Same as H2.0)} 35. Rb7 Bd4 {(Same as H2.0)} 36. c3 Bxc3 {(Same as H2.0)} 37. bxc3 Qxc3 {(Same as H2.0)} 38. Qf2 Rxh3 {(Same as H2.0)} 39. Qc5 Qb3+ {(Same as H2.0)} 40. Ka1 Qxa3+ {(Same as H2.0)} 41. Qxa3 Rxa3+ {(Same as H2.0)} 42. Kb2 Re3 {(Same as H2.0)} 43. Nc5 g5 {(Same as H2.0)} 44. Bf5 Kf6 {(Same as H2.0)} 45. Bc2 g4 {(Same as H2.0)} 46. Ne4+ Kg7 {(Same as H2.0)} 47. Ng5 Nd3+ {(H2.0 prefers 47... g3, could be that tablebases already taken over)} 48. Bxd3 Rxd3 49. Kc2 e4 0-1
Parent - - By Banned for Life (Gold) Date 2013-09-06 09:30
There is only one explanation. Robert must have plagiarized HIARCS to come up with Houdini! The ICGA should investigate! :lol:
Parent - By Venator (Silver) Date 2013-09-06 10:31
But..... Houdini never played in an ICGA tournament, now did it!? :wink:
Parent - - By Venator (Silver) Date 2013-09-06 09:30 Edited 2013-09-06 09:34
Here is the analysis of Jonny-Hiarcs from round 2. In total 41 Hiarcs moves examined, in 33 cases Houdini would pick the same (80% similarity). Excluding forced captures we get 31/39 = 79% similarity.

[Site "Yokohama, Japan"]
[Date "2013.08.13"]
[Round "2.1"]
[White "Jonny"]
[Black "Hiarcs"]
[Result "1-0"]
[ECO "A29"]
[PlyCount "133"]
[EventDate "2013.??.??"]

1. c4 e5 2. Nc3 Nf6 3. Nf3 Nc6 4. g3 d5 5. cxd5 Nxd5 6. Bg2 Nb6 7. O-O Be7 8. a3 O-O 9. b4 Be6 10. d3 f6 11. Ne4 Qd7 12. Qc2 a5 13. Nc5 Bxc5 14. bxc5 Nd5 15. Bb2 Nde7 16. d4 exd4 {(Until d25 H2.0 prefers 16... Nxd4, then switches to 16... exd4)} 17. Rad1 Bd5 {(H2.0 prefers 17... Bf5)} 18. Nxd4 Bxg2 {(Same as H2.0)} 19. Kxg2 Nxd4 {(Same as H2.0)} 20. Rxd4 Qc6+ {(Same as H2.0)} 21. f3 b5 {(H2.0 dislikes this move and prefers 21... Rfd8)} 22. Rfd1 Ng6 {(H2.0 prefers 22...a4)} 23. Qb3+ Kh8 {(Same as H2.0)} 24. Rd7 Rfe8 {(Same as H2.0)} 25. e4 Nf8 {(Same as H2.0)} 26. R7d5 Ne6 {(Same as H2.0)} 27. Qc2 b4 {(Same as H2.0)} 28. axb4 axb4 {(Same as H2.0)} 29. e5 fxe5 {(H2.0 prefers 29... f5)} 30. Rxe5 Nf8 {(H2.0 prefers 30... h6)} 31. Rg5 Ng6 {(Same as H2.0)} 32. Qc4 Re6 {(H2.0 prefers 32... Qe6)} 33. Rf5 h6 {(H2.0 prefers 33.... Rae8)} 34. h4 Qa6 {(Same as H2.0)} 35. Qxa6 Rexa6 {(Same as H2.0)} 36. Rd2 Rf8 {(Same as H2.0)} 37. Rfd5 Rf7 {(Same as H2.0)} 38. f4 Nf8 {(Same as H2.0)} 39. R2d4 Ra4 {(Same as H2.0)} 40. Kf3 Kg8 {(Same as H2.0)} 41. Re4 Kh7 {(Same as H2.0)} 42. Rd8 Ra5 {(Same as H2.0)} 43. Ree8 Kg8 {(Same as H2.0)} 44. Bd4 Ra3+ {(Same as H2.0)} 45. Be3 g5 {(Same as H2.0)} 46. Kg4 b3 {(Same as H2.0, that prefers 46... gxh4 47.Kxh4 b3, which is a transposition)} 47. Rb8 gxh4 {(Same as H2.0)} 48. Kxh4 Kh7 {(Same as H2.0)} 49. Kh5 Ra1 {(Same as H2.0)} 50. Bd4 Rd1 {(Same as H2.0)} 51. Be5 Rh1+ {(Same as H2.0)} 52. Kg4 Ng6 {(Same as H2.0)} 53. Rxb3 Re1 {(Same as H2.0, that prefers 53... Rd1 first, but at d27 switches to 53... Re1)} 54. Ra3 Rd1 {(H2.0 prefers 54... Re1)} 55. Re3 Rg1 {(Same as H2.0)} 56. Re6 Nxe5+ {
(Same as H2.0)} 57. R6xe5 Rg7+ 58. Kf5 Rf7+ 59. Ke6 Kg7 60. Kd5 Rd1+ 61. Kc6 Rf6+ 62. Re6 Rf8 63. Re7+ Kg6 64. Re8 Rf7 65. R3e7 Rxe7 66. Rxe7 Kh5 67. Re5+
1-0
Parent - - By Rebel (****) Date 2013-09-06 10:15
Here is the first game, Shredder-Hiarcs from round 4. Of 34 Hiarcs moves examined, Houdini 2.0 would pick 30 the same (88% similarity). When 6 forced captures are dismissed, we have 24 out of 28 moves the same, still an impressive 86% similarity.

Here is the analysis of Jonny-Hiarcs from round 2. In total 41 Hiarcs moves examined, in 33 cases Houdini would pick the same (80% similarity). Excluding forced captures we get 31/39 = 79% similarity.

For reasons of contrast you should do the same for Shredder (game-1) and Jonny (game-2). If the percentages are in the same high league then it's less suspect than if they are a lot lower.

Ultimately for more solid evidence one needs to use similarity tester from Don Dailey. Shall I ask Harvey for a copy of Hiarcs? :grin:
Parent - By Venator (Silver) Date 2013-09-06 10:22 Edited 2013-09-06 10:34
Shall I ask Harvey for a copy of Hiarcs?

Yup, but I suspect he would send you a Hiarcs version of 2005 or earlier :lol:
Parent - - By Venator (Silver) Date 2013-09-06 10:27
Of course this is only a very small sample, I realise that. But I have seen the trend for some years now and making it visible is the first step.

We could wait for the new commercial release of Hiarcs and do a much more extensive test. I suggest to pitch it against Houdini, and other Robbolito style programs.
Parent - - By Alkelele (****) Date 2013-09-06 10:44 Upvotes 1
Yes, it's kinda ironic.

Vas started with an eval based on a small set of well-known eval terms, along with some additional crucial and original eval and tuning ideas. Then he expanded hugely on that base along with Larry with many new eval ideas, I presume. For Rybka, it was a race towards new valuable knowledge, setting Rybka apart from everybody else.

Then Rybka got decompiled, and competitors took every Rybka idea that worked in their own programs, eval as well as search. For them, it was a race towards Rybka. Making everybody more similar.

Yet Rybka got banned for not being original, while everybody else are free to take advantage of all the things Rybka pioneered.
Parent - By Venator (Silver) Date 2013-09-06 10:55
Very well summarized, Dagh. I totally agree with you.

From a reliable source, who spoke with SMK about Shredder, I heard that SMK admitted to him that 'the only way I get Shredder much stronger, is taking things from Vas'.

Of course Uniacke is doing the same thing. And guess what, the Hiarcs folks claim it is original. A very bad joke.
Parent - - By Venator (Silver) Date 2013-09-06 11:02
Second game between Hiarcs and Shredder gives 15 moves identical with Houdini 2.0. Out of 17, this is 88%. Removing forced captures we get 12/14 = 86%.
Still, this example is less clear, as quite a lot of moves are rather obvious and probably a lot of other engines would pick them, too.

[Site "Yokohama, Japan"]
[Date "2013.08.17"]
[Round "9.1"]
[White "Hiarcs"]
[Black "Shredder"]
[Result "1-0"]
[ECO "B90"]
[PlyCount "74"]
[EventDate "2013.??.??"]

1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. Be3 e6 7. f3 Nc6 8. Qd2 Be7 9. O-O-O O-O 10. g4 Nxd4 11. Bxd4 b5 12. g5 Nd7 13. h4 b4 14. Ne2 Qa5 15. Kb1 Ne5 16. f4 Nc4 17. Qd3 e5 18. f5 Na3+ 19. bxa3 exd4 20. a4 Re8 21. Nf4 {(Same as H2.0)} Qe5 22. Nd5 {(Same as H2.0)} Bb7 23. Qxd4 {(Same as H2.0)} Bd8 24. Bc4 {(Same as H2.0)} Qxd4 25. Rxd4 {(Same as H2.0)} Kh8 26. Rh3 {(H2.0 prefers 26. Rf1)} Rb8 27. Rb3 {(Same as H2.0)} a5 28. a3 {(Same as H2.0)} Bc6 29. Nc3 {(Same as H2.0)} Bf6 30. gxf6 {(Same as H2.0)} bxc3 31. fxg7+ {(Same as H2.0)} Kxg7 32. Bd3 {(Same as H2.0)} h5 33. Kc1 {(Same as H2.0)} d5 34. Rxb8 {(Same as H2.0)} Rxb8 35. exd5 {(Same as H2.0)} Bd7 36. Bb5 {(H2.0 prefers 36. Rc4)} Bxb5 37. axb5 {(Same as H2.0)} Rxb5 1-0
Parent - - By Venator (Silver) Date 2013-09-06 11:10 Edited 2013-09-06 11:45
The first game between Hiarcs and Junior is a bit of a mess (regarding the notes of my friend), so I refrain from cleaning it up and just post the results:

40 Hiarcs moves to examine, 29 identical with Houdini 2.0. This is 72.5% similarity.

By the way, this is an interesting game with a positional piece sacrifice by Junior for long term compensation. Hiarcs realised too late it had to go for a draw, Houdini 2.0 didn't make that mistake.
Parent - - By Venator (Silver) Date 2013-09-06 12:31
Hiarcs vs Jonny from round 7: 15/18 = 83% similarity with Houdini 2.0. This is the kind of short, tactical struggle that is not a good example, though. I suspect many programs will play the same moves after 17. Nb5.

[Site "Yokohama, Japan"]
[Date "2013.08.15"]
[Round "7.3"]
[White "Hiarcs"]
[Black "Jonny"]
[Result "1/2-1/2"]
[ECO "E15"]
[PlyCount "60"]
[EventDate "2013.??.??"]

1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. g3 Bb4+ 5. Bd2 Be7 6. Bg2 c6 7. O-O d5 8. Nc3 O-O 9. Bf4 Nbd7 10. Rc1 dxc4 11. Nd2 {(Same as H2.0)} Nd5 12. Nxc4 {(Same as H2.0)} Nxf4 13. gxf4 {(Same as H2.0)} Bb7 14. e3 {(H2.0 prefers 14. Ne5)} Qc7 15. Ne5 {(Same as H2.0)} Nxe5 16. fxe5 {(Same as H2.0)} Rad8 17. Nb5 {(H2.0 prefers 17. Qa4 +0.00, 17. Nb5 gets the same score)} Qd7 18. Nxa7 {(Same as H2.0)} c5 19. Bxb7 {(Same as H2.0)} Qxb7 20. Nb5 {(Same as H2.0)} Qe4 21. h3 {(Same as H2.0)} cxd4 22. exd4 {(Same as H2.0)} Qg6+ 23. Kh2 {(H2.0 prefers 23. Qg4)} f6 24. Qb3 {(H2.0 prefers 24. Nc7 fxe5 25. Qb3, which is a transposition) } fxe5 25. Nc7 {(Same as H2.0)} Rf6 26. dxe5 {(Same as H2.0)} Rd3 27. Rc3 {(Same as H2.0)} Rxf2+ 28. Rxf2 {(Same as H2.0)} Qg3+ 29. Kh1 Qxh3+ 30. Kg1 Qg4+ 1/2-1/2
Parent - By Venator (Silver) Date 2013-09-06 15:40
Second game between Junior and Hiarcs: 46 Hiarcs moves examined, 36-38 are identical to Houdini 2.0 (there are two moves where Houdini switches back and forth to the same move Hiarcs played). This is a similarity of 78%-82%.

[Site "Yokohama, Japan"]
[Date "2013.08.16"]
[Round "8.2"]
[White "Junior"]
[Black "Hiarcs"]
[Result "1-0"]
[ECO "B90"]
[PlyCount "149"]
[EventDate "2013.??.??"]

1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. h3 e5 7. Nde2 h5 8. Bg5 Be7 9. Bxf6 Bxf6 10. Nd5 Bg5 11. Nec3 Be6 12. Bc4 Nc6 13. O-O Rc8 14. a4 g6 15. Ra3 O-O {(Same as H2.0)} 16. Bb3 Kg7 {(Same as H2.0)} 17. Qd3 Qa5 {(Same as H2.0)} 18. Qg3 Bd8 {(Same as H2.0)} 19. Qd3 Qc5 {(H2.0 switches between 19... Qc5 and 19... Nd4)} 20. Ne2 Na5 {(Same as H2.0)} 21. Nec3 Bh4 {(H2.0 prefers 21... Qd4)} 22. Ba2 Nc6 {(Same as H2.0)} 23. Qd2 Nd4 {(Same as H2.0)} 24. a5 Bd8 {(Same as H2.0)} 25. b4 Qc6 {(Same as H2.0)} 26. f4 Re8 {(H2.0 prefers 26... Qe8)} 27. f5 gxf5 {(Same as H2.0)} 28. exf5 Nxf5 {(Same as H2.0)} 29. Rf2 Ng3 {(Same as H2.0)} 30. Rf3 Nf5 {(Same as H2.0)} 31. Qd3 Nh4 {(Same as H2.0)} 32. Rf2 f5 {(Same as H2.0)} 33. g3 Ng6 {(Same as H2.0)} 34. Ne3 f4 {(Same as H2.0)} 35. Bxe6 Rxe6 36. Nf5+ Kh8 {(H2.0 prefers 36... Kg8)} 37. Qe2 Qd7 {(Same as H2.0)} 38. Qxh5+ Qh7 {(Same as H2.0)} 39. Qxh7+ Kxh7 40. Nd5 Bf6 {(H2.0 prefers 40... Bg5)} 41. Re2 Rc4 {(Same as H2.0)} 42. Kf1 Bg7 {(Same as H2.0)} 43. Rf2 Bh6 {(Same as H2.0)} 44. Rb3 Bf8 {(Same as H2.0, that first prefers 44... Bg7. On d25 it switches to 44... Bf8)} 45. Rbf3 e4 {(Same as H2.0)} 46. Nxf4 Nxf4 47. Rxf4 Rxb4 {(Same as H2.0)} 48. Rh4+ Kg8 {(Same as H2.0)} 49. Rg4+ Kh7 {(Same as H2.0)} 50. Rff4 Rg6 {(Same as H2.0)} 51. Rh4+ Kg8 {(Same as H2.0)} 52. Rxe4 Rb5 {(Same as H2.0)} 53. Rh5 d5 {(H2.0 prefers 53... Rf6)} 54. Reh4 Bc5 {(Same as H2.0)} 55. Rh8+ Kf7 56. R4h7+ Kf6 57. g4 Ke5 {(Same as H2.0)} 58. Rh5 Ke6 {(Same as H2.0)} 59. Rb8 Rf6 {(Same as H2.0)} 60. Re8+ Kf7 {(Same as H2.0)} 61. Re5 Bd6 {(Same as H2.0)} 62. Re2 Bf8 {(H2.0 switches between 62... Bf8 and 62... Bc5)} 63. Kg2 Kg8 {(H2.0 prefers 63... Rxa5)} 64. Re8 Rxa5 {(Same as H2.0, that switches between 64... Rb4 and 64... Rxa5)} 65. Nd4 b6 66. Rg5+ Kf7 67. Ra8 Be7 68. Ra7 Ra4 69. c3 Ra2+ 70. Kg3 Rd2 71. Nf5 Rxf5 72. Rxf5+ Ke6 73. Rxa6 Bd8 74. Rf3 Rd1 75. Ra8 1-0
Parent - - By user923005 (****) Date 2013-09-06 09:34
Adam Hair and others have done extensive automatic testing of this nature.

Now that the Rybka cat is out of the bag, all of the best programs magically rose 150 Elo for no apparent reason.
Since everyone has discussed all the clever goo in Strelka/Ippolit and friends, there is no way to keep the programmers from learning.
I don't blame them.  Once a good idea hits the wild, it spreads like wildfire.

Now, those who did the reverse engineering and then published the results, I feel there is a real problem there.  Imagine working for a couple years for your big release and a few months later, there is your stuff on the front page news.
I suspect that those doing the reverse engineering do not know that they are in violation of anything (and it is even possible that the laws are different in their countries, I do not know for sure.)

Anyway, if you go to Don's Komodo site, you will find a nifty-keen utility that will compare all sorts of engine pairs for you and tell you which ones "think alike".
Unfortunately, this information tends to be used for witch hunts by people who do not really know what the numbers mean (e.g. giving a program the cold shoulder for a similarity of 49, which will be found for some engine pair for almost any engine).

But why not save yourself some time, and read Adam Hair's analysis first.  Just google adam hair similarity and it will take you right to it, I am sure.
Parent - - By Venator (Silver) Date 2013-09-06 09:39 Edited 2013-09-06 09:41
Thanks for the tip! I have the "bad" habit, though, that I want to find out such things myself. In this forum I have said many times that the WCCC programs are starting to become like Robbo look-a-likes, well, here is such an example.

So far the ICGA quest about "original programming" :wink:

Also don't forget that the official public version might be different from the WCCC version. So it is interesting to examine the WCCC games.
Parent - - By user923005 (****) Date 2013-09-06 09:56
For an extremely tactical game, with lots of combinations, I expect many top engines will play an extremely similar path.
The quiet Botvinick games are where the engine differences will show up more.

When there is a pawn to snack on, somewhere just this side of the horizon, they will all find it.
But when you are going to gain the middle, open a rank, achieve superior battery or some other concept with a purpose that cannot bee seen in 10 moves, that is where you will see engines doing all sorts of different things.
Parent - - By Venator (Silver) Date 2013-09-06 10:05
No need to tell me, I have 30 years of experience in the chess computer world! I have worked with Ed, Richard Lang and Theron amongst others.

The above two games are rather positional, not tactical. Each move there are many possibilities to choose from, still we get 80% or more similarity. Doesn't the famous rule #2 say something about this?
Parent - - By user923005 (****) Date 2013-09-06 10:13 Upvotes 1
So that's how you got the shiny golden crown, and I only have three regular pawns and two ghost pawns.  I think someone killed them en passant, but they haven't realized that they are dead yet.
Parent - - By Venator (Silver) Date 2013-09-06 10:30
I am afraid it is a bit different: I got the golden crown because of my extensive discussions with Bob!
Parent - - By turbojuice1122 (Gold) Date 2013-09-06 11:38

> I got the golden crown


*ahem* Bronze crown :razz: :lol:
Parent - By Venator (Silver) Date 2013-09-06 11:41
Okay, fair enough! You can change it into a wooden one if you wish :wink:
Parent - By Venator (Silver) Date 2013-09-08 07:23
Now that the Rybka cat is out of the bag, all of the best programs magically rose 150 Elo for no apparent reason. Since everyone has discussed all the clever goo in Strelka/Ippolit and friends, there is no way to keep the programmers from learning. I don't blame them.  Once a good idea hits the wild, it spreads like wildfire.

That is obvious, but for me as a user of chess programs it is pointless to buy several chess programs that are playing nearly the same moves and have exactly the same style.

I still remember Fritz 11: I very much liked the old Fritz versions (10 and below) because of their aggressive and attacking playing style. Fritz 11 was a Rybka look-a-like, lost its former tactical power and became a positional player. Boring, I already had such a program which did it better. So I never bothered to buy any next Fritz version. It is the same with the rest of the Ippo/Robbo friends. Totally uninteresting for me.
Parent - - By nebulus (****) Date 2013-09-08 01:18
A quickie just to illustrate why the above is wrong. I'm sure one could make more precise analysis.

-------------+-----------+----------------------+---------------+---------------
             | Houdini 2 | Stockfish 4 (030913) | Stockfish 1.2 | Ruffian 1.0.5 
-------------+-----------+----------------------+---------------+---------------
13. g4       | h3        | h3                   | h4            | Nd5           
13... hxg4   | same      | same                 | same          | same          
14. h3       | same      | same                 | same          | same          
14... gxf3   | same      | same                 | same          | same          
15. Qxf3     | same      | same                 | same          | same          
15... g6     | same      | same                 | same          | b4            
16. Rg1      | Nd5       | same                 | same          | Bh6           
16... Kg7    | same      | same                 | same          | Re8           
17. Nd5      | same      | same                 | same          | same          
17... Rh8    | same      | same                 | Bxd5          | Bxd5          
18. Bg5      | same      | same                 | Nxe7          | Nxe7          
18... Bxd5   | same      | same                 | same          | same          
19. exd5     | same      | same                 | same          | same          
19... Nxd5   | Rh7       | Rh7                  | Kg8           | Rh5           
20. Rxd5     | same      | same                 | same          | same          
20... Bxg5   | same      | same                 | same          | same          
21. Rxd6     | same      | same                 | same          | same          
21... Be7    | same      | same                 | same          | same          
22. Rc6      | same      | same                 | same          | Rd1           
22... Nf6    | e4        | Bf6                  | Rh4           | same          
23. Bd3      | same      | same                 | same          | same          
23... Qe8    | same      | same                 | same          | Rh6           
24. Rc7      | same      | same                 | same          | same          
24... Rd8    | same      | same                 | same          | same          
25. Ra7      | same      | same                 | same          | same          
25... Rd5    | same      | same                 | same          | same          
26. Rf1      | a3        | same                 | same          | Be4           
26... Rh4    | same      | same                 | same          | same          
27. Qg2      | Qg3       | Qg3                  | same          | same          
27... Rd6    | Rf4       | Rf4                  | Rf4           | Rf4           
28. Nc5      | same      | same                 | same          | same          
28... Nd5    | Rf4       | Rf4 -> Nd5           | same          | same          
29. Qe2      | same      | same                 | same          | same          
29... Rf6    | same      | same                 | same          | same          
30. Rxf6     | same      | same                 | same          | same          
30... Bxf6   | same      | same                 | same          | same          
31. Be4      | same      | same                 | same          | same          
31... Nf4    | same      | same                 | same          | same          
32. Qf1      | same      | same                 | same          | same          
32... Qc8    | same      | same                 | Rxh3          | Rxh3          
33. Nd7      | same      | same                 | Qg1           | same          
33... Be7    | Rxh3      | Rxh3                 | same          | Rxh3          
34. a3       | same      | same                 | same          | same          
34... Bc5    | same      | same                 | same          | same          
35. Rb7      | same      | same                 | same          | Nxc5          
35... Bd4    | same      | same                 | same          | same          
36. c3       | same      | same                 | a4            | a4            
36... Bxc3   | same      | same                 | same          | same          
37. bxc3     | same      | same                 | same          | same          
37... Qxc3   | same      | same                 | same          | same          
38. Qf2      | same      | same                 | same          | same          
38... Rxh3   | same      | same                 | same          | same          
39. Qc5      | same      | same                 | same          | same          
39... Qb3+   | same      | same                 | same          | same          
40. Ka1      | same      | same                 | same          | same          
40... Qxa3+  | same      | same                 | same          | same          
41. Qxa3     | same      | same                 | same          | same          
41... Rxa3+  | same      | same                 | same          | same          
42. Kb2      | same      | same                 | same          | same          
42... Re3    | same      | same                 | same          | same          
43. Nc5      | same      | same                 | same          | same          
43... g5     | same      | same                 | same          | Re2+          
44. Bf5      | same      | same                 | same          | same          
44... Kf6    | same      | same                 | same          | same          
45. Bc2      | Bd7       | Bg4                  | Bg4           | Bg4           
45... g4     | same      | same                 | same          | same          
46. Ne4+     | same      | same                 | same          | same          
46... Kg7    | same      | same                 | same          | same          
47. Ng5      | Re7       | Nd6                  | Nd6           | Nd6           
47... Nd3+   | g3        | g3                   | g3            | g3            
48. Bxd3     | same      | same                 | same          | same          
48... Rxd3   | same      | same                 | same          | same          
49. Kc2      | same      | same                 | same          | same          
49... e4     | same      | Ra3                  | Rd5           | Ra3           


And suddenly those percentages do not look too suspicious.
Parent - By user923005 (****) Date 2013-09-08 03:55
And in highly tactical positions, they all think alike.
Most top engines will choose the same best moves 295 out of 300 times or more with WAC.EPD ( 98% ) and 299 out of 300 will happen more than once if you use a big collection of top engines and give them all at least 5 seconds per position.
That's the reason that the CCRL ponder hit test is useful.  It averages out over hundreds of thousands of positions.
Parent - - By Venator (Silver) Date 2013-09-08 07:15 Edited 2013-09-08 07:48
The above is not "wrong", it is simply statistical information. By the way, it would be interesting to include the previous commercial Hiarcs versions in this test.

Some more to consider:

1. The phrase "playing nearly the same moves" is still in ICGA rule number 2, so it can be used as a valid argument to have an opponent program examined.

2. In CSVN tournaments programs showing such high similarities can be refused, as the CSVN has a similarity test.
Parent - - By nebulus (****) Date 2013-09-08 13:05
Okay, I'll give you that. Wrong was probably not the word I should have used. I'll correct myself: A quickie just to illustrate why the above is misleading.
Parent - - By Venator (Silver) Date 2013-09-08 15:15
What is misleading about move statistics and the two rules (CSVN, ICGA) I gave you?
Parent - - By nebulus (****) Date 2013-09-08 15:34
It was not about the rules. You said: "The above is not "wrong", it is simply statistical information.", referring to my "A quickie just to illustrate why the above is wrong." I agreed "wrong" was a wrong word and said I should have used "misleading" instead. Hope this clarifies it for you.
Parent - - By Venator (Silver) Date 2013-09-08 15:39
So according to you, statistical information is misleading?

Your little inquiry is also badly chosen, as Turbojuice already pointed out. Only one game, using (apart from Ruffian) programs from the same source group. That is hardly a significant test. So you perform a one game test and call something misleading..... ;-).

Fact remains, that the 80% similarity can pose a problem for Hiarcs in CSVN and ICGA tournaments.
Parent - - By nebulus (****) Date 2013-09-08 16:42

> So according to you, statistical information is misleading?


Of course it is misleading.

Let me give you an example: Rybka 4.1 - Hiarcs 13.1: = = 0 = = (40% against Hiarcs)

Is this misleading? Of course! And so is your analysis.

> Your little inquiry is also badly chosen, as Turbojuice already pointed out. Only one game, using (apart from Ruffian) programs from the same source group. That is hardly a significant test. So you perform a one game test and call something misleading..... ;-).


I don't think it was. Stockfish is supposed to be a different beast, but it chooses the same moves. My point was not to disprove anything you said, but simply to illustrate the flaws in your methodology. Therefore, the statistical information you present is misleading.
Parent - - By Venator (Silver) Date 2013-09-08 16:57
I strongly disagree. There is no 'analysis'. Just 5 WCCC 2013 games played by Hiarcs, with a reported move similarity.

Let me give you an example: Rybka 4.1 - Hiarcs 13.1: = = 0 = = (40% against Hiarcs). Is this misleading? Of course!

Of course not. You have played 5 games and Hiarcs won that mini match 2-3. No analysis whatsoever and the score is correct (unless you made it up). It would be misleading if you would write 'see, Hiarcs is clearly better than Rybka 4.1, it scores 60%', without mentioning the number of games.

My point was not to disprove anything you said, but simply to illustrate the flaws in your methodology. Therefore, the statistical information you present is misleading.

You use just one game (less than my sample) to say that something is misleading. Funny.

Read the thread again and also my reaction to Ed. I have never claimed 'see, Hiarcs always has a 80% move similarity with Houdini 2.0'. Besides, there are only a few WCCC 2013 games played by the current Hiarcs version.
Parent - - By nebulus (****) Date 2013-09-09 06:32
Let's see... you reply to a discussion related to originality, post a link to this thread and then ask me what my take is on this (here's your post btw). And ask: "Original engine, yes or no? Violation of rule #2, yes or no?" 

It turns out there're only 5 games, the results show 80-something% similarity, comparison is done only to Houdini and only Hiarcs' moves are compared, and you're perfectly aware that the data has no statistical significance, since the number of games is order of magnitude lower than it should be. If one doesn't look any further one can certainly make a premature assumption that something is wrong. Not to mention that you wanted a yes or no answer on something that had no data to support a yes or no answer. And you don't see how it is misleading?
Parent - - By turbojuice1122 (Gold) Date 2013-09-09 10:40
In the past, such a low number of games has been enough for an investigation.
Parent - By Ray (****) Date 2013-09-09 12:23
And it would be enough for an investigation here too. But because Mark is one of the "old boys", one of the select club that can do no wrong, you can guarantee that the ICGA would find an excuse not to investigate.
Parent - - By Venator (Silver) Date 2013-09-09 15:01
1. The ICGA found a 5 game comparison between Quickstep and Mephisto Almeria enough to have Langer's source code examined. So your comment is bogus.

2. The example you give, about a 5 game match between Hiarcs and Rybka being misleading, is also total nonsense.

3. You use one game to prove your point and say something about statistics.

Conclusion: you are a troll and you go on ignore.
Parent - By nebulus (****) Date 2013-09-09 18:29
Someone replies this Rybka - Hiarcs statistics to the discussion on which engine is the strongest, fully knowing that it does not represent the whole picture, and it wouldn't be considered misleading? Oh, well... :roll:

> You use one game to prove your point and say something about statistics.


Don't worry, I'm going through all the games and post the results later.

> Conclusion: you are a troll and you go on ignore.


How mature... nonetheless, we live in a free world and you're free to do as you wish. If you wish to put someone critical to your view on ignore, so be it.
Parent - - By turbojuice1122 (Gold) Date 2013-09-08 10:26
The game Hiarcs vs. Jonny is more quiet, so one should expect some noticeable differences there (but it had an 80% match).  It would still be interesting to see what the other programs say, though.  Anyway, this is a phenomenon that I noticed before: Hiarcs 13 seemed to become more Rybka-like and Houdini-like in both its playing style and in its search.  This could easily be due to seeing the Robo code and the disassembled Rybka code and making lots of notes (which is fine according to me, but not according to the ICGA, apparently), but has taken the uniqueness out of Hiarcs, so that is a reason (the other being the ICGA scandal) that this was the last version of the program that I have or plan to purchase.
Parent - - By Ugh (*****) Date 2013-09-08 10:46
Slight correction! I think the take notes and develop own implementation would be ok for the ICGA, it's only Hyatt that opposes this (for Vas, but not for himself) because his "copy-modify" and "non-literal copy" don't work for him anymore, so he falls back on an (demonstrably untenable) "notes=copy" defence, rather than fess up. Cowardly, but the guilty usually plead not-guilty until the bitter end (something else he projected onto Vas of course).

The ICGA, and the panel voters in general, believed Hyatt's "copy-modify" and "non-literal copy" mantras because the panel was set up in such a way that there was no effective criticism available. Also set up such that there was no criticism of the panel bias structure. Since the technical criticisms are pretty damn obvious to anyone reading the ICGA attack documents in depth, the bias criticisms are there for all to see, and some voters had a distinct and obvious interest in convicting Vas; one can only wonder at the non-technical (humanitarian, social, psychological, historical) education that these computer engineers can possibly have had. None? Where was their Oppenheimer? Now, there's a role for Zach Wegner.
Parent - - By bob (Gold) Date 2013-09-08 14:44
There are several ways to "copy non-literally".

1.  Copy and translate, say mailbox to bitboards.  The copy won't be a literal copy, but it will play exactly the same until additional changes are made.

2.  take a source program, make detailed notes, then write your own following those notes carefully.  That is NOT original, and is also a "non-literal copy".

#2 is no better or worse than #1.  Same final result.

For the rest, no comment is needed.  Just ranting nonsense..
Parent - By Venator (Silver) Date 2013-09-08 15:44
'Non-literal copy' is just a trick you needed when you found out there was no copied code to prove. This nonsense has now aggravated to 'taking detailed notes', which is one step away from 'he studied code, memorized it and that is code copying'. I bet only your wife would support you on this one.

The rest is just ranting nonsense, but you have to keep this story going, otherwise the damage will be beyond control.
Parent - - By turbojuice1122 (Gold) Date 2013-09-08 12:35
I just realized something--you can't use Stockfish as a 'control group' because both it and Houdini obtain so much of their methods from the same source.
Parent - - By Venator (Silver) Date 2013-09-08 15:21
Agreed. All programs from the same control group will give similar results. One should test f.e. with older Hiarcs versions, that would be interesting.
Parent - - By nebulus (****) Date 2013-09-08 15:41
I don't have Hiarcs, nor do I have Shredder. Stockfish was chosen because it supposed to be rather different, at least with respect to this dendrogram.
Parent - - By Venator (Silver) Date 2013-09-08 15:55
I understand. But it still would be interesting if someone would run the tests using older Hiarcs versions. As Turbojuice noted (and I agree), Hiarcs clearly has made a move towards the Robbolito family in the past few years.
Parent - By nebulus (****) Date 2013-09-08 16:43
Yes, it would indeed be interesting to see the moves compared to an older Hiarcs. Shredder as well.
Parent - - By Peter Grayson (****) Date 2013-09-09 08:10
HIARCS 13.2 MP  released prior to Houdini 1.5a would play exact same first 20 moves as the tournament HIARCS engine. At move 33 ..Be7 it chose 33.. Rxh3 immediately and had not changed after 150s. So the tournament HIARCS engine played very similarly to the 13.2 version not Houdini. The first HIARCS 13.2 MP deviation at move 33..Rxh3 would have been selected by Houdini 2.0c Pro after initially selecting the game move. This confirms that the tournament HIARCS engine is a progressive developement of earlier HIARCS enngines.

Houdini 2.0c Pro did not select 15..g6 immediately, initially preferring 15..Rc8 changing to 15..g6 after 45s, with search sequence similar to HIARCS. Moving the pawn to allow space for the king to move off the back rank allowing a rook to go onto to the semi-open 'h' file is a plan that looks quite sensible and nothing out of the ordinairy here.

Given that Houdini would have changed the game so many times to HIARCS actual moves your accusation has no foundation and does nothing for the credibility of the VII supporters.

PeterG
Parent - - By Labyrinth (*****) Date 2013-09-09 10:43
That's the longest link I've ever seen!
Parent - By turbojuice1122 (Gold) Date 2013-09-09 11:11
Unfortunately, the goo.gl URL shortener won't allow shortening of such very long links--I tried to do this with my own, and this didn't work, so I had to take a screenshot instead.
Parent - By Akbarfan (***) Date 2013-09-09 22:57
me too :lol::lol::lol:
Up Topic Rybka Support & Discussion / Rybka Discussion / Move similarity between Hiarcs and Houdini
1 2 3 4 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill