Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Opening Books / Fauzi Gauntlet III
- - By Dark_wizzie (***) [us] Date 2014-10-13 07:31 Edited 2014-10-13 08:36
It's time for Fauzi Gauntlet III. Roster subjected to change at any time. This time Polyglot books are added. I try to focus on the books that did well last time to see if they can continue their good scores. The scores are all very close to each other, and the playing field is much stronger this time than the last two gauntlets. It will continue to get stronger in future gauntlets as I figure out which books are stronger than others against Fauzi. Once again, this gauntlet is designed for book strengthening purposes, not to assemble a rating list. But if you think it's a good show, feel free to tag along.

All of my public tests have been assembled here:
http://www.autochess.com/forum/dark-wizzie-s-testing-thread-t2169.html

    Engine                  Score                                                                                                                                                        
01: K8 Fauzi3.3             316.0/600 ······················································································································································
02: K8 Poly-OopsFIM         28.0/50   =========1===============1======11==1==1==========                                                                                                    
03: K8 Needle 5.3.0         26.0/50   =1======1===================0============1========                                                                                                    
04: K8 Poly-Stormworm       25.0/50   ====================0================1==1========0                                                                                                    
04: K8 Baracuda2014c        25.0/50   =====1======0===0=========11=======0==============                                                                                                    
06: K8 Poly-Rodent          24.5/50   ==0==================0=======1=======1====0=======                                                                                                    
07: K8 Poly-Mohammad Qhtane 24.0/50   ===============0=0===========0=1========0=1=======                                                                                                    
08: K8 Perfect2014FinalT    23.5/50   =====0========0====1==========0=======0==01=====01                                                                                                    
09: K8 Poly-Myfriends8      22.0/50   =====00==============0==============0=0=======0===                                                                                                    
10: K8 Poly-SFRW            21.5/50   0=======0=======0======1====0=0==1==0=0===0===0===                                                                                                    
10: K8 Poly-Titans          21.5/50   ======0==0=1010=========0=====0===0=======0===0===                                                                                                    
10: K8 Zandokan1            21.5/50   ==0=============0=0=====0===0===========0=0=======                                                                                                    
10: K8 Poly-Small           21.5/50   ======0=0===0===0=0=======0===================0===                                                                                                    
14: K8 Balanced-14          0.0/0                                                                                                                                                           
14: K8 HiarcsBook           0.0/0                                                                                                                                                           
14: K8 s-spitaleri-Traps    0.0/0                                                                                                                                                           
14: K8 CTbook               0.0/0                                                                                                                                                           
14: K8 fQuisinskyv03        0.0/0                                                                                                                                                           
14: K8 KingFighter          0.0/0                                                                                                                                                           
14: K8 Libra8               0.0/0                                                                                                                                                           
14: K8 T-X                  0.0/0                                                                                                                                                           
14: K8 Tomcat               0.0/0                                                                                                                                                           

SETTINGS:
15 minutes to play 75 moves.
4670k @ 4.4-4.5 ghz [Scored 8434 in Sedat's benchmark @ 4.5ghz found here: http://www.sedatcanbaz.com/chess/?page_id=19 ]
2048mb Hash
Games after 75 moves automatically drawn
Games after 41 moves with 0.00 eval on both sides automatically drawn
Games with both side's eval at +2.0 for one side is automatically adjudicated as a win
Some games are subjected to my manual adjudication
No endgame tablebases

All games are 15 minutes or less because time control is 15 minutes every 75 moves and after 75 moves, games are auto-drawn. This setup should in theory force engines to use more of their time than a normal 15 minute game.

This gauntlet will take several months to complete, and it will be much larger than both previous gauntlets.
Parent - By Dark_wizzie (***) [us] Date 2014-10-23 05:41
Engine                  Score                                                                                                                                                          K8
01: K8 Fauzi3.3             596.5/1100 ······················································································································································
02: K8 Poly-OopsFIM         28.0/50    =========1===============1======11==1==1==========                                                                                                    
03: K8 Needle 5.3.0         26.0/50    =1======1===================0============1========                                                                                                    
04: K8 Poly-Stormworm       25.0/50    ====================0================1==1========0                                                                                                    
04: K8 Baracuda2014c        25.0/50    =====1======0===0=========11=======0==============                                                                                                    
04: K8 Balanced-14          25.0/50    1======1==========0==00=====0======1======1=======                                                                                                    
04: K8 KingFighter          25.0/50    0=0==1=1========1====1=1====0=0=0=========01======                                                                                                    
08: K8 Poly-Rodent          24.5/50    ==0==================0=======1=======1====0=======                                                                                                    
09: K8 CTBook               24.0/50    0===========0=====0============1============0101==                                                                                                    
09: K8 Poly-Mohammad Qhtane 24.0/50    ===============0=0===========0=1========0=1=======                                                                                                    
11: K8 Perfect2014FinalT    23.5/50    =====0========0====1==========0=======0==01=====01                                                                                                    
12: K8 HiarcsBook           22.5/50    0=0========0==========0=========0=================                                                                                                    
12: K8 Kingliveson          22.5/50    =0==============0==1==========0===0=====0=====0=01                                                                                                    
14: K8 Tomcat               22.0/50    0=====0=0====10===============0=====0=0===========                                                                                                    
14: K8 Poly-Myfriends8      22.0/50    =====00==============0==============0=0=======0===                                                                                                    
16: K8 Zandokan1            21.5/50    ==0=============0=0=====0===0===========0=0=======                                                                                                    
16: K8 Poly-Small           21.5/50    ======0=0===0===0=0=======0===================0===                                                                                                    
16: K8 Libra8               21.5/50    0====0==0===========================00===00====01=                                                                                                    
16: K8 Poly-SFRW            21.5/50    0=======0=======0======1====0=0==1==0=0===0===0===                                                                                                    
16: K8 Poly-Titans          21.5/50    ======0==0=1010=========0=====0===0=======0===0===                                                                                                    
21: K8 Spitaleri-Traps      21.0/50    0===0===0=0===01======0===0==10==00===0=====1=====                                                                                                    
22: K8 T-X                  19.5/50    ============00=========0==0===0=0=0=0=01==0=0=0===                                                                                                    
23: K8 fQuinsinskyv03       16.5/50    =00=0=0==00=0===00==00=======001==0=0==01==00==0==
Parent - By Dark_wizzie (***) [us] Date 2014-10-23 12:01 Edited 2014-10-23 14:00
I have now assembled pgns for 4200 games from Fauzi Gauntlet II and what have been done so far in Gauntlet III plus a few games in between the two gauntlets. The results include Komodo and SF games. This rating list is far superior to the rating list I made for the last gauntlet because this one includes far more games. This has taken me hours to construct.
Parent - - By Dark_wizzie (***) [us] Date 2014-11-11 05:55 Edited 2014-11-12 08:37
-Fauzi got updated to 3.4
-Barracuda got an update too after about 75 games
-MyFriends got an incremental update
-Now using 5 man EGTB Syzygy TB on SSD, will move onto 6 man soon
-Various small tweaks to make the entire operation run smoother, including putting entire folder on SSD to shorten game loading times, book move time, tweaks to how to convert pgn into a unified rating list. This goes a long way to making assembling the rating list less of a PITA.
-I'd like to highlight the fact that the games in this Unified Rating List (ok, I'll just call it URL from now on) is from pieced together games from various tests. This means the engine used may be K8 or the latest SF beta at the time. Some books are stronger with SF and some are stronger with K. To fairly portray the strengths of each book I will be moving back to SF in the future. However I'm not sure when I will since the SF with Syzygy support is currently out of date.
-And once again I'd like to point out that unlike every other opening book rating list with a high number of games, the time control here remains around 10-15 minutes per side. This along with half-decent hardware means we are talking about a decent level of chess, not quick blitz games. This makes the URL unique.
-I intend to do some games centered around fleshing out this rating list. This includes adding other books I used to not test after it showed poor results. Also I will include some other new books I've never tested. The new 1337Free is doing very well, which popped up on my list of things to test recently. Despite putting MrQ3 and KingLiveson back for testing, they showed poor results. After this is all done, I intend to test the top books more, ignoring the lower-rated books, and for the very top of the chart I will be testing with SF and K, alternating often. Probably a good 500 games dedicated for this alone. Fauzi Gauntlet III will be truely monstrous!

         Engine                  Score                                                                                                                                                           K8
01: K8 Fauzi3.3             1219.5/2250 ······················································································································································
02: K8 Poly-OopsFIM         55.5/100    =========1===============1======11==1==1==========00=1=====1==01==01=1=====1=101=1=1================                                                  
03: K8 Needle 5.3.0         51.0/100    =1======1===================0============1=============0==01===101===1==0==1======0=================                                                  
04: K8 KingFighter          49.5/100    0=0==1=1========1====1=1====0=0=0=========01======1=====010==1========0===0=============0=1=======01                                                  
05: K8 Poly-Rodent          48.5/100    ==0==================0=======1=======1====0===============0===0==============1===============1==0=0=                                                  
05: K8 Poly-Mohammad Qhtane 48.5/100    ===============0=0===========0=1========0=1=======01=======1=======1========0======0==0=============                                                  
05: K8 Poly-1337Free        48.5/100    ======0=0===========0=====0=0===========0============1=====1==0==================1===1======0======1                                                  
08: K8 Perfect2014FinalT    47.5/100    =====0========0====1==========0=======0==01=====01======0========1==0=======0===01====0======1======                                                  
08: K8 Poly-Myfriends8      47.5/100    =====00==============0==============0=0=======0==========1=1====0=0==1010=====0===0====1===1=======1                                                  
10: K8 Balanced-14          47.0/100    1======1==========0==00=====0======1======1==========1==0===0===0=0===========0==0==========0=======                                                  
10: K8 HiarcsBook           47.0/100    0=0========0==========0=========0=================1==10===0====10===0=====0==1=====1====0=0=01=1====                                                  
10: K8 Poly-Stormworm       47.0/100    ====================0================1==1========00==0=======0==0===0=========1===0=====0===========                                                  
10: K8 Baracuda2014c        47.0/100    =====1======0===0=========11=======0==============0===0=============0===========0=====0=0===========                                                  
14: K8 CTBook               45.5/100    0===========0=====0============1============0101======0===00=1==00=====0====0=========0=============                                                  
15: K8 Tomcat               44.5/100    0=====0=0====10===============0=====0=0=============0===0=0=========================0====0==========                                                  
16: K8 Poly-Titans          44.0/100    ======0==0=1010=========0=====0===0=======0===0===========0=====0==1==0=====0==========0==========0=                                                  
17: K8 Zandokan1            43.5/100    ==0=============0=0=====0===0===========0=0============10=======0=============0===0=0=0=0=0=0==1=1==                                                  
18: K8 MrQ3                 43.0/100    ======0==0==0==00=====0===0==1====0=1=====0=======0===============0=====0=0=010==00=============1=0=                                                  
18: K8 Poly-SFRW            43.0/100    0=======0=======0======1====0=0==1==0=0===0===0===0=0=======0=====0===0====0=================0======                                                  
20: K8 Spitaleri-Traps      42.5/100    0===0===0=0===01======0===0==10==00===0=====1===============0=======0=====0===========0=0=====0=0===                                                  
21: K8 Kingliveson          42.0/100    =0==============0==1==========0===0=====0=====0=01======0=====0=0========0=1====0==0====0=000=00====                                                  
22: K8 Poly-Small           40.5/100    ======0=0===0===0=0=======0===================0===0=00==0=0===0=====1====0==0=0=0===0===00===1===0==                                                  
23: K8 Libra8               21.5/50     0====0==0===========================00===00====01=                                                                                                    
24: K8 T-X                  19.5/50     ============00=========0==0===0=0=0=0=01==0=0=0===                                                                                                    
25: K8 fQuinsinskyv03       16.5/50     =00=0=0==00=0===00==00=======001==0=0==01==00==0==
Parent - - By Dark_wizzie (***) [us] Date 2014-11-12 08:24
I have now upgraded my elo calculation to Ordo latest beta build, using a whopping overkill 1 million iterations for accurate elo stats. On top of this, LOS can now be easily calculated and the rating is now anchored around Fauzi book at 0 elo for easy comparison.
Parent - - By fauzi (**) [jo] Date 2014-11-12 19:05
Hi Eric,
Thanks for everything.
What does white advantage 40.99 means?
Parent - - By Fulcrum2000 (****) [nl] Date 2014-11-12 19:17
I suspect playing white gives you a 'free' 41 ELO advantage (which is a LOT)
Parent - By Dark_wizzie (***) [us] Date 2014-11-12 19:33 Edited 2014-11-12 20:18
Fulcrum is correct. Almost all of the wins are with white and most games are drawn. A loss with white is a big blow for the book's standings.
Parent - By Dark_wizzie (***) [us] Date 2014-11-28 12:16 Edited 2014-11-28 12:35
-EGTB updated to 6man Syzygy
-Fauzi updated to 3.5
-I've also fixed a small error in my large PGN file
-Some games are over the 15 minute time control because I manually adjusted it to be that way. This is because my computer was taking up a core to do something else. The extra time I add always overcompensates for the computing power loss, so don't worry.



Very good score for for Bigdaddy, considering it plays openings like the Fried Fox - Pork Chop opening shown here:


Updated rating list (Ordo w/ 1,000,000 iternations).


Now I look forward to testing the top books more often - mostly with Stockfish now instead of K8. Syzygy SF is finally released by default so that is great news. I'm hitting 5,000 games this gauntlet alone minimum.
Don't forget, during Gauntlet IV I fully intend to start testing CTG books! For example, the paid Hiarcs book is begging to be tested!
Gauntlet V will have extra long time controls to spice things up.
After Gauntlet V comes ABK Book Bash IV! (Opening book tournament instead of a testing-style gauntlet.)
Parent - - By Dark_wizzie (***) [us] Date 2014-12-22 11:12 Edited 2014-12-22 12:50
New update. Why did it take so long? Just cuz'.

-Fauzi updated to 3.6.

   Engine          Score                                                                                                                                                                                                                                                                       
1: K8 Poly-OopsFIM 136.0/261
2: K8 Fauzi3.3     125.0/261

These 261 games include the 50 already done previously in this gauntlet.
The following standings are seperate from the previous games because these are with SF.

   Engine                               Score                                                                                                                                                                                                                                                                        SF
1: SF Fauzi3.3                          427.5/800 ······································································································································································································································································
2: SF Poly-Mohammad Qhtane              49.0/100  =======1=====1==================0==10==10===============0=======1=======01==01====0===0=0===========                                                                                                                                                                  
2: SF Needle 5.3.0                      49.0/100  0=1=1===0=1=============================================0==0====1==========1=============00===0=====                                                                                                                                                                  
2: SF Poly-SandWorm                     49.0/100  ===1======0===================01========0===01==0=0====1========1======1=================0=======0==                                                                                                                                                                  
5: SF Poly-OopsFIM                      48.0/100  0=0=====================1==10=========0=============0===010====1==01=0===10====1==0=1==10=0=====0==1                                                                                                                                                                  
5: SF Poly-Rodent                       48.0/100  ======0=====0====================0==0=0================1===================1====0=====0==========1==                                                                                                                                                                  
7: SF Aaracia 2012                      47.0/100  =0=1=========0===========0======0=========00========01=====1===0===0================================                                                                                                                                                                  
8: SF Poly-1337Chess [Free Bin Version] 44.5/100  01=========0=======0==0=====0=====00======00======0=0============1==========0=====1===0=======0=====                                                                                                                                                                  
9: SF KingFighter                       38.0/100  0=====00==0===0==1==0=0===0=0===0=======0=0=0=0=0=0===========0=========0=0=======0===0=0=0===00====         

K8  standings (don't forget, new games with Oops, standings shown at top of this post.)
http://s1.postimg.org/ym5u8l9tr/Ufntitled.png
Parent - - By Dark_wizzie (***) [us] Date 2014-12-22 13:46 Edited 2014-12-22 14:38
An updated rating list is now available. I think the most interesting part is the rating of Needle and Oops. They are slipping! In addition, the top 10 places has shifted a lot under 4th place.
Parent - - By Adam Hair (**) [us] Date 2014-12-23 11:33
Would you like another bin book? I can give you the link to Book X, which tied for the top spot with Fauzi 3.6 in the GSEI book tournament.
Parent - - By Dark_wizzie (***) [us] Date 2014-12-23 14:52
Yes, I would. :cool:
Parent - By Dark_wizzie (***) [us] Date 2015-01-21 14:51 Edited 2015-01-21 15:01

No new Fauzi updates, still 3.6
Ordo has been updated 0.9.2 to 0.9.8.1.

Oops got 25 games.
Book X added with 400 games.

Oops dropped 0.2 elo.
More internal testing has been done for some experimental books.
Parent - - By Dark_wizzie (***) [us] Date 2015-03-16 06:12
Several hundred games have been done, possibly another thousand. However, I regret to say that Fauzi is short on time and cannot update the book right now from all the new games. Until he is back, the gauntlet is on hold.
Parent - By fauzi (**) [jo] Date 2015-03-16 13:10
Hi Dark Wizzie,

thank you for your updates, unfortunately as you correctly mentioned, this period is being very intense for me and cannot find much time for finalize the new version "Fauzi 3.9.abk"
I'm also working on a project of a strong opening book specific for Stockfish engine to be tested on SSDF website with the cooperation of Stockfish team, and I trust it will be a successful project, and I'm planning on making that opening book public to everybody despite of the Fauzi.abk series which remains private.
Regarding Fauzi 3.9.abk I will try to finalize it within this week and send it to you for testing.

Thanks and regards,
Parent - By Dark_wizzie (***) [us] Date 2015-04-28 01:15 Edited 2015-04-28 01:18

I'm not sure what I'm going to do now, now that development of the book has stopped.
There were some concerns of regression, but it doesn't seem to be that big of a deal, although Oops climbed quite a few points.

I can't show you the win/losses, so I took all the new games and ran it through elostat.


The latest batch of games were run with 15, 30, and 60 minute time controls. A lot of the games were 30 minutes.

-Ordo has been updated to version 1.
-1,000,000 simulations are run for Ordo.
-Perfect got updated to Perfect 2015.
-Barracuda got updated to Barracude 2015.
Up Topic The Rybka Lounge / Opening Books / Fauzi Gauntlet III

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill