Not logged inRybka Chess Community Forum
Up Topic Rybka Support & Discussion / Rybka Discussion / Parameters Experiment 38: 64 Elo over R4 default
1 2 3 4 5 6 Previous Next  
Parent - By mindbreaker (****) Date 2010-12-13 22:37 Edited 2010-12-13 22:43
Update:

1  Rybka 4 x64 Exp 61 v2    3371  1207
2  Rybka 4 x64 Exp 75 v1    3369  733
3  Rybka 4 x64 Exp 53 v1    3362  918
4  Rybka 4 x64 Exp 51v1    3362  919
5  Rybka 4 x64 Exp 54 v1    3361  918
6  Rybka 4 x64 Exp 71 v1    3359  1298
7  Rybka 4 x64 Exp 31 v1    3360  1115
8  Rybka 4 x64 Exp 37 v1    3360  1112
9  Rybka 4 x64 Exp 44 v1    3358  1239
10  Rybka 4 x64 Exp 49 v1    3358  919
11  Rybka 4 x64 Exp 42 v1    3357  1239
12  Rybka 4 x64 Exp 78 v1    3358  732
13  Rybka 4 x64 Exp 63 v1    3356  1207
14  Rybka 4 x64 Exp 34 v1    3356  1114
15  Rybka 4 x64 Exp 32 v1    3356  1114
16  Rybka 4 x64 Exp 47 v1    3355  1238
17  Rybka 4 x64 Exp 55 v1    3356  918
18  Rybka 4 x64 Exp 68 v1    3355  1298
19  Rybka 4 x64 Exp 73 v1    3353  1298
20  Rybka 4 x64 Exp 30 v1    3353  1115
21  Rybka 4 x64 Exp 52 v1    3354  918
22  Rybka 4 x64 Exp 67 v1    3352  1298
23  Rybka 4 x64 Exp 24 v2    3352  1131
24  Rybka 4 x64 Exp 26 v2    3352  1129
25  Rybka 4 x64 Exp 33 v1    3351  1114
26  Rybka 4 x64 Exp 79 v1    3352  732
27  Rybka 4 x64 Exp 56 v1    3349  918
28  Rybka 4 x64 Exp 28 v1    3349  1127
29  Rybka 4 x64 Exp 36 v1    3349  1112
30  Rybka 4 x64 Exp 38 v1    3348  1112
31  Rybka 4 x64 Exp 66 v1    3348  1299
32  Rybka 4 x64 Exp 40 v1    3347  1240
33  Rybka 4 x64 Exp 41 v1    3348  1239
34  Rybka 4 x64 Exp 57 v2    3347  1209
35  Rybka 4 x64 Exp 72 v1    3346  1298
36  Rybka 4 x64 Exp 48 v1    3345  920
37  Rybka 4 x64 Exp 77 v1    3345  733
38  Rybka 4 x64 Exp 74 v1    3344  1298
39  Rybka 4 x64 Exp 69 v1    3344  1298
40  Rybka 4 x64 Exp 60 v1    3345  1207
41  Rybka 4 x64 Exp 62 v1    3344  1207
42  Rybka 4 x64 Exp 64 v1    3345  1206
43  Rybka 4 x64 Exp 65 v1    3344  1206
44  Rybka 4 x64 Exp 46 v1    3343  1238
45  Rybka 4 x64 Exp 45 v1    3342  1239
46  Rybka 4 x64 Exp 59 v1    3342  1208
47  Rybka 4 x64 Exp 70 v1    3341  1298
48  Rybka 4 x64 Exp 39 v1    3341  1240
49  Rybka 4 x64 Exp 25 v2    3342  1130
50  Rybka 4 x64 Exp 43 v1    3341  1239
51  Rybka 4 x64 Exp 35 v1    3339  1114
52  Rybka 4 x64 Exp 58 v1    3338  1209
53  Rybka 4 x64 Exp 27 v1    3339  1128
54  Rybka 4 x64 Exp 50 v1    3337  919
55  Rybka 4 x64 Exp 76 v1    3337  733
56  Rybka 4 x64 Exp 29 v1    3336  1127
57  Rybka 4 x64 Exp 11 v1    3334  453
58  Rybka 4 x64 Exp 18 v1    3331  2649
59  Rybka 4 x64 Exp 22 v1    3326  1931
60  Rybka 4 x64 Exp 23 v1    3324  1884
61  Rybka 4 x64 Exp 9 v1    3320  429
62  Deep Rybka 4 x64 v1    3318  143
63  Rybka 4 x64 Exp 8 v1    3315  1212
64  Rybka 4 x64 Exp 14 Human    3315  798
65  Rybka 4 x64 Forum v1    3303  1078
66  Rybka 4 x64 Exp 15 v1    3302  1600
67  Rybka 4 x64 Exp 4 v1    3302  798
68  Rybka 4 x64 Exp 10 v1    3303  459
69  Rybka 4 x64 Exp 7 v1    3302  724
70  Rybka 3    3299  3186
71  Rybka 4 x64 Exp 1 v1    3299  814
72  Rybka 4 x64 Exp 21 v1    3298  800
73  Rybka 4 x64 Exp 3 v2    3295  375
74  Deep Rybka 4 x64    3294  1619 Default
75  Rybka 4 x64 Exp 16 v1    3294  494
76  Rybka 4 x64 Exp 20 v1    3291  524
77  Rybka 4 x64 Exp 2 v1    3289  418
78  Rybka 3 Dynamic    3287  2683
79  Rybka 4 x64 Exp 13 vC13510    3286  665
80  Deep Rybka 4 x64 Lasker    3285  424
81  Rybka 4 x64 Beta 15 v1    3285  378
82  Rybka 4 x64 Exp 19 v1    3281  168
83  Rybka 4 x64 Exp 12 v1    3280  797
84  Stockfish 1.9 JA 64bit    3276  4019
85  Rybka 3 Human    3270  2437
86  Rybka 4 x64 Exp 17 v1    3269  314
87  Rybka 4 x64 Exp 6 v1    3261  107
88  Stockfish 1.8 JA 64bit    3259  4036
89  Deep Rybka 4 x64 Human    3260  154
90  Stockfish 1.7.1 JA 64bit 4t    3240  3030
91  Critter 0.80 64-bit    3198  5918
92  Rybka 2.3.2a mp  Mod 8    3192  110
93  HIARCS 12 MP    3140  8070
94  spark-0.4    3126  5418
95  Deep Fritz 11    3118  8253
96  Deep Shredder 12    3116  5606
97  Naum 4 4t    3095  7543
98  Critter 0.70 64-bit    3078  2136
99  Deep Shredder 11 UCI    3068  2156
100  Zappa Mexico II    3045  5541
101  bright-0.5c    3038  2140
102  Komodo64 1.1 JA    3033  5690
103  Gull 1.0    2998  2358
104  Protector 1.3.6 x64 4t    2988  900
105  Protector 1.3.3 x64 4t    2977  2542
106  Thinker54AInert-MP64-UCI    2972  1800

Oddly, when I did a rating that included some other games I got this: (There really were not enough games to lead to such a large shift...whatever.)

Rybka 4 x64 Exp 75 v1    3370  775
Rybka 4 x64 Exp 71 v1    3363  1315
Rybka 4 x64 Exp 61 v2    3360  1409
Rybka 4 x64 Exp 73 v1    3360  1313
Rybka 4 x64 Exp 68 v1    3359  1313
Rybka 4 x64 Exp 78 v1    3357  770
Rybka 4 x64 Exp 67 v1    3355  1313
Rybka 4 x64 Exp 79 v1    3355  770
Rybka 4 x64 Exp 42 v1    3353  1440

In any case Exp 75 is certainly still in contention though not convincingly better. This looks like a close contest.
Parent - By mindbreaker (****) Date 2010-12-14 21:20 Edited 2010-12-14 21:24
Update:

1  Rybka 4 x64 Exp 61 v2    3371  1207
2  Rybka 4 x64 Exp 53 v1    3363  918
3  Rybka 4 x64 Exp 75 v1    3362  854
4  Rybka 4 x64 Exp 51v1    3362  919
5  Rybka 4 x64 Exp 54 v1    3361  918
6  Rybka 4 x64 Exp 71 v1    3359  1298
7  Rybka 4 x64 Exp 31 v1    3360  1115
8  Rybka 4 x64 Exp 37 v1    3360  1112
9  Rybka 4 x64 Exp 79 v1    3359  853
10  Rybka 4 x64 Exp 44 v1    3358  1239
11  Rybka 4 x64 Exp 49 v1    3358  919
12  Rybka 4 x64 Exp 42 v1    3357  1239
13  Rybka 4 x64 Exp 63 v1    3356  1207
14  Rybka 4 x64 Exp 34 v1    3356  1114
15  Rybka 4 x64 Exp 32 v1    3356  1114
16  Rybka 4 x64 Exp 47 v1    3355  1238
17  Rybka 4 x64 Exp 55 v1    3356  918
18  Rybka 4 x64 Exp 68 v1    3355  1298
19  Rybka 4 x64 Exp 73 v1    3353  1298
20  Rybka 4 x64 Exp 30 v1    3353  1115
21  Rybka 4 x64 Exp 52 v1    3354  918
22  Rybka 4 x64 Exp 78 v1    3354  853
23  Rybka 4 x64 Exp 67 v1    3352  1298
24  Rybka 4 x64 Exp 24 v2    3352  1131
25  Rybka 4 x64 Exp 26 v2    3353  1129
26  Rybka 4 x64 Exp 33 v1    3351  1114
27  Rybka 4 x64 Exp 56 v1    3349  918
28  Rybka 4 x64 Exp 28 v1    3349  1127
29  Rybka 4 x64 Exp 38 v1    3348  1112
30  Rybka 4 x64 Exp 36 v1    3349  1112
31  Rybka 4 x64 Exp 66 v1    3348  1299
32  Rybka 4 x64 Exp 40 v1    3347  1240
33  Rybka 4 x64 Exp 41 v1    3348  1239
34  Rybka 4 x64 Exp 57 v2    3347  1209
35  Rybka 4 x64 Exp 72 v1    3346  1298
36  Rybka 4 x64 Exp 64 v1    3345  1206
37  Rybka 4 x64 Exp 48 v1    3345  920
38  Rybka 4 x64 Exp 74 v1    3345  1298
39  Rybka 4 x64 Exp 69 v1    3345  1298
40  Rybka 4 x64 Exp 60 v1    3345  1207
41  Rybka 4 x64 Exp 62 v1    3344  1207
42  Rybka 4 x64 Exp 65 v1    3344  1206
43  Rybka 4 x64 Exp 46 v1    3343  1238
44  Rybka 4 x64 Exp 45 v1    3342  1239
45  Rybka 4 x64 Exp 59 v1    3342  1208
46  Rybka 4 x64 Exp 70 v1    3341  1298
47  Rybka 4 x64 Exp 39 v1    3341  1240
48  Rybka 4 x64 Exp 25 v2    3342  1130
49  Rybka 4 x64 Exp 77 v1    3342  853
50  Rybka 4 x64 Exp 43 v1    3341  1239
51  Rybka 4 x64 Exp 35 v1    3339  1114
52  Rybka 4 x64 Exp 58 v1    3338  1209
53  Rybka 4 x64 Exp 27 v1    3339  1128
54  Rybka 4 x64 Exp 50 v1    3337  919
55  Rybka 4 x64 Exp 76 v1    3338  853
56  Rybka 4 x64 Exp 29 v1    3336  1127
57  Rybka 4 x64 Exp 11 v1    3334  453
58  Rybka 4 x64 Exp 18 v1    3331  2649
59  Rybka 4 x64 Exp 22 v1    3326  1931
60  Rybka 4 x64 Exp 23 v1    3324  1884
61  Rybka 4 x64 Exp 9 v1    3320  429
62  Deep Rybka 4 x64 v1    3318  143
63  Rybka 4 x64 Exp 8 v1    3315  1212
64  Rybka 4 x64 Exp 14 Human    3316  798
65  Rybka 4 x64 Forum v1    3303  1078
66  Rybka 4 x64 Exp 15 v1    3303  1600
67  Rybka 4 x64 Exp 4 v1    3302  798
68  Rybka 4 x64 Exp 10 v1    3303  459
69  Rybka 4 x64 Exp 7 v1    3302  724
70  Rybka 3    3299  3186
71  Rybka 4 x64 Exp 1 v1    3299  814
72  Rybka 4 x64 Exp 21 v1    3298  800
73  Rybka 4 x64 Exp 3 v2    3295  375
74  Deep Rybka 4 x64    3294  1619 Default
75  Rybka 4 x64 Exp 16 v1    3294  494
76  Rybka 4 x64 Exp 20 v1    3291  524
77  Rybka 4 x64 Exp 2 v1    3289  418
78  Rybka 3 Dynamic    3287  2683
79  Rybka 4 x64 Exp 13 vC13510    3286  665
80  Deep Rybka 4 x64 Lasker    3285  424
81  Rybka 4 x64 Beta 15 v1    3285  378
82  Rybka 4 x64 Exp 19 v1    3281  168
83  Rybka 4 x64 Exp 12 v1    3280  797
84  Stockfish 1.9 JA 64bit    3276  4019
85  Rybka 3 Human    3270  2437
86  Rybka 4 x64 Exp 17 v1    3269  314
87  Rybka 4 x64 Exp 6 v1    3261  107
88  Stockfish 1.8 JA 64bit    3259  4036
89  Deep Rybka 4 x64 Human    3260  154
90  Stockfish 1.7.1 JA 64bit 4t    3240  3030
91  Critter 0.80 64-bit    3198  5918
92  Rybka 2.3.2a mp  Mod 8    3192  110
93  HIARCS 12 MP    3140  8070
94  spark-0.4    3127  5684
95  Deep Fritz 11    3118  8253
96  Deep Shredder 12    3115  5943
97  Naum 4 4t    3095  7543
98  Critter 0.70 64-bit    3078  2136
99  Deep Shredder 11 UCI    3068  2156
100  Zappa Mexico II    3045  5541
101  bright-0.5c    3038  2140
102  Komodo64 1.1 JA    3033  5690
103  Gull 1.0    2998  2358
104  Protector 1.3.6 x64 4t    2988  900
105  Protector 1.3.3 x64 4t    2977  2542
106  Thinker54AInert-MP64-UCI    2972  1800

Exp 75 has be very strong against strong players: The 3 Rybka 3s, Stockfish, and Critter but Exp 79 has be obliterating weaker players and as more weaker players are next as the cycles are in order of strength, I would not be surprised to see it reach the top.  Take a look at what it did to Deep Shredder 12 which at 3140elo and 5th place at CCRL is not a bad engine http://www.computerchess.org.uk/ccrl/404.live/   :
Parent - By mindbreaker (****) Date 2010-12-15 18:16
Update:

1  Rybka 4 x64 Exp 61 v2    3372  1207
2  Rybka 4 x64 Exp 79 v1    3366  953
3  Rybka 4 x64 Exp 75 v1    3364  954

4  Rybka 4 x64 Exp 53 v1    3363  918
5  Rybka 4 x64 Exp 51v1    3363  919
6  Rybka 4 x64 Exp 54 v1    3362  918
7  Rybka 4 x64 Exp 71 v1    3360  1298
8  Rybka 4 x64 Exp 31 v1    3361  1115
9  Rybka 4 x64 Exp 37 v1    3361  1112
10  Rybka 4 x64 Exp 44 v1    3359  1239
11  Rybka 4 x64 Exp 49 v1    3359  919
12  Rybka 4 x64 Exp 42 v1    3358  1239
13  Rybka 4 x64 Exp 78 v1    3358  953
14  Rybka 4 x64 Exp 63 v1    3357  1207
15  Rybka 4 x64 Exp 47 v1    3356  1238
16  Rybka 4 x64 Exp 32 v1    3357  1114
17  Rybka 4 x64 Exp 34 v1    3357  1114
18  Rybka 4 x64 Exp 55 v1    3357  918
19  Rybka 4 x64 Exp 68 v1    3356  1298
20  Rybka 4 x64 Exp 73 v1    3354  1298
21  Rybka 4 x64 Exp 52 v1    3355  918
22  Rybka 4 x64 Exp 67 v1    3353  1298
23  Rybka 4 x64 Exp 24 v2    3353  1131
24  Rybka 4 x64 Exp 26 v2    3353  1129
25  Rybka 4 x64 Exp 30 v1    3354  1115
26  Rybka 4 x64 Exp 33 v1    3352  1114
27  Rybka 4 x64 Exp 56 v1    3350  918
28  Rybka 4 x64 Exp 28 v1    3350  1127
29  Rybka 4 x64 Exp 36 v1    3350  1112
30  Rybka 4 x64 Exp 38 v1    3349  1112
31  Rybka 4 x64 Exp 66 v1    3349  1299
32  Rybka 4 x64 Exp 40 v1    3348  1240
33  Rybka 4 x64 Exp 41 v1    3349  1239
34  Rybka 4 x64 Exp 57 v2    3348  1209
35  Rybka 4 x64 Exp 77 v1    3349  953
36  Rybka 4 x64 Exp 72 v1    3347  1298
37  Rybka 4 x64 Exp 64 v1    3346  1206
38  Rybka 4 x64 Exp 48 v1    3346  920
39  Rybka 4 x64 Exp 69 v1    3346  1298
40  Rybka 4 x64 Exp 74 v1    3346  1298
41  Rybka 4 x64 Exp 60 v1    3346  1207
42  Rybka 4 x64 Exp 62 v1    3345  1207
43  Rybka 4 x64 Exp 65 v1    3345  1206
44  Rybka 4 x64 Exp 46 v1    3344  1238
45  Rybka 4 x64 Exp 45 v1    3343  1239
46  Rybka 4 x64 Exp 59 v1    3343  1208
47  Rybka 4 x64 Exp 70 v1    3342  1298
48  Rybka 4 x64 Exp 39 v1    3342  1240
49  Rybka 4 x64 Exp 25 v2    3343  1130
50  Rybka 4 x64 Exp 43 v1    3342  1239
51  Rybka 4 x64 Exp 76 v1    3342  953
52  Rybka 4 x64 Exp 35 v1    3340  1114
53  Rybka 4 x64 Exp 58 v1    3339  1209
54  Rybka 4 x64 Exp 27 v1    3340  1128
55  Rybka 4 x64 Exp 50 v1    3338  919
56  Rybka 4 x64 Exp 29 v1    3336  1127
57  Rybka 4 x64 Exp 11 v1    3335  453
58  Rybka 4 x64 Exp 18 v1    3332  2649
59  Rybka 4 x64 Exp 22 v1    3327  1931
60  Rybka 4 x64 Exp 23 v1    3325  1884
61  Rybka 4 x64 Exp 9 v1    3321  429
62  Deep Rybka 4 x64 v1    3319  143
63  Rybka 4 x64 Exp 8 v1    3316  1212
64  Rybka 4 x64 Exp 14 Human    3316  798
65  Rybka 4 x64 Forum v1    3304  1078
66  Rybka 4 x64 Exp 10 v1    3304  459
67  Rybka 4 x64 Exp 15 v1    3303  1600
68  Rybka 4 x64 Exp 4 v1    3303  798
69  Rybka 4 x64 Exp 7 v1    3302  724
70  Rybka 3    3301  3186
71  Rybka 4 x64 Exp 1 v1    3299  814
72  Rybka 4 x64 Exp 21 v1    3299  800
73  Rybka 4 x64 Exp 3 v2    3296  375
74  Deep Rybka 4 x64    3294  1619 Default
75  Rybka 4 x64 Exp 16 v1    3294  494
76  Rybka 4 x64 Exp 20 v1    3292  524
77  Rybka 4 x64 Exp 2 v1    3289  418
78  Rybka 3 Dynamic    3289  2683
79  Rybka 4 x64 Exp 13 vC13510    3287  665
80  Deep Rybka 4 x64 Lasker    3286  424
81  Rybka 4 x64 Beta 15 v1    3286  378
82  Rybka 4 x64 Exp 19 v1    3282  168
83  Rybka 4 x64 Exp 12 v1    3280  797
84  Stockfish 1.9 JA 64bit    3278  4019
85  Rybka 3 Human    3272  2437
86  Rybka 4 x64 Exp 17 v1    3269  314
87  Rybka 4 x64 Exp 6 v1    3262  107
88  Stockfish 1.8 JA 64bit    3260  4036
89  Deep Rybka 4 x64 Human    3260  154
90  Stockfish 1.7.1 JA 64bit 4t    3241  3030
91  Critter 0.80 64-bit    3200  5918
92  Rybka 2.3.2a mp  Mod 8    3193  110
93  HIARCS 12 MP    3141  8070
94  spark-0.4    3128  5684
95  Deep Fritz 11    3119  8253
96  Deep Shredder 12    3116  5943
97  Naum 4 4t    3093  8043
98  Critter 0.70 64-bit    3079  2136
99  Deep Shredder 11 UCI    3068  2156
100  Zappa Mexico II    3046  5541
101  bright-0.5c    3038  2140
102  Komodo64 1.1 JA    3034  5690
103  Gull 1.0    2999  2358
104  Protector 1.3.6 x64 4t    2989  900
105  Protector 1.3.3 x64 4t    2978  2542
106  Thinker54AInert-MP64-UCI    2973  1800

Well, Exp 79 has overtaken Exp 75 with another fine result this time holding Naum 4 to 11%. Not a record, but in actuality, possibly the strongest result against Naum 4 because the "own book" has been left on and Naum 4 has had 8000+ games of experience to hone its book.  7 more Elo is still possible as there are 4 more normal opponents and I may run a few more as well; I have not decided.
Parent - - By mindbreaker (****) Date 2010-12-16 22:26
Update:

1  Rybka 4 x64 Exp 61 v2    3372  1207
2  Rybka 4 x64 Exp 79 v1    3367  1153
3  Rybka 4 x64 Exp 75 v1    3365  1154

4  Rybka 4 x64 Exp 51v1    3363  919
5  Rybka 4 x64 Exp 53 v1    3364  918
6  Rybka 4 x64 Exp 54 v1    3363  918
7  Rybka 4 x64 Exp 71 v1    3360  1298
8  Rybka 4 x64 Exp 31 v1    3360  1115
9  Rybka 4 x64 Exp 37 v1    3361  1112
10  Rybka 4 x64 Exp 44 v1    3359  1239
11  Rybka 4 x64 Exp 49 v1    3359  919
12  Rybka 4 x64 Exp 42 v1    3358  1239
13  Rybka 4 x64 Exp 63 v1    3357  1207
14  Rybka 4 x64 Exp 55 v1    3357  918
15  Rybka 4 x64 Exp 47 v1    3356  1238
16  Rybka 4 x64 Exp 78 v1    3357  1153
17  Rybka 4 x64 Exp 34 v1    3357  1114
18  Rybka 4 x64 Exp 32 v1    3357  1114
19  Rybka 4 x64 Exp 68 v1    3356  1298
20  Rybka 4 x64 Exp 52 v1    3355  918
21  Rybka 4 x64 Exp 73 v1    3354  1298
22  Rybka 4 x64 Exp 67 v1    3353  1298
23  Rybka 4 x64 Exp 24 v2    3353  1131
24  Rybka 4 x64 Exp 26 v2    3354  1129
25  Rybka 4 x64 Exp 30 v1    3354  1115
26  Rybka 4 x64 Exp 77 v1    3353  1153
27  Rybka 4 x64 Exp 33 v1    3352  1114
28  Rybka 4 x64 Exp 56 v1    3351  918
29  Rybka 4 x64 Exp 28 v1    3350  1127
30  Rybka 4 x64 Exp 36 v1    3350  1112
31  Rybka 4 x64 Exp 38 v1    3349  1112
32  Rybka 4 x64 Exp 66 v1    3349  1299
33  Rybka 4 x64 Exp 40 v1    3348  1240
34  Rybka 4 x64 Exp 41 v1    3349  1239
35  Rybka 4 x64 Exp 57 v2    3348  1209
36  Rybka 4 x64 Exp 72 v1    3347  1298
37  Rybka 4 x64 Exp 64 v1    3346  1206
38  Rybka 4 x64 Exp 48 v1    3346  920
39  Rybka 4 x64 Exp 69 v1    3346  1298
40  Rybka 4 x64 Exp 74 v1    3346  1298
41  Rybka 4 x64 Exp 60 v1    3346  1207
42  Rybka 4 x64 Exp 62 v1    3345  1207
43  Rybka 4 x64 Exp 65 v1    3345  1206
44  Rybka 4 x64 Exp 46 v1    3344  1238
45  Rybka 4 x64 Exp 45 v1    3343  1239
46  Rybka 4 x64 Exp 59 v1    3343  1208
47  Rybka 4 x64 Exp 70 v1    3342  1298
48  Rybka 4 x64 Exp 39 v1    3342  1240
49  Rybka 4 x64 Exp 25 v2    3343  1130
50  Rybka 4 x64 Exp 43 v1    3341  1239
51  Rybka 4 x64 Exp 76 v1    3340  1153
52  Rybka 4 x64 Exp 35 v1    3340  1114
53  Rybka 4 x64 Exp 58 v1    3339  1209
54  Rybka 4 x64 Exp 27 v1    3340  1128
55  Rybka 4 x64 Exp 50 v1    3339  919
56  Rybka 4 x64 Exp 29 v1    3337  1127
57  Rybka 4 x64 Exp 11 v1    3335  453
58  Rybka 4 x64 Exp 18 v1    3332  2649
59  Rybka 4 x64 Exp 22 v1    3327  1931
60  Rybka 4 x64 Exp 23 v1    3325  1884
61  Rybka 4 x64 Exp 9 v1    3321  429
62  Deep Rybka 4 x64 v1    3319  143
63  Rybka 4 x64 Exp 8 v1    3316  1212
64  Rybka 4 x64 Exp 14 Human    3316  798
65  Rybka 4 x64 Forum v1    3304  1078
66  Rybka 4 x64 Exp 10 v1    3304  459
67  Rybka 4 x64 Exp 15 v1    3303  1600
68  Rybka 4 x64 Exp 4 v1    3302  798
69  Rybka 4 x64 Exp 7 v1    3302  724
70  Rybka 3    3301  3186
71  Rybka 4 x64 Exp 1 v1    3299  814
72  Rybka 4 x64 Exp 21 v1    3298  800
73  Rybka 4 x64 Exp 3 v2    3295  375
74  Deep Rybka 4 x64    3294  1619 Default
75  Rybka 4 x64 Exp 16 v1    3294  494
76  Rybka 4 x64 Exp 20 v1    3291  524
77  Rybka 3 Dynamic    3289  2683
78  Rybka 4 x64 Exp 2 v1    3289  418
79  Rybka 4 x64 Exp 13 vC13510    3287  665
80  Deep Rybka 4 x64 Lasker    3285  424
81  Rybka 4 x64 Beta 15 v1    3286  378
82  Rybka 4 x64 Exp 19 v1    3281  168
83  Rybka 4 x64 Exp 12 v1    3280  797
84  Stockfish 1.9 JA 64bit    3278  4019
85  Rybka 3 Human    3272  2437
86  Rybka 4 x64 Exp 17 v1    3269  314
87  Rybka 4 x64 Exp 6 v1    3261  107
88  Stockfish 1.8 JA 64bit    3260  4036
89  Deep Rybka 4 x64 Human    3260  154
90  Stockfish 1.7.1 JA 64bit 4t    3241  3030
91  Critter 0.80 64-bit    3200  5918
92  Rybka 2.3.2a mp  Mod 8    3194  110
93  HIARCS 12 MP    3141  8070
94  spark-0.4    3129  5684
95  Deep Fritz 11    3120  8253
96  Deep Shredder 12    3116  5943
97  Naum 4 4t    3093  8043
98  Critter 0.70 64-bit    3079  2136
99  Deep Shredder 11 UCI    3068  2156
100  Zappa Mexico II    3044  6041
101  bright-0.5c    3038  2140
102  Komodo64 1.1 JA    3030  6190
103  Gull 1.0    2999  2358
104  Protector 1.3.6 x64 4t    2989  900
105  Protector 1.3.3 x64 4t    2978  2542
106  Thinker54AInert-MP64-UCI    2973  1800
Parent - By mindbreaker (****) Date 2010-12-16 22:43
Sorry, there was a glitch, the new bunch have played 1200 games each; Exp 79 and Exp 75 are tied at 3366 instead.
Parent - By mindbreaker (****) Date 2010-12-19 01:21
Update: There are 15 or so rounds left, but nothing is going to change.

My list with the unmentionable engine games:


Rybka 4 x64 Exp 75 v1  3363  1487
Rybka 4 x64 Exp 71 v1  3363  1315
Rybka 4 x64 Exp 61 v2  3360  1409
Rybka 4 x64 Exp 73 v1  3359  1313
Rybka 4 x64 Exp 68 v1  3359  1313
Rybka 4 x64 Exp 79 v1  3358  1485
Rybka 4 x64 Exp 67 v1  3355  1313
Rybka 4 x64 Exp 49 v1  3353  1219
Rybka 4 x64 Exp 42 v1  3353  1440
Rybka 4 x64 Exp 54 v1  3352  1218
Rybka 4 x64 Exp 66 v1  3352  1313
Rybka 4 x64 Exp 63 v1  3351  1407
Rybka 4 x64 Exp 56 v1  3350  1218
Rybka 4 x64 Exp 77 v1  3350  1485
Rybka 4 x64 Exp 78 v1  3349  1485
Rybka 4 x64 Exp 44 v1  3350  1439
Rybka 4 x64 Exp 31 v1  3350  1324
Rybka 4 x64 Exp 72 v1  3350  1313
Rybka 4 x64 Exp 51v1  3350  1220
Rybka 4 x64 Exp 74 v1  3349  1313
Rybka 4 x64 Exp 55 v1  3349  1218
Rybka 4 x64 Exp 41 v1  3347  1439
Rybka 4 x64 Exp 47 v1  3347  1438
Rybka 4 x64 Exp 39 v1  3347  1440
Rybka 4 x64 Exp 62 v1  3347  1407
Rybka 4 x64 Exp 69 v1  3346  1313
Rybka 4 x64 Exp 52 v1  3347  1218
Rybka 4 x64 Exp 57 v2  3346  1409
Rybka 4 x64 Exp 70 v1  3345  1313
Rybka 4 x64 Exp 45 v1  3345  1439
Rybka 4 x64 Exp 37 v1  3345  1319
Rybka 4 x64 Exp 48 v1  3344  1220
Rybka 4 x64 Exp 64 v1  3343  1406
Rybka 4 x64 Exp 30 v1  3343  1323
Rybka 4 x64 Exp 40 v1  3343  1440
Rybka 4 x64 Exp 60 v1  3342  1407
Rybka 4 x64 Exp 24 v2  3343  1332
Rybka 4 x64 Exp 53 v1  3342  1218
Rybka 4 x64 Exp 38 v1  3342  1318
Rybka 4 x64 Exp 46 v1  3340  1438
Rybka 4 x64 Exp 26 v2  3340  1329
Rybka 4 x64 Exp 34 v1  3341  1322
Rybka 4 x64 Exp 59 v1  3339  1408

The list with them thrown out looks like this:

1  Rybka 4 x64 Exp 61 v2    3371  1207
2  Rybka 4 x64 Exp 53 v1    3362  918
3  Rybka 4 x64 Exp 51v1    3362  919
4  Rybka 4 x64 Exp 54 v1    3361  918
5  Rybka 4 x64 Exp 71 v1    3359  1298
6  Rybka 4 x64 Exp 31 v1    3360  1115
7  Rybka 4 x64 Exp 37 v1    3360  1112
8  Rybka 4 x64 Exp 75 v1    3358  1485
9  Rybka 4 x64 Exp 44 v1    3358  1239
10  Rybka 4 x64 Exp 49 v1    3358  919
11  Rybka 4 x64 Exp 42 v1    3356  1239
12  Rybka 4 x64 Exp 63 v1    3356  1207
13  Rybka 4 x64 Exp 34 v1    3356  1114
14  Rybka 4 x64 Exp 32 v1    3356  1114
15  Rybka 4 x64 Exp 55 v1    3356  918
16  Rybka 4 x64 Exp 68 v1    3355  1298
17  Rybka 4 x64 Exp 47 v1    3355  1238
18  Rybka 4 x64 Exp 79 v1    3353  1485
19  Rybka 4 x64 Exp 73 v1    3353  1298
20  Rybka 4 x64 Exp 52 v1    3354  918
21  Rybka 4 x64 Exp 67 v1    3352  1298
22  Rybka 4 x64 Exp 24 v2    3353  1131
23  Rybka 4 x64 Exp 26 v2    3353  1129
24  Rybka 4 x64 Exp 30 v1    3353  1115
25  Rybka 4 x64 Exp 33 v1    3351  1114
26  Rybka 4 x64 Exp 28 v1    3349  1127
27  Rybka 4 x64 Exp 56 v1    3349  918
28  Rybka 4 x64 Exp 36 v1    3349  1112
29  Rybka 4 x64 Exp 38 v1    3348  1112
30  Rybka 4 x64 Exp 66 v1    3348  1299
31  Rybka 4 x64 Exp 41 v1    3347  1239
32  Rybka 4 x64 Exp 57 v2    3347  1209
33  Rybka 4 x64 Exp 40 v1    3347  1240
34  Rybka 4 x64 Exp 77 v1    3345  1485
35  Rybka 4 x64 Exp 72 v1    3346  1298
36  Rybka 4 x64 Exp 64 v1    3345  1206
37  Rybka 4 x64 Exp 78 v1    3345  1485
38  Rybka 4 x64 Exp 74 v1    3344  1298
39  Rybka 4 x64 Exp 69 v1    3344  1298
40  Rybka 4 x64 Exp 62 v1    3344  1207
41  Rybka 4 x64 Exp 60 v1    3345  1207
42  Rybka 4 x64 Exp 65 v1    3344  1206
43  Rybka 4 x64 Exp 48 v1    3345  920
44  Rybka 4 x64 Exp 46 v1    3343  1238
45  Rybka 4 x64 Exp 59 v1    3342  1208
46  Rybka 4 x64 Exp 25 v2    3342  1130
47  Rybka 4 x64 Exp 70 v1    3341  1298
48  Rybka 4 x64 Exp 45 v1    3342  1239
49  Rybka 4 x64 Exp 39 v1    3341  1240
50  Rybka 4 x64 Exp 43 v1    3340  1239
51  Rybka 4 x64 Exp 27 v1    3339  1128
52  Rybka 4 x64 Exp 35 v1    3339  1114
53  Rybka 4 x64 Exp 58 v1    3338  1209
54  Rybka 4 x64 Exp 50 v1    3337  919
55  Rybka 4 x64 Exp 29 v1    3336  1127
56  Rybka 4 x64 Exp 76 v1    3334  1485
57  Rybka 4 x64 Exp 18 v1    3331  2649
58  Rybka 4 x64 Exp 11 v1    3332  453
59  Rybka 4 x64 Exp 22 v1    3326  1931
60  Rybka 4 x64 Exp 23 v1    3324  1884
61  Deep Rybka 4 x64 v1    3319  143
62  Rybka 4 x64 Exp 9 v1    3318  429
63  Rybka 4 x64 Exp 14 Human    3316  798
64  Rybka 4 x64 Exp 8 v1    3315  1212
65  Rybka 4 x64 Exp 15 v1    3303  1600
66  Rybka 4 x64 Forum v1    3302  1078
67  Rybka 4 x64 Exp 4 v1    3302  798
68  Rybka 4 x64 Exp 7 v1    3302  724
69  Rybka 4 x64 Exp 10 v1    3301  459
70  Rybka 4 x64 Exp 1 v1    3299  814
71  Rybka 3    3299  3186
72  Rybka 4 x64 Exp 21 v1    3299  800
73  Rybka 4 x64 Exp 3 v2    3296  375
74  Deep Rybka 4 x64    3294  1619 Default
75  Rybka 4 x64 Exp 16 v1    3294  494
76  Rybka 4 x64 Exp 20 v1    3291  524
77  Rybka 4 x64 Exp 2 v1    3289  418
78  Rybka 3 Dynamic    3287  2683
79  Rybka 4 x64 Exp 13 vC13510    3287  665
80  Deep Rybka 4 x64 Lasker    3286  424
81  Rybka 4 x64 Beta 15 v1    3286  378
82  Rybka 4 x64 Exp 19 v1    3282  168
83  Rybka 4 x64 Exp 12 v1    3280  797
84  Stockfish 1.9 JA 64bit    3276  4019
85  Rybka 3 Human    3269  2437
86  Rybka 4 x64 Exp 17 v1    3269  314
87  Rybka 4 x64 Exp 6 v1    3262  107
88  Deep Rybka 4 x64 Human    3260  154
89  Stockfish 1.8 JA 64bit    3258  4036
90  Stockfish 1.7.1 JA 64bit 4t    3241  3030
91  Critter 0.80 64-bit    3198  5918
92  Rybka 2.3.2a mp  Mod 8    3192  110
93  HIARCS 12 MP    3140  8070
94  spark-0.4    3128  5918
95  Deep Fritz 11    3119  8253
96  Deep Shredder 12    3115  5943
97  Naum 4 4t    3092  8043
98  Critter 0.70 64-bit    3079  2136
99  Deep Shredder 11 UCI    3069  2156
100  Zappa Mexico II    3042  6041
101  bright-0.5c    3039  2140
102  Komodo64 1.1 JA    3029  6190
103  Gull 1.0    2995  2858
104  Deep Sjeng c't 2010    2990  425
105  Protector 1.3.6 x64 4t    2989  900
106  Protector 1.3.3 x64 4t    2983  3042
107  Thinker54AInert-MP64-UCI    2973  1800

But complicating maters I ran 100 games each for all three personalities of Rybka 3.  That, I tend to think of as one opponent rather than three.  If we weight it as one opponent, then Exp 79 is rated above Exp 75.

I think I will include both in the upcoming tournament.


Here are the parameters for the group:

Experiment 75    3  100  150  3  2  11  8  11  8  11  8  17  13  41  33  100
Experiment 76    3  100  150  3  2  11  8  11  8  11  8  17  13  41  34  100
Experiment 77    3  100  150  3  2  11  8  11  8  11  8  17  13  42  33  100
Experiment 78    3  100  150  3  2  11  8  11  8  11  8  17  13  42  34  100
Experiment 79    3  100  150  3  2  12  9  12  9  12  9  17  13  41  34  100
Parent - - By mindbreaker (****) Date 2010-12-20 18:56
The Tournament of the Champions is underway.
The participants are: Exp 79, 75, 71, 61, 51, 31, 42, 24, 18, 8, Forum (Default with 3 100 150 timings), and Default.  The first opponent is Rybka 3 and will run 150 rounds.  That should take around 4 days.  Following that Stockfish 1.9 and then Critter 0.90, and all the other opponents in roughly decreasing strength.  The book is just a book made from 114,000 previous games up to 10 moves.  I ran a quick test of the book and it appeared to be strong so that's what will be used.
Parent - By mindbreaker (****) Date 2010-12-24 05:31
I don't normally post after only one opponent, but as this is the Tournament of the Champions, I thought: why not?  As expected, Forum (Default with 3,100,150 timings) did very well as the first opponent (Rybka 3) is the strongest one and "contempt" settings aren't the most brilliant idea against nearly equal strength, but better time usage should still be effective.  Actually, it is surprising it did not win this round; Exp 71 actually came in ahead of it.

Here is the effect on the overall ratings (with all games included):

1  Rybka 4 x64 Exp 71 v1    3340  1465
2  Rybka 4 x64 Exp 73 v1    3336  1313
3  Rybka 4 x64 Exp 68 v1    3335  1313
4  Rybka 4 x64 Exp 61 v2    3333  1559
5  Rybka 4 x64 Exp 67 v1    3331  1313
6  Rybka 4 x64 Exp 49 v1    3331  1219
7  Rybka 4 x64 Exp 54 v1    3331  1218
8  Rybka 4 x64 Exp 79 v1    3329  1745
9  Rybka 4 x64 Exp 44 v1    3329  1439
10  Rybka 4 x64 Exp 63 v1    3328  1407
11  Rybka 4 x64 Exp 66 v1    3328  1313
12  Rybka 4 x64 Exp 56 v1    3329  1218
13  Rybka 4 x64 Exp 75 v1    3328  1748
14  Rybka 4 x64 Exp 47 v1    3326  1438
15  Rybka 4 x64 Exp 72 v1    3326  1313
16  Rybka 4 x64 Exp 55 v1    3327  1218
17  Rybka 4 x64 Exp 41 v1    3326  1439
18  Rybka 4 x64 Exp 74 v1    3325  1313
19  Rybka 4 x64 Exp 31 v1    3324  1474
20  Rybka 4 x64 Exp 62 v1    3324  1407
21  Rybka 4 x64 Exp 51v1    3325  1370
22  Rybka 4 x64 Exp 37 v1    3325  1319
23  Rybka 4 x64 Exp 52 v1    3325  1218
24  Rybka 4 x64 Exp 45 v1    3324  1439
25  Rybka 4 x64 Exp 57 v2    3323  1409
26  Rybka 4 x64 Exp 26 v2    3323  1329
27  Rybka 4 x64 Exp 30 v1    3323  1323
28  Rybka 4 x64 Exp 24 v2    3322  1482
29  Rybka 4 x64 Exp 69 v1    3323  1313
30  Rybka 4 x64 Exp 48 v1    3322  1220
31  Rybka 4 x64 Exp 78 v1    3321  1595
32  Rybka 4 x64 Exp 40 v1    3322  1440
33  Rybka 4 x64 Exp 34 v1    3321  1322
34  Rybka 4 x64 Exp 38 v1    3322  1318
35  Rybka 4 x64 Exp 70 v1    3322  1313
36  Rybka 4 x64 Exp 77 v1    3321  1596
37  Rybka 4 x64 Exp 64 v1    3321  1406
38  Rybka 4 x64 Exp 53 v1    3320  1218
39  Rybka 4 x64 Exp 46 v1    3319  1438
40  Rybka 4 x64 Exp 60 v1    3320  1407
41  Rybka 4 x64 Exp 28 v1    3319  1327
42  Rybka 4 x64 Exp 25 v2    3318  1330
43  Rybka 4 x64 Exp 42 v1    3317  2642
44  Rybka 4 x64 Exp 59 v1    3317  1408
45  Rybka 4 x64 Exp 32 v1    3318  1322
46  Rybka 4 x64 Exp 36 v1    3318  1319
47  Rybka 4 x64 Exp 43 v1    3317  1439
48  Rybka 4 x64 Exp 65 v1    3316  1406
49  Rybka 4 x64 Exp 58 v1    3316  1409
50  Rybka 4 x64 Exp 33 v1    3315  1322
51  Rybka 4 x64 Exp 18 v1    3314  3190
52  Rybka 4 x64 Exp 76 v1    3313  1596
53  Rybka 4 x64 Exp 27 v1    3313  1328
54  Rybka 4 x64 Exp 50 v1    3312  1219
55  Rybka 4 x64 Exp 11 v1    3312  453
56  Rybka 4 x64 Exp 22 v1    3310  2231
57  Rybka 4 x64 Exp 35 v1    3310  1321
58  Rybka 4 x64 Exp 39 v1    3307  1549
59  Rybka 4 x64 Exp 29 v1    3306  1327
60  Rybka 4 x64 Exp 23 v1    3305  2178
61  Rybka 4 x64 Exp 8 v1    3301  1461
62  Rybka 4 x64 Forum v1    3298  1310
63  Rybka 4 x64 Exp 14 Human    3297  898
64  Rybka 4 x64 Exp 9 v1    3298  429
65  Rybka 4 x64 Exp 7 v1    3294  815
66  Deep Rybka 4 x64    3293  3295 default
67  Rybka 4 x64 Exp 4 v1    3291  899
68  Rybka 4 x64 Exp 20 v1    3290  590
69  Deep Rybka 4 x64 v1    3291  161
70  Rybka 4 x64 Exp 1 v1    3289  916
71  Rybka 4 x64 Exp 15 v1    3289  1799
72  Rybka 4 x64 Exp 3 v2    3289  422
73  Rybka 4 x64 Exp 21 v1    3286  900
74  Rybka 4 x64 Exp 16 v1    3287  555
75  Deep Rybka 4 x64 Lasker v5    3285  441
76  Rybka 4 x64 Exp 19 v1    3282  189
77  Rybka 4 x64 Exp 10 v1    3281  459
78  Deep Rybka 4 x64 Lasker    3278  477
79  Rybka 4 x64 Exp 13 vC13510    3276  749
80  Rybka 4 x64 Exp 2 v1    3275  471
81  Rybka 4 x64 Beta 15 v1    3275  426
82  Rybka 4 x64 Exp 12 v1    3275  896
83  Rybka 4 x64 Exp 17 v1    3262  355
84  Rybka 3    3262  8192 The opponent this round
85  Stockfish 1.9 JA 64bit    3252  4100
86  Rybka 3 Dynamic    3247  4698
87  Deep Rybka 4 x64 Human    3247  478
88  Rybka 3 Human    3240  4453
89  Stockfish 1.8 JA 64bit    3237  4067
90  Stockfish 1.7.1 JA 64bit 4t    3227  3218
91  Critter 0.80 64-bit    3173  5948
92  Rybka 3 1t    3125  211
93  HIARCS 12 MP    3119  8292
94  Naum 4 4t    3112  14802
95  spark-0.4    3103  5948
96  Deep Fritz 11    3098  8473
97  Deep Shredder 12    3091  5974
98  Critter 0.70 64-bit    3071  2326
99  Deep Shredder 11 UCI    3057  2346
100  bright-0.5c    3028  2329
101  Zappa Mexico II    3018  6059
102  Komodo64 1.1 JA    3012  6433
103  Deep Sjeng c't 2010    2982  500
104  Gull 1.0    2974  2858
105  Komodo64 1.0 JA    2974  252
106  Protector 1.3.6 x64 4t    2965  900
107  Protector 1.3.3 x64 4t    2961  3042
108  bright-0.3a nb    2956  527
109  Thinker54AInert-MP64-UCI    2945  1800

Next up is Stockfish 1.9 (1.9.1 is the same strength)
Parent - By mindbreaker (****) Date 2010-12-28 21:23
Update:  Stockfish 1.9 did well with the new book.  Here is the effect on the ratings:

1  Rybka 4 x64 Exp 71 v1  3333  1615
2  Rybka 4 x64 Exp 61 v2  3331  1709
3  Rybka 4 x64 Exp 51v1  3328  1520
4  Rybka 4 x64 Exp 75 v1  3326  1902
5  Rybka 4 x64 Exp 79 v1  3326  1900
6  Rybka 4 x64 Exp 31 v1  3324  1624
7  Rybka 4 x64 Exp 24 v2  3323  1632
8  Rybka 4 x64 Exp 18 v1  3315  3340
9  Rybka 4 x64 Exp 42 v1  3316  2792
10  Rybka 4 x64 Forum v1  3302  1460
11  Rybka 4 x64 Exp 8 v1  3302  1611
12  Deep Rybka 4 x64  3294  3445

Critter is next.
Parent - - By mindbreaker (****) Date 2010-12-30 20:14
Update:

Effect on ratings after 3 opponents:
1  Rybka 4 x64 Exp 61 v2  3329  1859
2  Rybka 4 x64 Exp 51v1  3328  1670
3  Rybka 4 x64 Exp 71 v1  3326  1765
4  Rybka 4 x64 Exp 79 v1  3322  2050
5  Rybka 4 x64 Exp 31 v1  3321  1774
6  Rybka 4 x64 Exp 75 v1  3320  2052
7  Rybka 4 x64 Exp 42 v1  3317  2942
8  Rybka 4 x64 Exp 24 v2  3316  1782
9  Rybka 4 x64 Exp 18 v1  3312  3490
10  Rybka 4 x64 Forum v1  3307  1610
11  Rybka 4 x64 Exp 8 v1  3301  1761
12  Deep Rybka 4 x64  3294  3595

Critter was the last of the heavyweights, so the contempt settings should begin to move up in the standings.    I was impressed with Critter 0.90's endgame which seems even better where 0.80 was already very good.  Critter 0.90 earned a rating of 3193 where 0.80 was 3170 for a gain of 23 Elo.

Next up is Hiarcs 12, which after 8292 games on my computer, is rated 3116.
Parent - - By Razor (****) Date 2010-12-30 21:15
Are you saying R4 Exp 61v2 has an estimated rating of 3329?  Surely this cannot be right - this looks like 200-300 too high doesn't it?  What am I missing?
Parent - By mindbreaker (****) Date 2010-12-30 21:34
Ratings are based on Deep Rybka 4 default rating earned at CCRL on the 40/4 list: http://www.computerchess.org.uk/ccrl/404.live/  All ratings on my list are adjusted relative to the 3294Elo earned there.  So the 3329Elo is performing at +35 Elo.  But part of that gain is a result of the timings (the first three numbers in the parameters settings) being a better choice than the default ones at fast time controls.  As such, one would not expect as large a gain at longer time controls like 40/40 or 40/120, and in fact, it is probably better to change those back to normal in long games.
At least 2 other sites have tested one or two if the Exps and found similar gains.
Parent - By mindbreaker (****) Date 2011-01-03 18:22 Edited 2011-01-03 18:27
Update:

After 4 opponents here is the effect on the ratings:

Rybka 4 x64 Exp 61 v2  3330  2009
Rybka 4 x64 Exp 51v1  3329  1820
Rybka 4 x64 Exp 71 v1  3327  1915
Rybka 4 x64 Exp 31 v1  3324  1924
Rybka 4 x64 Exp 79 v1  3321  2200
Rybka 4 x64 Exp 24 v2  3321  1932
Rybka 4 x64 Exp 75 v1  3318  2202
Rybka 4 x64 Exp 42 v1  3317  3092
Rybka 4 x64 Exp 18 v1  3311  3640
Rybka 4 x64 Forum v1  3304  1760 (3 100 150 timings, everything else Default)
Rybka 4 x64 Exp 8 v1  3304  1911
Deep Rybka 4 x64  3294  3745 (Default Parameters)


Hiarcs 12 did not do as well as usual, possibly as a result of the book.  Forum finished last this time.  Apparently the time management 3 100 150 by itself is not very good against Hiarcs 12 in fast games.  Exp 24 performed close to the record.

Next up: Naum 4
Parent - By mindbreaker (****) Date 2011-01-06 20:48
TOC Update: 5 opponents complete

Ratings effect:

Rybka 4 x64 Exp 61 v2  3333  2159
Rybka 4 x64 Exp 51v1  3331  1970
Rybka 4 x64 Exp 71 v1  3330  2065
Rybka 4 x64 Exp 31 v1  3324  2074
Rybka 4 x64 Exp 24 v2  3324  2082
Rybka 4 x64 Exp 79 v1  3323  2350
Rybka 4 x64 Exp 42 v1  3317  3242
Rybka 4 x64 Exp 75 v1  3317  2352
Rybka 4 x64 Exp 18 v1  3311  3790
Rybka 4 x64 Forum v1  3306  1910
Rybka 4 x64 Exp 8 v1  3305  2061
Deep Rybka 4 x64  3294  3895 Default settings


Next is Spark 1.0 should be fun to watch.  Spark is an interesting competitor. There are 9 more opponents scheduled to play including Spark.  But if there are any new free strong opponents released before the end of the tournament, I will attempt to add those.  But I am trying to stick to one author each and over 3000 Elo on the CCRL 40/4 list: http://www.computerchess.org.uk/ccrl/404.live/

I have switched back to Perfect 2010.ctg.  My book experienced a variety drain.  I guess I have a lot to learn about making opening books.  It was strong, I think, but when playing itself went down to one line. Anyone know about any tutorials on how to make books using Fritz 11 (I don't have Chessbase).  Learning was not checked in the tournament setup window.  I don't want to alter each engine's book parameters.  There must be some trick that can be done to the book itself as Perfect 2010.ctg does not appear to have this issue.
Parent - By mindbreaker (****) Date 2011-01-11 12:12
TOC Update 6 opponents complete:

Effect on ratings:

Rybka 4 x64 Exp 61 v2  3331  2310
Rybka 4 x64 Exp 51v1  3329  2121
Rybka 4 x64 Exp 71 v1  3327  2216
Rybka 4 x64 Exp 24 v2  3324  2233
Rybka 4 x64 Exp 79 v1  3323  2501
Rybka 4 x64 Exp 31 v1  3322  2225
Rybka 4 x64 Exp 42 v1  3317  3393
Rybka 4 x64 Exp 75 v1  3315  2503
Rybka 4 x64 Exp 18 v1  3311  3941
Rybka 4 x64 Forum v1  3306  2061
Rybka 4 x64 Exp 8 v1  3303  2212
Deep Rybka 4 x64  3294  4046


Spark 1.0 complete.  Next up Deep Shredder 12.  There still is a lot of room for movement even though there are many games complete because there are over half the opponents to come and they are the weaker half as it is run roughly in order of strength.
Parent - - By mindbreaker (****) Date 2011-01-12 00:24
TOC Update after 7 opponents:

Effect on ratings:

1  Rybka 4 x64 Exp 61 v2    3331  2460
2  Rybka 4 x64 Exp 51v1    3328  2271
3  Rybka 4 x64 Exp 24 v2    3326  2383
4  Rybka 4 x64 Exp 71 v1    3326  2366
5  Rybka 4 x64 Exp 79 v1    3325  2651
6  Rybka 4 x64 Exp 31 v1    3325  2375
7  Rybka 4 x64 Exp 75 v1    3318  2653
8  Rybka 4 x64 Exp 42 v1    3317  3543
9  Rybka 4 x64 Exp 18 v1    3314  4091
10  Rybka 4 x64 Forum v1    3306  2211
11  Rybka 4 x64 Exp 8 v1    3303  2362
12  Deep Rybka 4 x64    3294  4196


Deep Shredder 12 did much better than expected.  18.5% or 19% would be expected but it got 21.25%.  I added 32GB of 6-man endgame tables on a USB flash drive...it made very good use of them. 3-4-5-man on flash has been present from the start.
Parent - - By Ingo (***) Date 2011-01-12 18:27
Hi

I did a 1900 game test of the DR4 61v2 on IPON conditions as well:

Bayes: 
DR4 Exp61V2                       1900.0 (1457.5 : 442.5)
                                   100.0 ( 39.5 :  60.5) ******* ***              3008
                                   100.0 ( 50.5 :  49.5) Stockfish 2.01 JA        2921
                                   100.0 ( 62.0 :  38.0) Critter 0.90 SSE42       2891
                                   100.0 ( 69.0 :  31.0) Komodo64 1.3 JA          2828
                                   100.0 ( 76.5 :  23.5) Naum 4.2                 2820
                                   100.0 ( 74.0 :  26.0) Deep Shredder 12         2800
                                   100.0 ( 74.5 :  25.5) Gull 1.1                 2788
                                   100.0 ( 75.0 :  25.0) Deep Sjeng c't 2010 32b  2777
                                   100.0 ( 74.0 :  26.0) spark-1.0 SSE42          2754
                                   100.0 ( 77.5 :  22.5) HIARCS 13.2 MP 32b       2752
                                   100.0 ( 84.5 :  15.5) Zappa Mexico II          2713
                                   100.0 ( 85.5 :  14.5) Protector 1.3.2 JA       2700
                                   100.0 ( 85.5 :  14.5) Deep Onno 1-2-70         2684
                                   100.0 ( 84.0 :  16.0) Hannibal 1.0a            2683
                                   100.0 ( 86.0 :  14.0) Deep Junior 12           2679
                                   100.0 ( 91.5 :   8.5) Toga II 1.4 beta5c BB    2670
                                   100.0 ( 91.5 :   8.5) Umko 1.1 SSE42           2631
                                   100.0 ( 85.0 :  15.0) Jonny 4.00 32b           2616
                                   100.0 ( 91.5 :   8.5) Loop 13.6/2007           2614

Deep Rybka 4                      1900.0 (1456.0 : 444.0)
                                   100.0 ( 42.0 :  58.0) ******* ***              3008
                                   100.0 ( 53.5 :  46.5) Stockfish 2.01 JA        2921
                                   100.0 ( 59.5 :  40.5) Critter 0.90 SSE42       2891
                                   100.0 ( 69.5 :  30.5) Komodo64 1.3 JA          2828
                                   100.0 ( 71.0 :  29.0) Naum 4.2                 2820
                                   100.0 ( 78.5 :  21.5) Deep Shredder 12         2800
                                   100.0 ( 76.0 :  24.0) Gull 1.1                 2788
                                   100.0 ( 76.5 :  23.5) Deep Sjeng c't 2010 32b  2777
                                   100.0 ( 75.0 :  25.0) spark-1.0 SSE42          2754
                                   100.0 ( 75.5 :  24.5) HIARCS 13.2 MP 32b       2752
                                   100.0 ( 82.0 :  18.0) Zappa Mexico II          2713
                                   100.0 ( 81.5 :  18.5) Protector 1.3.2 JA       2700
                                   100.0 ( 84.5 :  15.5) Deep Onno 1-2-70         2684
                                   100.0 ( 87.0 :  13.0) Hannibal 1.0a            2683
                                   100.0 ( 83.5 :  16.5) Deep Junior 12           2679
                                   100.0 ( 84.0 :  16.0) Toga II 1.4 beta5c BB    2670
                                   100.0 ( 91.0 :   9.0) Umko 1.1 SSE42           2631
                                   100.0 ( 91.0 :   9.0) Jonny 4.00 32b           2616
                                   100.0 ( 94.5 :   5.5) Loop 13.6/2007           2614


Elostat:

DR4 Exp61V2                   :1900 (+1221,=473,-206), 76.71 %

Zappa Mexico II               : 100 (+ 75,= 19,-  6), 84.5 %
Toga II 1.4 beta5c BB         : 100 (+ 85,= 13,-  2), 91.5 %
Protector 1.3.2 JA            : 100 (+ 75,= 21,-  4), 85.5 %
Naum 4.2                      : 100 (+ 63,= 27,- 10), 76.5 %
Loop 13.6/2007                : 100 (+ 85,= 13,-  2), 91.5 %
Deep Onno 1-2-70              : 100 (+ 75,= 21,-  4), 85.5 %
Deep Shredder 12              : 100 (+ 58,= 32,- 10), 74.0 %
Hannibal 1.0a                 : 100 (+ 70,= 28,-  2), 84.0 %
Jonny 4.00 32b                : 100 (+ 77,= 16,-  7), 85.0 %
Deep Junior 12                : 100 (+ 77,= 18,-  5), 86.0 %
Deep Sjeng c't 2010 32b       : 100 (+ 61,= 28,- 11), 75.0 %
Gull 1.1                      : 100 (+ 61,= 27,- 12), 74.5 %
Critter 0.90 SSE42            : 100 (+ 48,= 28,- 24), 62.0 %
Umko 1.1 SSE42                : 100 (+ 86,= 11,-  3), 91.5 %
spark-1.0 SSE42               : 100 (+ 59,= 30,- 11), 74.0 %
******* ***                   : 100 (+ 25,= 29,- 46), 39.5 %
HIARCS 13.2 MP 32b            : 100 (+ 65,= 25,- 10), 77.5 %
Komodo64 1.3 JA               : 100 (+ 49,= 40,- 11), 69.0 %
Stockfish 2.01 JA             : 100 (+ 27,= 47,- 26), 50.5 %

Deep Rybka 4                  :1900 (+1152,=608,-140), 76.63 %

Zappa Mexico II               : 100 (+ 69,= 26,-  5), 82.0 %
Toga II 1.4 beta5c BB         : 100 (+ 72,= 24,-  4), 84.0 %
Protector 1.3.2 JA            : 100 (+ 68,= 27,-  5), 81.5 %
Naum 4.2                      : 100 (+ 50,= 42,-  8), 71.0 %
Loop 13.6/2007                : 100 (+ 89,= 11,-  0), 94.5 %
Deep Onno 1-2-70              : 100 (+ 74,= 21,-  5), 84.5 %
Deep Shredder 12              : 100 (+ 59,= 39,-  2), 78.5 %
Hannibal 1.0a                 : 100 (+ 77,= 20,-  3), 87.0 %
Jonny 4.00 32b                : 100 (+ 83,= 16,-  1), 91.0 %
Deep Junior 12                : 100 (+ 70,= 27,-  3), 83.5 %
Deep Sjeng c't 2010 32b       : 100 (+ 58,= 37,-  5), 76.5 %
Gull 1.1                      : 100 (+ 57,= 38,-  5), 76.0 %
Critter 0.90 SSE42            : 100 (+ 33,= 53,- 14), 59.5 %
Umko 1.1 SSE42                : 100 (+ 83,= 16,-  1), 91.0 %
spark-1.0 SSE42               : 100 (+ 58,= 34,-  8), 75.0 %
******* ***                   : 100 (+ 21,= 42,- 37), 42.0 %
HIARCS 13.2 MP 32b            : 100 (+ 58,= 35,-  7), 75.5 %
Komodo64 1.3 JA               : 100 (+ 47,= 45,-  8), 69.5 %
Stockfish 2.01 JA             : 100 (+ 26,= 55,- 19), 53.5 %


Much lower draw rate for the Setting. It won more games with an impressive attacking style, but it lost as much additionaly ...

Even if the individual results are very different, the overall score was only 1.5 points or 0.07894...% better than DR4 default. Errorbar at 1900 games is 15 Elo. Interesting is, that even if there is virtually zero difference, 61 would come out 10 ELo over the default with Bayeselo.

2 DR4 Exp61V2              2962   15   15  1900   77%  2754   25%
3 Deep Rybka 4             2952   11   11  3600   77%  2745   30%

This is just because the default setting has 1700 games more and the performance there was worse ... So a Rating list can cheat, a direct comparision with same openings for ALL game-sets and engines is much more accurate. Developers should keep this in mind if they care for 10 Elo improvements - always test identical and not into a rating list.

As it is hard to see any difference and by far nothing of 60 or 80 Elo I will terminate the testing.

Bye
Ingo
Parent - By Banned for Life (Gold) Date 2011-01-12 19:40
Thanks for running the tests! It's always good to see independent testing from someone who knows how to generate statistically valid results.

Although there wasn't a significant increase in Elo from the new settings, the decrease in draw percentage is still very useful in many applications, so I still consider Mindbreakers work to be very useful.
Parent - - By mindbreaker (****) Date 2011-01-13 08:52
I am running a completely identical condition tournament: The Tournament of the Champions.  It is only half-way complete.  I suspect the best setting, whatever that is, to be roughly 30 Elo stronger than the default Rybka 4 settings rather than 60.  But whatever happens, happens.

I'd be interested in what your endgame conditions were.  The combination of the resistance to trading and generous use of time early on in games means that endgame tables become very important.  If the tables are on a regular hard drive it will hurt it.  If they are on flash it should do much better.  I am running 3-4-5-man on a flash memory card and 32 Gb of 6-man on a USB thumb drive.  No tables should be on disk for access by the engine...that slows things down dramatically.

The tables become important because there is less time remaining near the end of the games but the engine has invested a lot of time to get better but complex endgames or just plain complex endgames.  If it is bogged down by slow tables on a platter hard drive, it is more likely to make major blunders.

In fact, from what I have seen, it is what the settings achieve in the endgame in the messes they make that brings home the points even in inferior but complex endgames and near endgame positions.  Of course, it is really just the regular strength of R4 that is converting these endings, but the settings set it up for doing what it does best, especially promotion.

There are so many things I want to try:  it all depends on what the final finishing order is of the TOC.  I defiantly want to see the effect of other timings and also tablebase look-up frequency.  But if the finishing order is a surprise then I will follow up those to see if there is still room to improve on piece values.

It is not just strength...these engines play some very attractive chess.  There is a wide variety of styles played by the experiments.  Exp 31 plays very interestingly as does Exp 18.  I also like Exp 79 against weaker engines.  People are missing out if they are not seeing the games these things conjure up and at no cost in strength.
Parent - - By Ingo (***) Date 2011-01-13 13:18
Hi

>I'd be interested in what your endgame conditions were.


It is all on my website which I am 'cant' mention here ...

But in short:

I use just 4 piece table bases, 64 MB cache. In the past I used a USB stick as a mapped web-drive to load them, at the moment I put the Tbs on the local HD.
But actually with 4pc Tbs that doesn't matter, you only get a few access and then the 4pcs are completely in the windows cache and there is no access on the HD at all. Basically the 4pc Tbs work as memory based bit-bases then (just better as the DTM is in)

When you run 'ponder on' games you might have problems with two engines accessing the Tbs at the same drive and at the same time (Regardless if HD or stick) as they compete about the same resource ... That is the main reason why I started with 4pc Tbs - to avoid conflicts.

Bye
Ingo
Parent - By mindbreaker (****) Date 2011-01-14 08:36
4-man and 3-man are fairly small, you might have enough RAM to put them on a virtual RAM drive.  The access and read is so fast that conflicts should be lower in probability.

I suspect only using 4-man and less and it being on disk is hurting the Exps but I suppose a lot of people are running that sort of arrangement so it is still a real world result.
Parent - - By mindbreaker (****) Date 2011-01-15 05:01 Edited 2011-01-15 05:04
Perhaps you can use 2 identical computers and use the built-in auto-2.3.2 support in most chess GUIs.  Then you can have separate tables on each machine without conflict. You can also double the threads.

I think most people are using 3-4-5-man sets.  http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=20422  That will fit on an 8 GB USB thumb drive.  It does not have to have really fast read...it is the access time that is most important and random read rates of small files.  It is hard to get that info.  The best rule of thumb is to get ones made with chips with the smallest manufacture process. 

We are talking $15 each.  It also extends the life of your hard drives.

For SSD the best is Intel for tables but the prices are not friendly compared to little thumb drives.
Parent - - By Ingo (***) Date 2011-01-15 09:25
Hi

>Perhaps you can use 2 identical computers and use the built-in auto-2.3.2 support in most chess GUIs.  Then you can have separate tables on each machine without >conflict. You can also double the threads.


I do not see any good reasoning in testing 2,3,4 ... cores. I am interested in the engine development and SMP is just a smaller part of it. Adding 40 Elo for eacht doubeling of the cores in my list is enough for me. This might not be that accurat, as some engines add 50 while others do just 30 but none of the engines will get into a "new ball park" as you americans say. So ... wasted energy, I concentrate on a reasonable number of games in a reasonable amount of time.

>I think most people are using 3-4-5-man sets.  http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=20422  That will fit on an 8 GB USB thumb drive.  It does >not have to have really fast read...it is the access time that is most important and random read rates of small files.  It is hard to get that info.  The best rule of thumb
>is to get ones made with chips with the smallest manufacture process.
>We are talking $15 each.  It also extends the life of your hard drives.
>For SSD the best is Intel for tables but the prices are not friendly compared to little thumb drives.


I know all this but:
Years ago there was a bigger test about the elo increase of an engine with 5pc Tbs and without. The outcome was 0. That was with ordinary hard drives so I agree that there might be a few elo with the five piece Tbs on a USB stick (I give 5 Elo) ... but that is to NO Tbs at all. So from 4 to 5 The difference might be 2#elo , maybe 3 Elo (I doubt that). In both cases the difference is that small, that it is impossible to "prove". As it is impossible for me to pay in $ I will not invest the 10€ for each of my computers for a theoretical increase which is smaller than the resolution of any public rating list in the world.
The only argument you have is, that it is less stressing for the harddrives ... but then, with the 4 pieces there are just a few accesses each game, than they are in windows cache, which is as good as a Ramdrive ... and I could simply install a ramdrive for the 30MB to save even the 10€ - but I havent done this for years now ...
Parent - By mindbreaker (****) Date 2011-01-15 23:51
I guess I like better looking games more.  One can see 50 Elo more.  One can defiantly see 96.4 Elo...the average difference between 4 threads and 1 thread on the CCRL 40/4 for the top 10 engines where both numbers are available.  Sure this is 4x rather than 2x but 4 thread or even 6 would be possible with ponder if two machines are used.

As for how much tables contribute, that requires testing.  I think at short time controls it is bigger, especially for weaker engines.
Parent - By mindbreaker (****) Date 2011-01-14 20:03
I guess I am not allowed to link to it, but Exp 42 just got 11.5/12 to go through the next stage at SWCR
Champions-League 2011

That is being played at a long time control.
Parent - - By mindbreaker (****) Date 2011-01-15 03:21 Edited 2011-01-15 03:26
TOC update after 8 opponents:

Effects on ratings (128807 game set):

1  Rybka 4 x64 Exp 61 v2    3331  2610
2  Rybka 4 x64 Exp 51v1    3327  2421
3  Rybka 4 x64 Exp 79 v1    3327  2801
4  Rybka 4 x64 Exp 24 v2    3325  2533
5  Rybka 4 x64 Exp 31 v1    3326  2525
6  Rybka 4 x64 Exp 71 v1    3326  2516
7  Rybka 4 x64 Exp 75 v1    3319  2803
8  Rybka 4 x64 Exp 42 v1    3317  3693
9  Rybka 4 x64 Exp 18 v1    3312  4241
10  Rybka 4 x64 Forum v1    3306  2361
11  Rybka 4 x64 Exp 8 v1    3301  2512
12  Deep Rybka 4 x64    3294  4346

And here are ratings derived internally from the tournament (14400 games):

Rybka 4 x64 Exp 51v1    3306
Rybka 4 x64 Exp 79 v1    3305
Rybka 4 x64 Exp 24 v2    3305
Rybka 4 x64 Exp 42 v1    3305
Rybka 4 x64 Forum v1    3302
Rybka 4 x64 Exp 61 v2    3302
Rybka 4 x64 Exp 31 v1    3300
Rybka 4 x64 Exp 18 v1    3300
Deep Rybka 4 x64    3294
Rybka 4 x64 Exp 71 v1    3292
Rybka 4 x64 Exp 75 v1    3287
Rybka 4 x64 Exp 8 v1    3279
Rybka 3    3239
Stockfish 1.9 JA 64bit    3238
Critter 0.90 64-bit    3173
spark-1.0    3091
Deep Shredder 12    3076
Deep Fritz 11    3063
HIARCS 12 MP    3035
Naum 4 4t    2999


There is still a lot of room for things to change because only 8 of 14 engines have played against the competitors.
Parent - By mindbreaker (****) Date 2011-01-15 04:10
If we look at the second ratings table, the results pose a question: Why after Default is put at its CCRL 40/4 rating are the opponents on average 44 Elo under rated when compared to the CCRL 40/4?

Sorry about column alignment...never seems to work.

Rybka 3    3239  3260
Stockfish 1.9 JA 64bit    3238  3244
Critter 0.90 64-bit    3173  3228
spark-1.0    3091  3112
Deep Shredder 12    3076  3140
Deep Fritz 11    3063   3096 est.*
HIARCS 12 MP    3035  3034
Naum 4 4t    2999  3158
    Ave 3114.25  Ave 3159
     
      44.75 Elo

Any theories?

*Had to use an adjusted CEGT 4/40 rating because despite my complaint of Apr 23, 2009, CCRL have yet to provide a Deep Fritz 11 rating for 40/4.
Parent - By mindbreaker (****) Date 2011-01-18 00:11
TOC update after 9 opponents:

Effects on ratings (130607 game set):

1  Rybka 4 x64 Exp 61 v2  3331  2760
2  Rybka 4 x64 Exp 51v1  3327  2571
3  Rybka 4 x64 Exp 79 v1  3326  2951
4  Rybka 4 x64 Exp 31 v1  3325  2675
5  Rybka 4 x64 Exp 71 v1  3324  2666
6  Rybka 4 x64 Exp 24 v2  3324  2683
7  Rybka 4 x64 Exp 75 v1  3321  2953
8  Rybka 4 x64 Exp 42 v1  3316  3843
9  Rybka 4 x64 Exp 18 v1  3314  4391
10  Rybka 4 x64 Forum v1  3306  2511
11  Rybka 4 x64 Exp 8 v1  3301  2662
12  Deep Rybka 4 x64  3294  4496

And here are ratings derived internally from the tournament (16200 games):

1  Rybka 4 x64 Exp 51v1    3305  1350
2  Rybka 4 x64 Exp 18 v1    3303  1350
3  Rybka 4 x64 Exp 79 v1    3303  1350
4  Rybka 4 x64 Exp 61 v2    3302  1350
5  Rybka 4 x64 Exp 24 v2    3301  1350
6  Rybka 4 x64 Exp 42 v1    3301  1350
7  Rybka 4 x64 Forum v1    3299  1350
8  Rybka 4 x64 Exp 31 v1    3298  1350
9  Deep Rybka 4 x64    3294  1350
10  Rybka 4 x64 Exp 75 v1    3290  1350
11  Rybka 4 x64 Exp 71 v1    3289  1350
12  Rybka 4 x64 Exp 8 v1    3279  1350
13  Rybka 3    3239  1800
14  Stockfish 1.9 JA 64bit    3237  1800
15  Critter 0.90 64-bit    3172  1800
16  spark-1.0    3090  1800
17  Deep Shredder 12    3075  1800
18  Deep Fritz 11    3062  1800
19  HIARCS 12 MP    3034  1800
20  Zappa Mexico II    3012  1800
21  Naum 4 4t    2998  1800


Exp 18 managed to regain most of its losses from last round.
Parent - - By mindbreaker (****) Date 2011-01-20 06:26 Edited 2011-01-20 06:30
TOC update after 10 opponents:

Effects on ratings (132407 game set):

1  Rybka 4 x64 Exp 61 v2  3332  2910
2  Rybka 4 x64 Exp 79 v1  3327  3101
3  Rybka 4 x64 Exp 24 v2  3326  2833
4  Rybka 4 x64 Exp 71 v1  3326  2816
5  Rybka 4 x64 Exp 31 v1  3325  2825
6  Rybka 4 x64 Exp 51v1  3325  2721
7  Rybka 4 x64 Exp 75 v1  3321  3103
8  Rybka 4 x64 Exp 42 v1  3316  3993
9  Rybka 4 x64 Exp 18 v1  3313  4541
10  Rybka 4 x64 Forum v1  3305  2661
11  Rybka 4 x64 Exp 8 v1  3303  2812
12  Deep Rybka 4 x64  3294  4646

And here are ratings derived internally from the tournament (18000 games):

1  Rybka 4 x64 Exp 79 v1  3304  1500
2  Rybka 4 x64 Exp 61 v2  3304  1500
3  Rybka 4 x64 Exp 24 v2  3304  1500
4  Rybka 4 x64 Exp 51v1  3302  1500
5  Rybka 4 x64 Exp 31 v1  3299  1500
6  Rybka 4 x64 Exp 42 v1  3299  1500
7  Rybka 4 x64 Exp 18 v1  3299  1500
8  Rybka 4 x64 Forum v1  3296  1500
9  Deep Rybka 4 x64  3293  1500
10  Rybka 4 x64 Exp 71 v1  3292  1500
11  Rybka 4 x64 Exp 75 v1  3291  1500
12  Rybka 4 x64 Exp 8 v1  3283  1500
13  Rybka 3  3239  1800
14  Stockfish 1.9 JA 64bit  3238  1800
15  Critter 0.90 64-bit  3172  1800
16  spark-1.0  3090  1800
17  Deep Shredder 12  3075  1800
18  Deep Fritz 11  3062  1800
19  HIARCS 12 MP  3034  1800
20  Zappa Mexico II  3012  1800
21  Komodo64 1.3 JA  3012  1800
22  Naum 4 4t  2998  1800


Komodo 1.3 appears to be a nice improvement over Komodo 1.1
Appears to be a dead heat at the top.
Only 4 opponents to go.
Parent - By mindbreaker (****) Date 2011-01-20 06:35 Edited 2011-01-20 06:43
I get an improvement of 28Elo between Komodo 1.1 and Komodo 1.3.  The 1.1 rating is based on 6433 games and the 1.3 rating on 1800 games.  But in essence, these are against one opponent: Rybka 4, but there is some variety in the styles because of the differences in the variants...but still... it will be more limited compared to what else is out there. 

It will be interesting to see what it can do when multithreading is implemented.
Parent - - By mindbreaker (****) Date 2011-01-23 23:17
TOC update after 11 opponents:

Effects on ratings (134207 game set):

1  Rybka 4 x64 Exp 61 v2    3330  3060
2  Rybka 4 x64 Exp 79 v1    3325  3251
3  Rybka 4 x64 Exp 31 v1    3325  2975
4  Rybka 4 x64 Exp 24 v2    3325  2983
5  Rybka 4 x64 Exp 71 v1    3325  2966
6  Rybka 4 x64 Exp 51v1    3324  2871
7  Rybka 4 x64 Exp 75 v1    3320  3253
8  Rybka 4 x64 Exp 42 v1    3315  4143
9  Rybka 4 x64 Exp 18 v1    3312  4691
10  Rybka 4 x64 Exp 8 v1    3304  2962
11  Rybka 4 x64 Forum v1    3304  2811
12  Deep Rybka 4 x64    3293  4796

And here are ratings derived internally from the tournament (19800 games):

1  Rybka 4 x64 Exp 24 v2    3305  1650
2  Rybka 4 x64 Exp 61 v2    3304  1650
3  Rybka 4 x64 Exp 79 v1    3304  1650
4  Rybka 4 x64 Exp 51v1    3304  1650
5  Rybka 4 x64 Exp 31 v1    3302  1650
6  Rybka 4 x64 Exp 42 v1    3301  1650
7  Rybka 4 x64 Forum v1    3298  1650
8  Rybka 4 x64 Exp 18 v1    3298  1650
9  Rybka 4 x64 Exp 71 v1    3294  1650
10  Deep Rybka 4 x64    3294  1650
11  Rybka 4 x64 Exp 75 v1    3292  1650
12  Rybka 4 x64 Exp 8 v1    3287  1650
13  Rybka 3    3242  1800
14  Stockfish 1.9 JA 64bit    3241  1800
15  Critter 0.90 64-bit    3175  1800
16  spark-1.0    3092  1800
17  Deep Shredder 12    3077  1800
18  Deep Fritz 11    3064  1800
19  HIARCS 12 MP    3036  1800
20  Zappa Mexico II    3014  1800
21  Komodo64 1.3 JA    3013  1800
22  Naum 4 4t    2999  1800
23  Deep Sjeng c't 2010    2960  1800


Exp 24 has taken the lead, but it is still very close at the top. 
3 opponents to go: Gull, Protector, and Thinker.
Parent - - By Vempele (Silver) Date 2011-01-24 09:46
How about normalizing for Rybka 4 default = 0 Elo?

(also: posting with the Post button so we don't have to use Ctrl+F <today, yesterday, ...> to find the latest results. And starting a new thread when this one reaches 300 posts).
Parent - - By Uly (Gold) Date 2011-01-24 10:16

> How about normalizing for Rybka 4 default = 0 Elo?


Why? These results are only useful for analysis since parameters experiments use different TC settings. For games, you'd like to normalize with "Rybka 4 Forum v1" = 0 Elo (for the 5-10 elo that the TC settings are worth).

>posting with the Post button so we don't have to use Ctrl+F


(Here javascript teleports me to the relevant post.)
Parent - By Vempele (Silver) Date 2011-01-24 10:22 Edited 2011-01-24 10:25

> Why? These results are only useful for analysis since parameters experiments use different TC settings. For games, you'd like to normalize with "Rybka 4 Forum v1" = 0 Elo (for the 5-10 elo that the TC settings are worth).


Fine, whatever. Just so long as it's normalized to 0.

> (Here javascript teleports me to the relevant post.)


I mean, if you want to visit the thread more than once. Plus the forum shortcuts (asd) for stuff I pretty much never want to do conflict with my browser shortcuts for stuff I do all the time. Once in a while it gets so annoying I disable javascript for the site.
Parent - - By mindbreaker (****) Date 2011-01-24 20:33
There was a difference of opinion as to which setting should be considered "Default". 

Not sure what you mean by using the post button, unless you want a hundred different threads.  If there is a trick for getting a post to appear all the way to the left, I guess I will use it on the new post if I am told what it is.  Someone posted further to the left...what I was posting was not a reply to that and I did not figure out how to post again all the way to the left.  Right now, the TOC is together in one place, detaching the last three rounds would just make it hard to follow.

I do intend to start a new thread...probably when the TOC is complete.  I don't like the title of this one especially after this tournament.  I probably should have started another one at the begining of this tournament.

Also, if you log-in while you are on the main forum page, it shows all the new postings, so you can go directly to the latest post in this thread if and when there is a new one.
Parent - By Vempele (Silver) Date 2011-01-24 20:46

> Not sure what you mean by using the post button


The one at the top (and bottom) of the thread.
Parent - - By Uly (Gold) Date 2011-01-24 22:55
Oh yeah, why do we have two links called "Post" that do entirely different things?

Anyway, if you want your posts to appear at the very bottom of the page instead of at the middle of the thread with the increasing indentation, use



^at this page you're reading (not the Rybka Discussion page) the link at the bottom of the page that says Post, instead of the one that says Reply (there's another Post link at the top of the page).
Parent - - By mindbreaker (****) Date 2011-01-25 06:21
Ok. Thanks.  I will bear that in mind next time I want to continue at the bottom.
Parent - - By Uly (Gold) Date 2011-01-26 05:05

> I will bear that in mind next time I want to continue at the bottom.


Why don't you post your updates at the bottom so they are easier to find to people that revisit this thread?
Parent - By mindbreaker (****) Date 2011-01-26 06:34
The TOC is almost complete...it may as well all be together.  After it is complete, I will start a new thread.  In it I can have a link to the start and end of the TOC if people are interested in what preceded that post.
Parent - By mindbreaker (****) Date 2011-01-25 20:10
TOC update after 12 opponents:

Effects on ratings (136007 game set):

Rybka 4 x64 Exp 61 v2    3333  3210
Rybka 4 x64 Exp 79 v1    3327  3401
Rybka 4 x64 Exp 24 v2    3326  3133
Rybka 4 x64 Exp 31 v1    3327  3125
Rybka 4 x64 Exp 71 v1    3326  3116
Rybka 4 x64 Exp 51v1    3323  3021
Rybka 4 x64 Exp 75 v1    3322  3403
Rybka 4 x64 Exp 42 v1    3316  4293
Rybka 4 x64 Exp 18 v1    3314  4841
Rybka 4 x64 Forum v1    3306  2961
Rybka 4 x64 Exp 8 v1    3304  3112
Deep Rybka 4 x64    3294  4946

And here are ratings derived internally from the tournament (21600 games):

1  Rybka 4 x64 Exp 61 v2    3310  1800
2  Rybka 4 x64 Exp 24 v2    3308  1800
3  Rybka 4 x64 Exp 79 v1    3307  1800
4  Rybka 4 x64 Exp 42 v1    3306  1800
5  Rybka 4 x64 Exp 31 v1    3306  1800
6  Rybka 4 x64 Exp 51v1    3303  1800
7  Rybka 4 x64 Exp 18 v1    3302  1800
8  Rybka 4 x64 Forum v1    3301  1800
9  Rybka 4 x64 Exp 71 v1    3297  1800
10  Rybka 4 x64 Exp 75 v1    3296  1800
11  Deep Rybka 4 x64    3294  1800
12  Rybka 4 x64 Exp 8 v1    3287  1800
13  Rybka 3    3245  1800
14  Stockfish 1.9 JA 64bit    3244  1800
15  Critter 0.90 64-bit    3178  1800
16  spark-1.0    3096  1800
17  Deep Shredder 12    3081  1800
18  Deep Fritz 11    3068  1800
19  HIARCS 12 MP    3039  1800
20  Zappa Mexico II    3017  1800
21  Komodo64 1.3 JA    3017  1800
22  Naum 4 4t    3003  1800
23  Deep Sjeng c't 2010    2964  1800
24  Gull 1.1 x64    2955  1800


Just Protector and Thinker to go.
Parent - By mindbreaker (****) Date 2011-01-28 07:27
TOC update after 13 opponents:

Effects on ratings (137807 game set):

3  Rybka 4 x64 Exp 61 v2  3333  3360
4  Rybka 4 x64 Exp 79 v1  3329  3551
5  Rybka 4 x64 Exp 31 v1  3329  3275
7  Rybka 4 x64 Exp 24 v2  3328  3283
9  Rybka 4 x64 Exp 71 v1  3327  3266
13  Rybka 4 x64 Exp 51v1  3323  3171
16  Rybka 4 x64 Exp 75 v1  3322  3553
29  Rybka 4 x64 Exp 42 v1  3317  4443
34  Rybka 4 x64 Exp 18 v1  3315  4991
60  Rybka 4 x64 Forum v1  3306  3111
61  Rybka 4 x64 Exp 8 v1  3304  3262
68  Deep Rybka 4 x64  3294  5096

And here are ratings derived internally from the tournament (23400 games):

1  Rybka 4 x64 Exp 79 v1  3313  1950
2  Rybka 4 x64 Exp 24 v2  3313  1950
3  Rybka 4 x64 Exp 61 v2  3313  1950
4  Rybka 4 x64 Exp 31 v1  3312  1950
5  Rybka 4 x64 Exp 42 v1  3310  1950
6  Rybka 4 x64 Exp 18 v1  3306  1950
7  Rybka 4 x64 Exp 51v1  3305  1950
8  Rybka 4 x64 Forum v1  3302  1950
9  Rybka 4 x64 Exp 71 v1  3301  1950
10  Rybka 4 x64 Exp 75 v1  3298  1950
11  Deep Rybka 4 x64  3294  1950
12  Rybka 4 x64 Exp 8 v1  3290  1950
13  Rybka 3  3249  1800
14  Stockfish 1.9 JA 64bit  3247  1800
15  Critter 0.90 64-bit  3182  1800
16  spark-1.0  3099  1800
17  Deep Shredder 12  3084  1800
18  Deep Fritz 11  3071  1800
19  HIARCS 12 MP  3043  1800
20  Protector 1.4.0 x64 JA 4t  3032  1800
21  Komodo64 1.3 JA  3020  1800
22  Zappa Mexico II  3020  1800
23  Naum 4 4t  3006  1800
24  Deep Sjeng c't 2010  2967  1800
25  Gull 1.1 x64  2959  1800


Just one opponent to go: Thinker.
Parent - - By mindbreaker (****) Date 2011-02-01 00:15 Edited 2011-02-01 02:08
The 2100 round Tournament of the Champions is now complete.  Here are the final ratings and standings:

Effects on ratings (139607 game set):

3  Rybka 4 x64 Exp 61 v2  3333  3360
4  Rybka 4 x64 Exp 79 v1  3329  3551
5  Rybka 4 x64 Exp 31 v1  3329  3275
7  Rybka 4 x64 Exp 24 v2  3328  3283
9  Rybka 4 x64 Exp 71 v1  3327  3266
13  Rybka 4 x64 Exp 51v1  3323  3171
16  Rybka 4 x64 Exp 75 v1  3322  3553
29  Rybka 4 x64 Exp 42 v1  3317  4443
34  Rybka 4 x64 Exp 18 v1  3315  4991
60  Rybka 4 x64 Forum v1  3306  3111 (3-100-150 timings other settings Default)
61  Rybka 4 x64 Exp 8 v1  3304  3262
68  Deep Rybka 4 x64  3294  5096 (The Default Deep Rybka 4 settings)
90  Rybka 3  3256  8192
92  Stockfish 1.9 JA 64bit  3252  5900
103  Critter 0.90 64-bit  3197  1800
109  spark-1.0  3114  1800
110  HIARCS 12 MP  3104  10092
111  Naum 4 4t  3099  16614
113  Deep Fritz 11  3092  10273
114  Deep Shredder 12  3088  7774
117  Protector 1.4.0 x64 JA 4t  3047  1800
118  Komodo64 1.3 JA  3035  1800
120  Zappa Mexico II  3018  7859
122  Deep Sjeng c't 2010  2980  2300
123  Gull 1.1 x64  2973  1800
129  Thinker54AInert-MP64-UCI  2940  1800

And here are ratings derived internally from the tournament (25200 games):

1  Rybka 4 x64 Exp 79 v1  3317  2100
2  Rybka 4 x64 Exp 24 v2  3315  2100
3  Rybka 4 x64 Exp 31 v1  3314  2100
4  Rybka 4 x64 Exp 42 v1  3312  2100
5  Rybka 4 x64 Exp 61 v2  3311  2100
6  Rybka 4 x64 Exp 18 v1  3307  2100
7  Rybka 4 x64 Exp 51v1  3308  2100
8  Rybka 4 x64 Exp 71 v1  3304  2100
9  Rybka 4 x64 Forum v1  3300  2100 (3-100-150 timings other settings Default)
10  Rybka 4 x64 Exp 75 v1  3301  2100
11  Deep Rybka 4 x64  3294  2100 (The Default Deep Rybka 4 settings)
12  Rybka 4 x64 Exp 8 v1  3290  2100
13  Rybka 3  3251  1800
14  Stockfish 1.9 JA 64bit  3249  1800
15  Critter 0.90 64-bit  3184  1800
16  spark-1.0  3101  1800
17  Deep Shredder 12  3086  1800
18  Deep Fritz 11  3073  1800
19  HIARCS 12 MP  3044  1800
20  Protector 1.4.0 x64 JA 4t  3034  1800
21  Zappa Mexico II  3022  1800
22  Komodo64 1.3 JA  3022  1800
23  Naum 4 4t  3008  1800
24  Deep Sjeng c't 2010  2969  1800
25  Gull 1.1 x64  2960  1800
26  Thinker54AInert-MP64-UCI  2938  1800

The second list is probably more accurate between Rybka 4 settings. 
There is however the mater of the opponents being underrated by about 50 Elo when Rybka 4 Default is set at 3294.  There are also many losses on time by the Exps.  The losses are nearly all from using too much time on the very last move of the time control.  I have not thrown out any of these losses.  It might be interesting to see the ratings with them thrown out...to see if the finishing order would change.  The probability of an Exp loosing this way should not very as they all have the same timings but statistically strange things can happen.  Some might have been relatively unlucky.

As far as the outcome...I was surprised that the Elo advantage is only 23 Elo but Default could have been lucky, there is still a margin of error.  It performed roughly 50 Elo above expectation which is why all the opponents are underrated.  Whether this is just good luck or due to some aspects of the conditions of the tournament is not clear.  There are some differences...I suppose.  The time control was 2 main stage with a third with an increment to reflect the time controls of long time control games but just with less time.   And I had 3-4-5 man tables and some 6-man on flash memory.  I don't think the shorter time control itself was a large factor as my machine is faster than the standard machine for CCRL 40/4 though it could account for some of it...but 50 Elo...I highly doubt it.

The order of the engines is also a bit of a surprise particularly the low finish of Exp 71 and Exp 75.  It was also interesting to see Deep Shredder 12 finish higher than Deep Fritz 11...something that should have happened in the larger set but never did despite many thousands of games.  Exp 24's high finish is also interesting.  Exp 61's fall to 5th is also puzzling...but at the top the margin of error could change things radically.  2100 games may sound like a lot but the differences between settings can be very small.

I am not sure if this is the largest one-section computer tournament ever with 25,200 games, but it is probably close.

I will see about giving more details and statistics later.  Perhaps I will run another rating list generator or two.
Parent - By mindbreaker (****) Date 2011-02-01 01:26 Edited 2011-02-01 01:28
I thought this might be interesting.  I took the games from just the top 7 opponents and bottom 7 opponents and made separate tournament tables.  Exps vs the top 7 is on the left.  Exps vs bottom 7 is on the right.

Ignore the titles.
Parent - - By mindbreaker (****) Date 2011-02-01 22:14 Edited 2011-02-02 02:18
Here is the ratings result from bayselo on the 25,200 game TOC set:

Rank     Name                           Elo    +    - games score oppo. draws
   1 Rybka 4 x64 Exp 79 v1       139   15   15  2099   80%  -106   20%
   2 Rybka 4 x64 Exp 31 v1       135   15   15  2100   80%  -106   21%
   3 Rybka 4 x64 Exp 24 v2       134   15   15  2100   80%  -106   21%
   4 Rybka 4 x64 Exp 61 v2       132   15   15  2100   80%  -106   21%
   5 Rybka 4 x64 Exp 42 v1       130   15   14  2100   80%  -106   22%
   6 Rybka 4 x64 Exp 51v1        126   15   14  2100   79%  -106   23%
   7 Rybka 4 x64 Exp 71 v1       123   15   14  2100   79%  -106   22%
   8 Rybka 4 x64 Exp 18 v1       123   15   14  2100   79%  -106   24%
   9 Rybka 4 x64 Exp 75 v1       122   15   14  2100   79%  -106   22%
  10 Rybka 4 x64 Forum v1        109   14   14  2099   79%  -106   29%
  11 Deep Rybka 4 x64            103   14   14  2099   78%  -106   30%
  12 Rybka 4 x64 Exp 8 v1        101   14   14  2099   78%  -106   29%
  13 Rybka 3                      67   13   13  1800   41%   123   39%
  14 Stockfish 1.9 JA 64bit       64   13   13  1800   41%   123   35%
  15 Critter 0.90 64-bit           3   14   14  1800   32%   123   33%
  16 spark-1.0                   -78   15   15  1796   23%   123   25%
  17 Deep Shredder 12            -87   15   15  1800   21%   123   26%
  18 Deep Fritz 11               -94   15   15  1800   20%   123   28%
  19 HIARCS 12 MP               -126   16   16  1800   18%   123   23%
  20 Protector 1.4.0 x64 JA 4t  -139   16   16  1800   17%   123   20%
  21 Komodo64 1.3 JA            -145   16   16  1800   16%   123   21%
  22 Zappa Mexico II            -145   16   17  1800   16%   123   21%
  23 Naum 4 4t                  -161   17   17  1800   15%   123   19%
  24 Deep Sjeng c't 2010        -202   18   19  1800   12%   123   15%
  25 Gull 1.1 x64               -207   18   19  1800   12%   123   15%
  26 Thinker54AInert-MP64-UCI   -228   19   20  1800   10%   123   14%

Not sure why 4 games seem to be missing said "4 game(s) with unknown result ignored."

This places Exp 79's performance at 36 Elo higher than Default.
Parent - By InspectorGadget (*****) Date 2011-02-07 19:38
mindbreaker,

I'm busy playing around with your settings. I'm also adjusting some of them for test purposes. I'm running them against the strongest available engine now. I think that Exp 79 is quite interesting. I will see how it does against... you know what :lol:
- - By Uly (Gold) Date 2010-11-20 19:06
Hi, just passing around to thank you for the settings, I used Experiment 38 settings to win, or get a winning position in this correspondence game that I call the most interesting game I've played, due to its beauty when analyzing, the great variations that were a possibility to play, and that most of it I thought it was a dead draw but kept going and suddenly a winning variation jumped to my face, that is, I don't know at what point black blundered.

Vytron Vs. x64 Round 2

1. e4 c5 2. Nf3 d6 3. d4 Nf6 4. Nc3 cxd4 5. Nxd4 Bd7 6. Bg5 Nc6 7. Qd2 a6 8. O-O-O b5 9. Nxc6 Bxc6 10. Qe3 Rc8 11. Kb1 Nd7 12. Nd5 h6 13. Bh4 g5 14. Bg3 Bg7 15. h4 Bxd5 16. exd5 Qb6 17. Qe2 Nc5 18. Re1 Rc7 19. Qe3 g4 20. f3 O-O 21. Bf4 Qa5 22. Qd2 b4 23. Bc4 Na4 24. Bb3 Bxb2 25. Bxh6 Bf6 26. Qf4 Rfc8 27. Qxg4+ Kh7 28. Qh5 Nc3+ 29. Kc1 Bg7 30. Bd2+ Kg8 31. Bxc3 Bxc3 32. Re2 Kf8 33. Kd1 Qa3 34. Re4 Qb2 35. Qh6+ Bg7 36. Qc1 Qf6 37. Ke1 a5 38. Kf2 Rc3 39. Qd2 Bh6 40. Qe2 Bf4 41. g3 Be5 42. Kg2 Qg6 43. Rh3 f5 44. Rc4 R8xc4 45. Bxc4 a4 46. Bd3 Qf7 47. h5 Qxd5 48. h6 b3 49. axb3 a3 50. Bc4 Qd4 51. Rh5 e6 52. h7 Re3 53. Qf2 Rc3 54. Qxd4 Bxd4 55. h8=R+ Bxh8 56. Rxh8+ Kg7 57. Ra8 Rxc2+ 58. Kh3 d5 59. Bb5 Ra2 60. g4 Kf6 61. Kg3 Ra1 62. Ra6 a2 63. Kf4 Rg1 64. Rxa2 e5+ 65. Ke3
1-0

8/8/5k2/1B1ppp2/6P1/1P2KP2/R7/6r1 b - -


Here things go a bit blurry, I offered 7 takebacks to my opponent and he accepted, but then he resigned, but then we agreed to replay in the far future a game from after 59. Bb5 to see if it's really won.

The thing is that, what I called Rybka MindBreaker (100, 150, 3, 2, 10, 8, 10, 8, 10, 8, 17, 13, 40, 32) guided me to promising variations while all the other engines had some 0.00 score all along, and suggested easy draws.

It's evidence that contempt can be successfully used for analysis of corr games!
Parent - By mindbreaker (****) Date 2010-11-20 22:51
:)
- - By fustuk25 Date 2010-11-28 03:41
can somebody tell me plz the values of experiment 67?
Parent - By mindbreaker (****) Date 2010-11-28 06:01
I will post parameters of Experiments 66-74 at the conclusion of this cycle.  Is 67 your favorite number? ;)
Up Topic Rybka Support & Discussion / Rybka Discussion / Parameters Experiment 38: 64 Elo over R4 default
1 2 3 4 5 6 Previous Next  

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill