Not logged inRybka Chess Community Forum
Up Topic The Rybka Lounge / Computer Chess / Komodo 13-Ending Evaluation Wrong!
- - By cghori (**) Date 2019-07-03 13:21
Looks like Komodo 13 does not have some basic EG knowledge given its evaluation of this ending.

See attached image.
Attachment: Komodo13b.jpg (166k)
Parent - - By MrKris (**) Date 2019-07-04 08:30
Komodo's evaluation was just a few pawns until it shot up a few seconds before this:
[Event "?"]
[Site "?"]
[Date "2019.07.04"]
[Round "?"]
[White "Komodo-13.02"]
[Black ""]
[Result "*"]
[BlackElo "2000"]
[Time "00:13:56"]
[WhiteElo "2200"]
[TimeControl "0+300"]
[SetUp "1"]
[FEN "k7/8/pK6/P7/8/4B3/8/8 w - - 0 1"]
[Termination "unterminated"]
[PlyCount "63"]
[WhiteType "program"]
[BlackType "program"]

1. Kxa6 {Komodo-13.02, analysis PV, no Syzygy, 1.76 Gnodes, 55 plies,
time 01:47, evaluation +20.28} Kb8 2. Kb5 Ka8 3. a6 Kb8 4. a7+ Ka8 5. Kb4 Kb7 6. Kc3
Ka8 7. Kc4 Kb7 8. Kb4 Ka8 9. Bg1 Kb7 10. Bf2 Ka8 11. Bb6 Kb7 12. Ka4 Ka8
13. Bd4 Kb7 14. Bg1 Ka8 15. Bb6 Kb7 16. Bc5 Ka8 17. Kb4 Kb7 18. Bd4 Ka8 19.
Kc3 Kb7 20. Bc5 Ka8 21. Bg1 Kb7 22. Bf2 Ka8 23. Bb6 Kb7 24. Kb4 Ka8 25. Kb3
Kb7 26. Be3 Ka8 27. Bg1 Kb7 28. Bf2 Ka8 29. Kc3 Kb7 30. Kc4 Ka8 31. Bc5 Kb7
32. Be3 *

Stockfish, no Syzygy, was stuck at the 58/97 00:54 line until Analyze was pushed off at 9:03.
Arena's Variation Board/Add to Game adds the PV to the game, as shown.

- Stockfish's depth 'froze' under 100 plies, even though it got to 98 very quickly, which would be a draw -- some sort of dysfunction.

- Stockfish's last several moves in its PV were K+B vs. K insufficient material -- a 2nd dysfunction because its still +0.13.
Parent - - By cghori (**) Date 2019-07-04 11:49
Hopefully Kaufman can fix this glitch in Komodo 14.
Parent - By MrKris (**) Date 2019-07-04 17:38 Edited 2019-07-04 18:00
KomodoMCTS 13.02 :

- Syzygy does not work in MCTS, I have 5 & 6 and before 1.Kxa6 it is a 5-man draw.
Here Arena instantly says "15 TB hits"  then no more and with no effect. (Of course "MCTS off", Komodo regular, with Syzygy its instantly 0.00.)

- Ko.MCTS's evaluations, all legal moves (the 2 stalemates it gave 0.00) :
started low, then peaked at 4.08, then at a half hour had declined to 2.14 with 1.Kxa6 the worst at 1.93.
FEN: k7/8/pK6/P7/8/4B3/8/8 w - - 0 1
amdphIIx6, Syzygy 5 & 6, 2G MCTS hash
KomodoMCTS-13.02 --15 TB hits--
PV=13 - best at bottom -----------------------------------------------------------------------
 33/23  34:04   8,702k  4k   0.00  1.Kc7
 33/23  34:04   8,702k  4k   0.00  1.Bf4
 33/23  34:04   8,702k  4k  +1.93  1.Kxa6 Kb8 2.Bc5 Ka8 3.Bg1 Kb8 4.Bh2+ Ka8 5.Kb5 Kb7 6.Bg3
 33/23  34:04   8,702k  4k  +2.13  1.Bh6 Kb8 2.Kxa6 Ka8 3.Kb5 Kb8 4.Kb6 Ka8 5.Bg7 Kb8 6.Be5+ Ka8 7.Kb5
 33/23  34:04   8,702k  4k  +2.13  1.Bc5 Kb8 2.Kxa6 Ka8 3.Kb5 Kb8 4.Kb6 Ka8 5.Bd4 Kb8 6.Be5+ Ka8 7.Kb5
 33/23  34:04   8,702k  4k  +2.14  1.Bc1 Kb8 2.Kxa6 Ka8 3.Kb5 Kb8 4.Ka4 Ka7 5.Be3+ Ka8 6.a6
 33/23  34:04   8,702k  4k  +2.13  1.Bg5 Kb8 2.Kxa6 Ka8 3.Kb5 Kb8 4.Bf4+ Kb7 5.a6+ Ka8 6.Ka5
 33/23  34:04   8,702k  4k  +2.14  1.Bd2 Kb8 2.Kxa6 Ka8 3.Kb5 Kb8 4.Ka4 Ka7 5.Kb4 Kb7 6.Bc3
 33/23  34:04   8,702k  4k  +2.14  1.Kc5 Kb7 2.Bd4 Kc7 3.Be5+ Kb7 4.Bf6 Kc7 5.Bd4 Kb7 6.Bg7 Ka7 7.Kd6
 33/23  34:04   8,702k  4k  +2.16  1.Kc6 Kb8 2.Kc5 Kb7 3.Bd2 Kc7 4.Bc3 Kb7 5.Bd4 Kc7 6.Bb2 Kb7 7.Bg7 Ka7 8.Be5
 33/23  34:04   8,702k  4k  +2.14  1.Bf2 Kb8 2.Kc5 Kb7 3.Bd4 Kc7 4.Be5+ Kb7 5.Bc3 Ka7 6.Bd4 Kb7 7.Bg7 Ka7 8.Kb4
 33/23  34:04   8,702k  4k  +2.14  1.Bg1 Kb8 2.Kc5 Kb7 3.Bd4 Kc7 4.Be5+ Kb7 5.Bg7 Kc7 6.Bf6 Kb7 7.Bd8 Ka7 8.Bb6+ Ka8
 33/23  34:04   8,702k  4k  +2.14  1.Bd4 Kb8 2.Kc5 Kb7 3.Bc3 Kc7 4.Be5+ Kb7 5.Bb2 Kc7 6.Bg7 Kb7 7.Bh6 Ka7 8.Bf4
---------------------------------------------------------------------------
...
---------------------------------------------------------------------------
 26/16  02:18   319k  2k   0.00  1.Kc7
 26/16  02:18   319k  2k   0.00  1.Bf4
 26/16  02:18   319k  2k  +1.39  1.Bg1 Kb8 2.Kc5 Kb7 3.Kb4
 26/16  02:18   319k  2k  +1.51  1.Bc1 Kb8 2.Ba3 Ka8 3.Kxa6 Kb8 4.Bd6+ Ka8
 26/16  02:18   319k  2k  +1.46  1.Kc6 Kb8 2.Bg5 Ka7 3.Bh4 Ka8 4.Kb6
 26/16  02:18   319k  2k  +3.35  1.Bf2 Kb8 2.Be1 Ka8 3.Kxa6 Kb8 4.Bg3+ Ka8 5.Kb5 Kb7
 26/16  02:18   319k  2k  +3.42  1.Bd2 Kb8 2.Be1 Ka8 3.Kxa6 Kb8 4.Bg3+ Ka8 5.Kb5 Kb7 6.a6+
 26/16  02:18   319k  2k  +1.89  1.Bh6 Kb8 2.Bf8 Ka8 3.Kxa6 Kb8 4.Bd6+ Ka8
 26/16  02:18   319k  2k  +1.89  1.Bg5 Kb8 2.Bd8 Ka8 3.Kxa6 Kb8 4.Bg5 Ka8 5.Kb6
 26/16  02:18   319k  2k  +1.90  1.Kc5 Ka7 2.Kc6+ Kb8 3.Bf4+ Ka8 4.Bc1
 26/16  02:18   319k  2k  +1.93  1.Kxa6 Kb8 2.Bc5 Ka8 3.Bg1 Kb8 4.Bh2+ Ka8 5.Kb5 Kb7 6.Bg3
 26/16  02:18   319k  2k  +2.77  1.Bc5 Kb8 2.Bd4 Ka8 3.Kxa6 Kb8 4.Kb5 Kb7 5.a6+ Ka8 6.Bb6
 26/16  02:18   319k  2k  +4.08  1.Bd4 Kb8 2.Kxa6 Ka8 3.Bc5 Kb8 4.Bb4 Ka8 5.Kb5 Kb8 6.Bd6+ Ka8
---------------------------------------------------------------------------
...
---------------------------------------------------------------------------
 16/6  00:04   2k  467   0.00  1.Kc7
 16/6  00:04   2k  467   0.00  1.Bf4
 16/6  00:04   2k  467  +0.72  1.Bh6 Kb8 2.Bf4+ Ka8
 16/6  00:04   2k  467  +0.72  1.Bg5 Kb8 2.Bf4+ Ka8
 16/6  00:04   2k  467  +0.78  1.Bc5 Kb8 2.Bd6+ Ka8
 16/6  00:04   2k  467  +0.64  1.Bc1 Kb8 2.Bf4+ Ka8
 16/6  00:04   2k  467  +0.64  1.Bd4 Kb8 2.Be5+ Ka8
 16/6  00:04   2k  467  +0.64  1.Bf2 Kb8 2.Kc6 Ka8
 16/6  00:04   2k  467  +0.64  1.Bg1 Kb8 2.Bf2 Ka8
 16/6  00:04   2k  467  +0.64  1.Bd2 Kb8 2.Kc6 Ka7
 16/6  00:04   2k  467  +0.63  1.Kc6 Kb8 2.Bf4+ Ka8
 16/6  00:04   2k  467  +1.91  1.Kxa6 Kb8 2.Bc1 Ka8 3.Bb2 Kb8 4.Be5+
 16/6  00:04   2k  467  +1.90  1.Kc5 Kb7 2.Bf2 Ka8 3.Bg3
---------------------------------------------------------------------------
... first full depth:
 10/3  00:02   119  59   0.00  1.Kc7
 10/3  00:02   119  59  +0.05  1.Kxa6
 10/3  00:02   119  59  +0.05  1.Kc5
 10/3  00:02   119  59  +0.05  1.Bh6
 10/3  00:02   119  59  +0.05  1.Bg5
 10/3  00:02   119  59  +0.05  1.Bc5
 10/3  00:02   119  59   0.00  1.Bf4
 10/3  00:02   119  59  +0.65  1.Bd4 Kb8 2.Be5+ Ka8
 10/3  00:02   119  59  +0.65  1.Bc1 Kb8 2.Bf4+ Ka8
 10/3  00:02   119  59  +0.64  1.Bf2 Kb8 2.Bg3+ Ka8
 10/3  00:02   119  59  +0.64  1.Bd2 Kb8 2.Bf4+ Ka8
 10/3  00:02   119  59  +0.64  1.Bg1 Kb8 2.Bh2+ Ka8
 10/3  00:02   119  59  +0.64  1.Kc6 Kb8 2.Bf4+ Ka8
Parent - By MrKris (**) Date 2019-07-05 02:47 Edited 2019-07-05 02:52
Thanks for the 'wrong color bishop' position!

Its hard for me to be optimistic about any engine making any substantial improvements though.

Top A/B, or CPU, engines still do not know what a win or advantage actually is.
In the screenshot Komodo is closest, it started with only +2.91 at 1 second.
After 1 hour, 2 cores each simultaneously AMDPhIIX6 Sy.6.
Billions of nodes each with at most a check if the black king moves to a dark square, nothing else legally possible (except if white's B moves, ...a4-a3?? bxa3 letting the N to b2).

So now it looks like neural-net "zero knowledge" might beat 61 years of programmers' A/B efforts after only 2 years for Lc0.
https://en.wikipedia.org/wiki/Computer_chess
...
1957 – The first programs that can play a full game of chess are developed, one by Alex Bernstein[3] and one by Russian programmers using a BESM.
1958 – NSS becomes the first chess program to use the alpha-beta search algorithm.
1962 – The first program to play credibly, Kotok-McCarthy, is published at MIT.
1963 – Grandmaster David Bronstein defeats an M-20 running an early chess program.[4]
1966–67 – The first chess match between computer programs is played. Moscow Institute for Theoretical and Experimental Physics (ITEP) defeats Kotok-McCarthy at Stanford University by telegraph over nine months.
1967 – Mac Hack Six, by Richard Greenblatt et al. introduces transposition tables and becomes the first program to defeat a person in tournament play.
1968 – Scottish chess champion David Levy makes a bet with AI pioneers John McCarthy and Donald Michie that no computer program would win a chess match against him within 10 years.
...
1978 – David Levy wins the bet made 10 years earlier, defeating Chess 4.7 in a six-game match by a score of 4½–1½. ...
...
2017 – AlphaZero, a neural net-based digital automaton, beats Stockfish [8] 28–0, with 72 draws, in a 100-game match.
2019 – Leela Chess Zero (LCZero v0.21.1-nT40.T8.610) defeats Stockfish 19050918 in a 100 game match 53.5 to 46.5 for TCEC season 15 title.[40]

{Note: the early hardware was incredible, very roughly: 1958 a few tubes a bit each for "RAM", in the 60's a room sized IBM 360 about like a cell phone, etc.
- Yet that is what A/B was designed for!?}
Up Topic The Rybka Lounge / Computer Chess / Komodo 13-Ending Evaluation Wrong!

Powered by mwForum 2.27.4 © 1999-2012 Markus Wichitill