Category: Go

KataGo and ChatGPT Failures

AlphaGo was an amazing breakthrough and very impressive in its ability to win against professional go players. It was really surprising that evaluating a go position using neural nets and machine learning would work as well as it did. And yet it kept beating professionals.

But now, some cracks are showing. In their excellent paper “Adversarial Policies Beat Superhuman Go AIs” (, Tony Wang, Adam Gleave, et al. are using an adversarial approach to figure out techniques that work against KataGo. And indeed: after a few tries, I managed to kill a huge group and win, and KataGo did not see it coming until too late. It doesn’t realize that its circular group that surrounds a dead group with a large eye needs enough outside liberties to actually remove that dead group.

Katago losing
KataGo doesn’t realize the dead white group has more liberties than the surrounding black group

This technique takes advantage of KataGo (1) not knowing enough about liberties and (2) not knowing enough about the topology of blocks on the go board. When training its model, there are special inputs only for 1, 2, or 3 liberties. There’s also no explicit concept of adjacent blocks; the neural net has to learn that concept from the board position. And the way the neural net is trained, it doesn’t create good enough abstractions for those concepts. In normal games, this is sufficient to play better than any human, but in corner cases, it falls apart.

I would argue that blocks of stones, liberty counts, and race to capture are an essential part of the underlying model you need when playing go (see e.g. Richard Hunter’s book “Counting Liberties and Winning Capturing Races”). And machine learning (at least the way we’re doing it now) is not a great way to build that model. You’ll end up with gaps of knowledge and approximations that will fall down at critical points.

Against that background, the failures of ChatGPT make more sense. Machine learning didn’t build a model of the world, it just learned to put words together in a way that seems to make sense. Often impressive, but a lot of recent examples demonstrate that it doesn’t actually understand what’s going on.

And machine learning for self-driving cars is also based on lots of inputs, but only a very limited model of the world. Like KataGo, it will fail in corner cases. And that’s scary.

SmartGo Too Strong

SmartGo One incorporates KataGo, a really strong go engine. For most players, it’s too strong. While you can increase handicap until you manage to win, getting beaten mercilessly is not that enjoyable.

Smartgo computer levels

The newest version tries to make playing against the computer more interesting. When you start a new game, you can choose between five different levels, from ‘SmartGo++’ (full strength, playing as well as it can) to ‘SmartGo—’ (making more mistakes and even some blunders). The default is the middle of the road ‘SmartGo’ that will try to make the game more balanced.

The trick with dumbing down computer play is that you don’t want the computer to play non-sensical moves. The approach I’ve taken is that SmartGo will still select one of the moves that KataGo generates, but instead of always picking the best move, allow it to choose moves that lose points or reduce its chance of winning. If the computer is further ahead, it can lose more, bringing the game closer to balance.

This is a first stab at this feature, I will likely refine this in versions to come. Please let me know how this works for you: Do you always keep the computer at maximum strength? Is the lowest level still way too strong? Do you enjoy playing against the computer more?

Read UGF, NGF, and GIB files

SmartGo One can now read go games in file formats other than SGF: UGF, NGF, and GIB (with file extensions .ugi, .ngf, and .gib). UGF is popular in Japan, NGF is used by the Korean wBaduk server, and GIB is used by the Korean Tygem server.

SmartGo One will show thumbnails for these files, with the file extension indicated in the lower right corner.

Thumbnails of 4 games in different file formats

Support for these file formats is read-only. If you make changes to the game, it will be saved as SGF, overwriting the imported file. Paste SGF will also accept games in those formats.

Unfortunately, I don’t have the spec for any of these file formats, so the files are interpreted based on reverse engineering as well as comments at Sensei’s Library. I’d love to get more sample files so I can make sure my code handles all the corner cases. And if you run into files that are not handled as well as they could be, please send me a copy at Thanks!

Also, please let me know if I should revive my old code for reading the Ishi (.go) file format. Is anybody still using that one?

Analyze Game

Version 1.3 of SmartGo One adds a much requested feature: analyzing a whole game. I’ve also added a score graph so you can easily see where your game went off the rails. This is a bare-bones first implementation of that feature, will be refined in future versions.

To start a game analysis, tap and hold on the Hint icon, then choose Analyze Game. SmartGo will start with a quick pass through the game, then go back through spending more time on each move. Tap the Hint icon again to stop the analysis.

Analyze game popup
Tap and hold on Hint icon

The score graph shows the score lead for Black or White so you can easily see lead changes. It’s currently using a log scale to better show small changes when the game is close.

Analyze game score graph
Score graph in SmartGo One

You can also have it display the winning percentage. It uses a black or white horizontal line to indicate the projected winner – definitely a minimalist implementation at this point.

Analyze game winrate
Score and winning percentage

You can also tap and hold on the tree view to see the whole score graph (most useful on the iPhone).

Analyze game whole view
Tap and hold on tree view

This version also adds Spanish localization (thanks to Alejandro Montiel), reduces the app size by 200 MB to just 53 MB (by downloading large KataGo models on demand), and fixes some bugs, in particular one where empty text labels were wrongly written out in SGF.

More Useful Hints

Tap on the Hint icon, wait a few seconds, and a dot on the board shows you where KataGo thinks you should play. This minimalist version of Hint is now a thing of the past.

The newest version of SmartGo One makes Hint much more useful:

(1) It can show you multiple top moves that KataGo is considering. It uses the familiar color scheme from Lizzie: teal for the top move, green for other good moves, brown for moves that were considered but are not as good.

Smartgo hint top moves

(2) It can show you the principal expected move sequence.

Smartgo hint sequence

(3) If you like more information density, you can see top move choices and move sequence at the same time.

Smartgo hint both

(4) You can easily switch between these display modes with a long press on the Hint icon.

Smartgo hint menu

(5) You can give KataGo more time: Settings > Computer Play > Time for Hint. If you set it to ‘Unlimited’, tap on the Hint icon again to stop computing.

Smartgo hint time for hint

(6) If you only see part of the board and the best move is off-screen, SmartGo pans the board to bring the best hint move into view.

Smartgo hint pan visible

(7) There’s now a keyboard shortcut for Hint: Cmd-K (think KataGo).

(8) Hint is now available on the toolbar in Replay mode, not just Play.