The problem is that ChatGPT doesn't really know letters, it writes in wordpieces (BPE), which may be one or more letters.

For example, something like "running" might get tokenizef like "runn"+"ing", being only two tokens for ChatGPT.

It'll learn to infer some of these things over the course of training, but limited.

Same reason it's not great at math.