*So the model thinks ahead but cannot reason about its own thinking in a real wa...

So the model thinks ahead but cannot reason about its own thinking in a real way. It is rationalizing, not rational.

My understanding is that we can’t either. We essentially make up post-hoc stories to explain our thoughts and decisions.