> Yes, but at least now we're comparing artificial to real neural networks, so the way it works at least has a chance of being similar.
Indeed, and I wasn't even saying it's wrong, it may be pretty close.
> What would be cool would be if we could to some "mechanistic interpretability" work on the brain's language generation circuits, and perhaps discover something similar to induction heads.
Yeah, I wouldn't be surprised. And maybe the more we find out about the brain, it could lead to some new insights about how to improve AI. So we'd sort of converge from both sides.
Indeed, and I wasn't even saying it's wrong, it may be pretty close.
> What would be cool would be if we could to some "mechanistic interpretability" work on the brain's language generation circuits, and perhaps discover something similar to induction heads.
Yeah, I wouldn't be surprised. And maybe the more we find out about the brain, it could lead to some new insights about how to improve AI. So we'd sort of converge from both sides.