I think there is a correlation between when you can you expect from something when I know their internals vs someone that doesn’t know but is not like who knows internals is much much better.
Example: many people created websites without a clue of how they really work. And got millions of people on it. Or had crazy ideas to do things with them.
At the same time there are devs that know how internals work but can’t get 1 user.
pc manufacturers never were able to even imagine what random people were able to do with their pc.
This to say that even if you know internals you can claim you know better, but doesn’t mean it’s absolute.
Sometimes knowing the fundamentals it’s a limitation. Will limit your imagination.
I'm a big fan of the concept of 初心 (Japanese: Shoshin aka "beginners mind" [0] ) and largely agree with Sazuki's famous quote:
> “In the beginner’s mind there are many possibilities, but in the expert’s there are few”
Experts do tend to be limited in what they see as possible. But I don't think that allows carte blanche belief that a fancy Markov Chain will let you transcend humanity. I would argue one of the key concepts of "beginners mind" is not radical assurance in what's possible but unbounded curiosity and willingness to explore with an open mind. Right now we see this in the Stable Diffusion community: there are tons of people who also don't understand matrix multiplication that are doing incredible work through pure experimentation. There's a huge gap between "I wonder what will happen if I just mix these models together" and "we're just a few years from surrendering our will to AI". None of the people I'm concerned about have what I would consider an "open mind" about the topic of AI. They are sure of what they know and to disagree is to invite complete rejection. Hardly a principle of beginners mind.
Additionally:
> pc manufacturers never were able to even imagine what random people were able to do with their pc.
Belies a deep ignorance of the history of personal computing. Honestly, I don't think modern computing has still ever returned to the ambition of what was being dreampt up, by experts, at Xerox PARC. The demos on the Xerox Alto in the early 1970s are still ambitious in some senses. And, as much as I'm not a huge fan, Gates and Jobs absolutely had grand visions for what the PC would be.
I think this is what is blunted by mass education and most textbooks. We need to discover it again if we want to enjoy our profession with all the signals flowing from social media about all the great things other people are achieving. Staying stupid and hungry really helps.
I think this is more about mechanistic understanding vs fundamental insight kind of situation. The linear algebra picture is currently very mechanistic since it only tells us what the computations are. There are research groups trying to go beyond that but the insight from these efforts are currently very limited.
However, the probabilistic view is very much clearer. You can have many explorable insights, both potentially true and false, by jıst understanding the loss functions, what the model is sampling from, what is the marginal or conditional distributions are and so on. Generative AI models are beautiful at that level. It is truly mind blowing that in 2025, we are able to sample from the megapixel image distributions conditioned on the NLP text prompts.
If you dig ml/vision papers from old, you will see that formulation-wise they actually did, but they lacked the data, compute, and the mechanistic machinery provided by the transformer architecture. The wheels of progress are slow and requires many rotations to finally reach somewhere.
Example: many people created websites without a clue of how they really work. And got millions of people on it. Or had crazy ideas to do things with them.
At the same time there are devs that know how internals work but can’t get 1 user.
pc manufacturers never were able to even imagine what random people were able to do with their pc.
This to say that even if you know internals you can claim you know better, but doesn’t mean it’s absolute.
Sometimes knowing the fundamentals it’s a limitation. Will limit your imagination.