Upshot - poetry expertise does not seem to be the primary focus these days, perhaps to the detriment of the entire world. We did move on from training scaling to “test time” scaling (which I hate as a name btw), Ilya does not seem to have been needed, (although I am really curious what he’s building).
My prediction that you want to be deeply embedded and really rich and part of global infrastructure feels good. My suggestion that oAI / MS would be able to use the lead in 2024 to extend was wrong.
Neither of us talked much about coding as a product that would drive value and behavior, which is super interesting to me, we were probably six months from seeing real competence of any sort there way back in June 2024.
We both seemed to think there would be a single breakout company, or could be one, (although I did suggest buying the basket), clearly not the case with GOOG oAI and Anthropic all posting serious revenues this last quarter / year.
One area of Anthropic that was nascent in 2024, but that I have come to think is super valuable is their mechinterp group. I still don’t see work done by other labs (at least published) to nearly the quality of Anthropic. And the group has clearly moved into a period of productivity; there’s a good chance in my mind it could provide a truly enduring strategic advantage as a tool to be used by the taste makers steering the ship. In 2024, interpretability seemed almost impossible to get a handle on — today, the sustained chipping away at the problem makes a lot more look possible.
Mechinterp in general is just completely undervalued right now (and agreed Anthropic's team is doing the most rigorous work, now accompanied by Goodfire). They're doing the closest work to neuroscience's in vivo 'thought-tracing', which is just the most wild science fiction sort of thing to be working on, and yet I feel the average person has no idea this sort of work is happening. When combined with the idea of the 'universal subspace hypothesis' (explored under the paper of the same name), you really start to bridge the gap from engineering to something more philosophical and spiritual. But I digress...
Ya, super interesting research area the authors explored of basically trying to answer the question: "Is there a canonical/intrinsic way that concepts/representations/information are 'stored' in the universe/reality?".
They tested that by performing "spectral analysis of over 1100 models - including 500 Mistral-7B LoRAs, 500 Vision Transformers, and 50 LLaMA-8B models ... by applying spectral decomposition techniques to the weight matrices of various architectures", and concluding that "deep neural networks trained across diverse tasks exhibit remarkably similar low-dimensional parametric subspaces", showing that "neural networks systematically converge to shared spectral subspaces regardless of initialization, task, or domain".
Not just philosophically interesting but also has practical implications for being smarter about how to reuse models, model merging, developing more sustainable training and inference algos, etc.
Did you also talk about "head and shoulders" and "pennant" patterns in stock charts? Or where the "smart money" is at? I'd like to subscribe to your paid newsletter.
This is a super low quality comment. I try to put my current thoughts out here largely because high quality comments refine my thinking.
What about the original conversation seems correct / incorrect / interestingly correct or incorrect to you? What about my summary now seems correct/incorrect, etc?
Ultimately the big value is to me — I get to look back at some dialogue and then check where I had it right, wrong and interestingly wrong. My personal hypothesis is that doing this a lot compounds in a good way.
Upshot - poetry expertise does not seem to be the primary focus these days, perhaps to the detriment of the entire world. We did move on from training scaling to “test time” scaling (which I hate as a name btw), Ilya does not seem to have been needed, (although I am really curious what he’s building).
My prediction that you want to be deeply embedded and really rich and part of global infrastructure feels good. My suggestion that oAI / MS would be able to use the lead in 2024 to extend was wrong.
Neither of us talked much about coding as a product that would drive value and behavior, which is super interesting to me, we were probably six months from seeing real competence of any sort there way back in June 2024.
We both seemed to think there would be a single breakout company, or could be one, (although I did suggest buying the basket), clearly not the case with GOOG oAI and Anthropic all posting serious revenues this last quarter / year.
One area of Anthropic that was nascent in 2024, but that I have come to think is super valuable is their mechinterp group. I still don’t see work done by other labs (at least published) to nearly the quality of Anthropic. And the group has clearly moved into a period of productivity; there’s a good chance in my mind it could provide a truly enduring strategic advantage as a tool to be used by the taste makers steering the ship. In 2024, interpretability seemed almost impossible to get a handle on — today, the sustained chipping away at the problem makes a lot more look possible.