This is an important clarification; from the abstract and title I was super confused how they identified a "subspace" that could be consistently identified across model structures (I was assuming they meant that they saw stability in the dimension of the weight subspace or something), but if they're just referring to one model class that clears things up substantially. It's definitely also a much weaker result IMO, basically just confirming that the model's loss function has a well-posed minima, which...duh? I mean I guess I'm glad someone checked that, but called it "the universal weight subspace hypothesis" seems a bit dramatic.
Basic rule of MLE is to have guardrails on your model output; you don't want some high-leverage training data point to trigger problems in prob. These guardrails should be deterministic and separate from the inference system, and basically a stack of user-defined policies. LLMs are ultimately just interpolated surfaces and the rules are the same as if it were LOESS.
> correction that wipes out the incumbents who absolutely are working to maintain the masqerade
You need to also have a robust alternative that grows quickly in the cleared space. In 2008 we got a correction that cleared the incumbents, but the ensuing decade of policy choices basically just allowed the thing to re-grow in a new form.
I thought we pretty explicitly bailed out most of the incumbents. A few were allowed to be sacrificed, but most of the risk wasn't realized, and instead rolled into new positions that diffused it across the economy. 2008's "correction" should have seen the end of most of our investment banks and auto manufacturers. Say what you want to about them (and I have no particular love for either), but Tesla and Bitcoin are ghosts of the timeline where those two sectors had to rebuild themselves from scratch. There should have been more, and Goldman Sachs and GM et al. should not currently exist.
> A few were allowed to be sacrificed, but most of the risk wasn't realized, and instead rolled into new positions that diffused it across the economy.
Yeah that's a more accurate framing, basically just saying that in '08 we put out the fire and rehabbed the old growth rather than seeding the fresh ground.
> Tesla and Bitcoin are ghosts of the timeline where those two sectors had to rebuild themselves from scratch
I disagree, I think they're artifacts of the rehab environment (the ZIRP policy sphere). I think in a world where we fully ate the loss of '08 and started in a new direction you might get Tesla, but definitely not TSLA, and the version we got is really (Tesla+TSLA) IMO. Bitcoin to me is even less of a break with the pre-08 world; blockchain is cool tech but Bitcoin looks very much "Financial Derivatives, Online". I think an honest correction to '08 would have been far more of a focus on "hard tech and value finance", rather than inventing new financial instruments even further distanced from the value-generation chain.
> Goldman Sachs and GM et al. should not currently exist.
I would say yes and no on Tesla. Entities that survived becaue of the rehab environment actually expected it to fail, and shorted it heavily. TSLA as it currently exists is a result of the short squeeze on the stock that ensued when it became clear that the company was likely to become profitable. Its current, ridiculous valuation isn't a product of its projected earnings, but recoil from those large shorts blowing up.
In our hypothetical alternate timeline, I imagine that there would have still been capital eager to fill the hole left by GM, and possibly Ford. Perhaps Tesla would have thrived in that vacuum, alongside the likes of Fisker, Mullen, and others, who instead faced incumbent headwinds that sunk their ventures.
Bitcoin, likewise, was warped by the survival of incumbents. IIUC, those interests influenced governance in the early 2010s, resulting in a fork of the project's original intent from a transactional medium that would scale as its use grew, to a store of value, as controlled by them as traditional currencies. In our hypothetical, traditional banks collapsed, and even survivors lost all trust. The trustless nature of Bitcoin, or some other cryptocurrency, maybe would have allowed it to supercede them. Deprived of both retail and institutional deposits, they simply did not have the capital to warp the crypto space as they did in the actual 2010s.
I call them "ghosts" because, yes, whatever they might have been, they're clearly now just further extensions of that pre-2008 world, enabled by the our post-2008 environment (including ZIRP).
"In 2008 we got a correction that cleared the incumbents,"
I thought in 2008 we told the incumbents "you are the most important component of our economy. We will allow everybody to go down the drain but you. That's because you caused the problem, so you are the only ones to guide us out of it"
> This is what Thiel’s apocalyptic geopolitics looks like in practice: a twisted military-industrial eschatology where an AI-powered genocide is understood to be “restraining” rather than enacting the end of the world.
I could have used some more explication on the connection between Thiel's ideology and Palantir's project portfolio. I felt like this article was structured like "Part 1: Thiel is Crazy, Part 2: Palantir is Awful, Conclusion: They are Related", without really making clear what the relationship between them was. It seems pretty contradictory that someone concerned about "The New One World Order" would create a global police technology apparatus, so deep-diving into the cognitive dissonance there (and how it is soothed by the ideology) would have been interesting (to me).
Okay :) I actually do have a pretty deep familiarity with all of this and that's why I'm criticizing the article. It felt like a rehash of known facts (Thiel is a christian fascist, his company Palantir does innovatively horrible stuff), rather than drawing new connections between those known facts. At a fine-grain I want to know how his ideology underpins the business decisions of his company, I don't need someone to gesture at the two together and mumble something about eschatology.
> At a fine-grain I want to know how his ideology underpins the business decisions of his company, I don't need someone to gesture at the two together and mumble something about eschatology.
I think we (and probably HN in general) aren't the target audience here. Last time I met up with some non-tech friends, Peter Thiel came up a lot in the pub chat - a year ago I'm not sure they would have known who he is. The more people outside of tech who understand what a deranged loon he really is, the better.
> With the emergence of AI in science, we are witnessing the prelude to a curious inversion – our human ability to instrumentally control nature is beginning to outpace human understanding of nature, and in some instances, appears possible without understanding at all.
A while ago I read "Against Method" by Paul Feyerabend and there's a section that really stuck with me, where he talks about the "myth" of Galileo. His point is that Galileo serves as sort of the mythological prototype of a scientist, and that by picking at the loose ends of the myth one can identify some contradictory elements of the popular conception of "scientific method". One of his main points of contention is Galileo's faith in the telescope, his novel implementation of bleeding edge optics technology. Feyerebend argues that Galileo invented the telescope as primarily a military invention, it revolutionized the capabilities of artillery guns (and specifically naval artillary). Having secured his finances with some wealthy patrons, he then began to hunt for nobler uses of his new tool, and landed on astronomy.
Feyerabend's point (and what I'm slowly working up to) is that applying this new (and untested) military tool to what was a very ancient and venerable domain of inquiry was actually kind of scandalous. Up until that point all human knowledge of astronomy had been generated by direct observation of the phenomenon; by introducing this new tool between the human and the stars Galileo was creating a layer of separation which had never been there before, and this was the source of much of the contemporary controversy that led to his original censure. It was one thing to base your cosmology on what could be detected by the human eye, but it seemed very "wrong" (especially to the church) to insert an unfeeling lump of metal and glass into what had before been a very "pure" interaction, which was totally comprehensible to the typical educated human.
I feel like this article is expressing a very similar fear, and I furthermore think that it's kind of "missing the point" in the same way. Human comprehension is frequently augmented by technologly; no human can truly "understand" a gravitational wave experientially. At best we understand the n-th order 'signs' that the phenomenon imprints on the tools we construct. I'd argue that LLMs play a similar role in their application in math, for example. It's about widening our sensor array, more than it is delegating the knowledge work to a robot apprentice.
Fascinating point, and one I think can definitely apply here.
Though there is a key difference – Galileo could see through his telescope the same way, every time. He also understood what the telescope did to deliver his increased knowledge.
Compare this with LLMs, which provide different answers every time, and whose internal mechanisms are poorly understood. It presents another level of uncertainty which further reduces our agency.
> Though there is a key difference – Galileo could see through his telescope the same way, every time.
Actually this is a really critical error- a core point of contention at the time was that he didn't see the same thing every time. Small variations in the lens quality, weather conditions, and user error all contributed to the discovery of what we now call "instrument noise" (not to mention natural variation in the astronomical system which we just couldn't detect with the naked eye, for example the rings of Saturn). Indeed this point was so critical that it led to the invention of least-squares curve fitting (which, ironically, is how we got to where we are today). OLS allowed us to "tame" the parts of the system that we couldn't comprehend, but it was emphatically not a given that telescopes had inter-measurement reliability when they first debuted.
LLMs can be deterministic machines, you just need to control the random seeds and run it on the same hardware to avoid numerics differences.
Gradient descent is not a total black box, although it works so well as to be unintuitive. There is ongoing "interpretability" research too, with several key results already.
Deterministic doesn't necessarily mean that can be understood by an human mind. You can think about a process entirely deterministic but so complex and with so many moving parts (and probably chaotic) that a humble human cannot comprehend.
I have been meaning to read Feyerband for a while but never did. I think Against Method sounds like a good starting point.
Did Feyerband also not argue that Galileo's claim that Copernicus's theory was proved was false given it was not the best supported hypothesis by the evidence available at the time.
I very much agree with your last paragraph. Telescopes are comprehensible.
> Did Feyerband also not argue that Galileo's claim that Copernicus's theory was proved was false
My reading of AM was that it's less about what's "true" or "false" and more about how the actual structure of the scientific argument compares to what's claimed about it. The (rough) point (as I understand it) is that Galileo's scientific "findings" were motivated by human desires for wealth and success (what we might call historically contingent or "poltical" factors) as much as they were by "following the hard evidence".
> Telescopes are comprehensible.
"Comprehensible" is a relative measure, I think. Incomprehensible things become comprehensible with time and familiarity.
> I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
This is the new line now that LLMs are being commoditized, but in the post-Slate Star Codex AI/Tech Accelerationist era of like '20-'23 the Pascal's wager argument was very much a thing. In my experience it's kind of the "true believer" argument, whereas the ROI/productivity thing is the "I'm in it for the bag" argument.
The adoption and use of technology frequently (even typically) has a political axis, it's kind of just this weird world of consumer tech/personal computers that's nominally "apolitical" because it's instead aligned to the axis of taste/self-identity so it'll generate more economic activity.
