> Said another way, LIGO can't work because the ruler itself is squashing as space squashes, so you can't measure if space is compressing.
I think the chief confusion here is that you may be thinking the light arrives at the detector in the same amount of time regardless of spacetime curvature. That is, the the ruler is itself squishing.
But what needs to be considered is the constant speed of light.
This implies that what happens is, in the presence of additional curvature, and constant speed of light, the additional distance traveled will have appeared to slow the light.
In the laser interferometer this registers as interference.
It is also worth noting that any claims of detection are thoroughly investigated and confirmed with other detectors.
> It is difficult for a single LIGO detector to confirm a gravitational wave signal on its own. The initial discovery of gravitational waves required that the signal be seen in both detectors (Hanford and Livingston).
> It is also worth noting that any claims of detection are thoroughly investigated and confirmed with other detectors.
This is the thing I (as a layperson) can't really wrap my head around yet; there's gotta be so much interference from so many different sources, there's gotta be some impressive data processing going on to filter out anything not relevant to their core measurements, and then THAT data will have to be compared to that of other detectors.
This device, the LHC, space telescopes, really cool and all if you look at the published pop-sci results, but the actual data is like... individual photons captured by a worldwide network of detectors and processed / data analyzed into the first "photo" of a black hole.
You didn't misunderstand. A sudden stretching of space cannot change the number of peaks and troughs as they go past, but since those peaks and troughs are now slightly further apart, the frequency of the light is slightly lower, and the light takes slightly longer to travel. This applies to light that is already in transit. Of course, as the gravitational wave passes by, the length/frequency returns to normal again.
New light that is emitted at one end of the journey while there is a stretching status will have the same frequency as normal, and just see the longer journey. In fact, the gravitational waves that LIGO is able to detect are slower than the time it takes for light to make the journey, so the stretching is effectively gradual, and the detector is basically an extremely accurate length measurement. The gravity waves aren't fast enough to make the light changing frequency a thing that needs to be worried about.
> This Connection is Untrusted
Go Back
The owner of costantini.pw has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website.
I tried to see what it built for you, and this was the only url in your bio.
That's a personal profile page which has an expired certificate, sorry for that.
If you're (or anyone really) interested in what I did and would like to try it out for curiosity and feedback, please feel free to reach me at j@costantini.pw as I'd be pleased to share it. Thanks
> Is parallel programming hard? Without any further details or specifics, yes it is. It is far harder to conceptualize code instructions executing simultaneously, than one-at-a-time in a sequential order.
If I program (map inc [0 1 2 3]) is it really any more difficult to conceptualize the (inc ) function performing on each element sequentially than in parallel?
I think the difficulty of parallel programming is less innate and more two fold:
1) languages often default to sequential so to do async requires introducing additional primitives to the programmer
2) knowing when to effectively use parallel programming
When I have a list or stream that I know has independent elements that require wholly independent calculations then parallel programming is straightforward
Where people get hung up is trying to shoe horn async where it is either unnecessary (performance is equal or worse than sequential) or introduces breaking behavior (the computations are in fact interdependent).
(Fun fact: I once had someone call HR on me because they didn't know embarrassingly parallel was a technical term, and they thought I was belittling them)
That requires + to be associative. And scan is one of the core skeletons of parallel skeletons, so obviously if you express everything as parallel skeletons, parallel programming remains manageable.
I agree that if we define the individual instructions to always be wholly independent, then sure, it is more straightforward.
While I'd probably argue that it is still more difficult to conceptualize, the statement we're discussing is presented as broad and general. I'd call it far less misleading if it said something like:
There is a common myth in software development that parallel programming *has* to be hard.
> Or by "async" do you just mean concurrent code? I'm reading "async" to mean lightweight coroutines or similar.
Yeah, my bad, I was utilizing a colloquial definition of a term that has a technical definition in a technical conversation. A lamentation lo the lossyness of language.
I guess I assumed we were talking about something other than in terms of red/blue because I'd argue red/blue's "hard"ness transcends myth to mathematical fact.
> Weird thing is it was designed to model language. It’s surprising that it returns sound answers as often as it does.
Is this surprising? Can you point to researchers in the field being “surprised” by LLMs returning sound answers?
> “surprising”, i.e. we don’t really know what happened.
This ie reads like a sort of popsci conclusion.
We know exactly what happened. We programmed it to perform these calculations. It’s actually rather straightforward elementary mathematics.
But, what happens is so many interdependent calculations grow the complexity of the problem until we are unable to hold it in it our minds, and to analyze its decisions computationally necessitates similar levels of computation for each decision being made as what was used to compute the weights.
As for its effectiveness, familiarity with the field of computational complexity points to high dimensional polynomial optimization problems being broadly universal solvers.
> Is this surprising? Can you point to researchers in the field being “surprised” by LLMs returning sound answers?
It's surprising because it wasn't the intent of LLMs. LLMs are just predictive models that guess the most likely next word. Having the results make sense was never a priority. Early version, GPT1/2, all return mostly complete nonsense. It was only with GPT3 when the model got large enough that it started returning results that are convincing and might even make sense often enough.
Even more mind boggling is the fact that randomness is part of its algorithm, i.e. temperature, and that without it the output is kind of meh.
> It's surprising because it wasn't the intent of LLMs. LLMs are just predictive models that guess the most likely next word. Having the results make sense was never a priority.
If you took the same amount of data for the GPT3+ but scrambled it's tokenization before training THEN I would agree with you that its current behaviour is surprising, but the model was fed data that has large swaths that are literal question and answer constructions. It's over fitting behavior is largely why it's parent company is facing so much legal backlash.
> Even more mind boggling is the fact that randomness is part of its algorithm
The randomness is for token choice rather than any training time tunable so fails to support the "i.e. we don’t really know what happened" sentiment. We do know, we told it to flip a coin, and it did.
> i.e. temperature, and that without it the output is kind of meh.
Both without it and with it. You can turn up the temperature and get bad results as well as you can turn it down and get bad results.
If adding a single additional dimension to the polynomial of the solution space turned a nondeterministic problem into a deterministic one, then yes, I would agree with you, that would be surprising.
> so fails to support the "i.e. we don’t really know what happened" sentiment
It's less that we don't know what's happening on a micro-level but more that it's surprising that it's producing anything coherent at all on a macro-level - especially with a (necessary) element of randomness in the process.
For most part we don't seem particularly knowledgeable about what happens on a macro-level. Hallucinations remain an unsolved problem. AI companies can't even make their "guardrails" bulletproof.
I think this is an uncharitable reading of this thread.
I’m arguing against the breathless use of “surprising”.
My gp explains what I think you overlooked in this dismissive response.
> to analyze its decisions computationally necessitates similar levels of computation for each decision being made as what was used to compute the weights.
Explainable but intractable is still far from surprising for me.
> It's surprising because it wasn't the intent of LLMs. LLMs are just predictive models that guess the most likely next word. Having the results make sense was never a priority.
If you read through what Hinton or any of his famous students have said, it genuinely was and is surprising. Everything from AlexNet to the jump between GPT-2 to GPT-3 was surprising. We can't actually explain that jump in a formal way, just reasonable guesses. If something is unexplainable, it's unpredictable. Prediction without understanding is a vague guess and the results will come as a surprise.
>Is this surprising? Can you point to researchers in the field being “surprised” by LLMs returning sound answers?
Lol researchers were surprised by the mostly incoherent nonsense pre-transformer RNNs were spouting years go, nevermind the near perfect coherency of later GPT models. To argue otherwise is just plain revisionism.
> What made this result so shocking at the time was that the common wisdom was that RNNs were supposed to be difficult to train (with more experience I’ve in fact reached the opposite conclusion). Fast forward about a year: I’m training RNNs all the time and I’ve witnessed their power and robustness many times, and yet their magical outputs still find ways of amusing me.
This reads more like humanizing the language of the post then any legitimate surprise from the author.
The rest of the post then goes into great detail showing that “we DO really know what happened” to paraphrase the definition the op provides for their use of “surprise”.
> Conclusion We’ve learned about RNNs, how they work, why they have become a big deal, we’ve trained an RNN character-level language model on several fun datasets, and we’ve seen where RNNs are going.
I am pushing back on people conflating the innate complexity of a high dimensional polynomial with a misplaced reverence of incomprehensibility.
> In fact, it is known that RNNs are Turing-Complete in the sense that they can to simulate arbitrary programs (with proper weights).
Mathematically proven to be able to do something is about as far from surprise as one can get.
>This reads more like humanizing the language of the post then any legitimate surprise from the author.
Lol Sure
>I am pushing back on people conflating the innate complexity of a high dimensional polynomial with a misplaced reverence of incomprehensibility.
We don't know what the models learn and what they employ to aid in predictions. That is fact. Going on a grad descent rant is funny but ultimately meaninglessness. It doesn't tell you anything about the meaning of the computations.
There is no misplaced incomprehensibility because the internals and how they meaningfully shape predictions is incomprehensible.
>Mathematically proven to be able to do something is about as far from surprise as one can get.
Magic the gathering is turing complete. I'm sorry but "therotically turing complete" is about as meaningless as it gets. Transformers aren't even turing complete.
This comment from "50 days ago" absolutely rocked me then and continues to provide a sobering and depressing real life metric to the climate crisis. I've showed anyone who will listen this chart.
I am having trouble following your opinion through this thread.
> Your argument demonstrates the usefulness of mathematics, but does not demonstrate that it isn't a language.
What is “a language” to you? What is an “isn’t a language” to you?
I can grok you referring to axioms and lemmas as a “mathematical language”, but I see such as just the way we communicate something more essential and wholly independent of any need to have been communicated.
A lot of contemporary research mathematics is layered and wrought of “useful” complexities for its desired domain, but how do you dismiss the essential and seemingly unrealness of its abstraction from our perceived reality?
Counting is an example.
Subjective boundaries illuminate the essentialism of distinctness. 2 apples describe the same abstract phenomena as 2 atoms, or 2 galaxies, or 2 orientations of stereoisomers.
What is the “language” here? The word/symbol 2? The subjective boundary that separates something more continuous into discrete forms?
Transcendentals and irrationals alight my meditation on what the hell all this is that we’re experiencing.
You have a triangle with edges that terminate at each vertex, but if two of those edges have equal length than you can interpret their length as unit 1 where the third edge then has a length of (sqrt 2) which is a number without a finite decimal expansion.
What language can be used to defend an infinitesimal equating to a finite value?
This points at an essentialism to me.
Any amount of “language” is incapable of both explaining this completely or explaining it away.
Similar with pi and its relation to a circle which has a well defined circumference that somehow expresses itself with a number that is itself incapable of being expressed or defined.
As you brought up the incompleteness theorems, they too have a similar “infinite in finite” quality.
I am unsure how you can understand godel but argue against the essentialism of the sur-real abstractions he brings attention to.
> "Infinitesimal" is just an idea, as far as I know. Nothing real is infinitesimal.
The unreal (re: abstracted) aspect is what places it outside the confines of “language” for me.
Are black holes real? Do they have singularities? If yes, that can be an example of your “real” infinitesimal.
My opinion is that infinitesimals are more than real they are essential. They are the building blocks of all that is “real”.
Ultimately, what we’re talking about is a philosophical debate that would require one to step “outside” reality to confirm or deny outright so we are just providing our opinions on an unknowable concept.
What is “real” in this context?
Is pi “real”? Is the plank constant? The former was my path to the essentialism of infinitesimals. The latter my path to the essentialism of discrete counting.
The math leads us there but I don't think anyone is particularly happy about it.
> Is pi “real”?
¯\_(ツ)_/¯
> Is the plank constant?
¯\_(ツ)_/¯
Fuck man, I can't tell you if a quark is real. I'm also not aware of anyone who can. The best we got is our interpretation that the model being indistinguishable from the real thing might as well be the real thing. Metaphysics and metamathematics are mind bending areas that require a deep understanding of the non-meta concepts first.
But given all you've said, I highly suggest looking into the various set theories I mentioned previously. Specifically start with Finite ZF set theory and Peano Arithmetic, where you'll find you can indeed operate on such concepts as pi without infinities.
> What is “a language” to you? What is an “isn’t a language” to you?
A language is an abstract concept that describes a method of communication. It need not be spoken (such as English), written (such as what we're doing now). We frequently use body language to communicate, and so do many animals. We have braille, smoke signals, maritime flags, we communicate with knots on a string, and so many more things. You're right that language is quite a broad and vague thing. But recognize that all these things are also not of the universe, but of us humans (or similar of other animals). Something like English is something we may better refer to as a social construct, as it is a collective agreement, though body language may be a bit more ingrained but I still do not think you would refer to it as something other than language or something of the universe (distinct from us being of the universe in the trivial sense).
> What is the “language” here? The word/symbol 2? The subjective boundary that separates something more continuous into discrete forms?
(This is HN, so I'm going to assume you're familiar with programming languages.) If I give you these 14 characters (p, t, k, s, m, n, l, j, w, a, e, i, o, u) are we able to communicate? Maybe after some trial and error, but certainly not something we could throw into a translation machine. It'd be hard to call these even tokens since we have not distinguished consonants from vowels or if that even is a thing here, so we can't really lex. We need words, phrases, and context before you can even from syntax. Then we need to build our syntax, which is equally non trivial despite looking so (build a PL, it is a great exercise for any computer scientist or mathematician. For the latter, build your own group, ring, field, ideal, and algebra. You'd do this in an abstract algebra course). We need all this to really start making a real means to communicate. These are things we take for granted but are far more complex when we actually have to do them from scratch, forcing our hands.
Do you have a problem calling a programming language a language? I'd assume not because we collectively do so tautologically. Great, you agree that math is a language. Thank you lambda calculus. We can have an isomorphic relationship between programming languages and various mathematical systems. I'll point out here that there are different algebras and calculus with different rules and forms, though many that are not deep in mathematics may not be exactly familiar with these. I think this is often where the confusion arises, since we most often are using our descriptions that are most useful, just like how no one programs in brainfuck and just how most drawings are communicative visualizations rather than abstract art. I again remind you of Poincare who says that mathematics is not the study of numbers, but the study of relationships. He does not specify numbers in the latter part, on purpose. Category theory may be something you wish to take up in this case, as it takes the abstraction to the extreme. Speaking of which
> but how do you dismiss the essential and seemingly unrealness of its abstraction from our perceived reality?
I could ask you the same about English. Why is this any different? Is that because you are aware of other languages that people speak? Or is it because you recognize that these languages are a schema of encoding and decoding mechanisms which result in a lossy communication of information between different entities?
You discuss counting, but are not recognizing that you can not place an apple into text, nor atoms, galaxies, or stereoisomers. It is because mathematics is the map, the language, not the thing itself. We can duplicate these at will or modify them in any way. Math is not bound to physical laws like an apple is. Its bound is the same of the apple that exists in my mind, not in my hand. (If you want to make this argument in the future, a stronger one might revolve around discussion of primes and their invariances)
> What language can be used to defend an infinitesimal equating to a finite value?
If this is the essential part, I think this is probably the best point to focus on. Specifically because infinities are not real. Nor are they even numbers. If you disagree then you disagree with physics. Rather infinities are a conceptual tool that is extremely useful. But if we were able to count and use infinities then we'd have the capacity for magic via the Banach-Tarski Pardaox, and completely violate the no cloning theorem. But our universe does not appear to actually have arbitrary precision, rather our tool does due to its semantics. Maybe finite ZF is a better choice than ZF or ZFC set theory. Why not NBG which has a finite number of axioms or why not MK which isn't?
Infinities, singularities, and such are not things in our universe. You may point to a black hole but this would represent a misunderstanding of our understandings of them. We cannot peer in beyond the event horizon, which certainly is not a singularity and has real measurable and finite volume. It is what is inside that is the singularity. But can you say that this is not in fact just an error in the math? It wouldn't be the first time such a thing has happened. Maybe it is at the limits of our math and so thus is a result of the inconsistency of axiomatic systems? There are many people working on this problem, and I do not want to undermine their hard work, and neither should you.
You're biased because you're looking at how we use the tool rather than what the tool is itself. We use mathematics as the main descriptive language for science because of its precision. But we've also had to do a lot of work to ensure its consistency and make it more precise along the way. But I think you may have not been exposed to the levels of abstraction that math has, as this is not seen by most people until well beyond a calculus class.
> I am unsure how you can understand godel but argue against the essentialism of the sur-real abstractions he brings attention to.
And I cannot see the reverse. Are you saying that the universe is incomplete? Are you saying that the universe is not consistent? This sounds like a better argument for the idea that the universe is a simulation (as in we are being simulated, not as in you can represent and draw parallels between the universe and simulation. The former begs the question "on what" and we get turtles all the way down). Rather, as the old saying goes, I do not believe that the map is the territory. Just like how our brain creates an incomplete model of the world we live in, mathematics too is used to create an incomplete model to help describe not only what we can see but what we don't. But do not trivialize or diminish the notion of a model, as I certainly would not claim our brains and senses are useless. Models are quite powerful things, there is a reason we use them. But a model is not the thing itself.
For me one major aspect of this “debate” is that the people who see or espouse what I see as over extensions of the abilities of technologies are the ones most unfamiliar with it.
There is an old bit of unscrupulous advice that if someone over assumes your abilities that you should refrain from correcting them.
That is, it benefits the NSA that people think they are actively recording all of their conversations all the time because it forces compliance without the necessary competence, but the people who hold these opinions are often wholly ignorant of the kind of technology required to achieve that level of surveillance.
Have you built your own minigpt? Have you implemented rudimentary transformers?
Are you projecting your desires onto something wholly unworthy of your devotion?
Because the people behind these things are financially incentivized to nod along as your impart more ability than what they know they put into them.
For clarity, my “religion” is math. I believe existence fundamentally is a mathematical construct and as such so are all of its creations.
The brain is to me a mathematical byproduct, but even still, when I familiarized myself with the math of llms and their abilities I recognized that they fall short of being, simulating, or explaining the former.
Llms are stochastic next token pickers, full stop.
Any perceived “intelligence” is projection and anthropomorphising by the agent using them.
I saw a comment on here in another thread stating that the capacity for coherent use of language falls short of being evidence of “intelligence” as children show signs of human “intelligence” long before they can form coherent sentences.
No, but did follow along to an Andrej Karpathy video along those lines at the beginning of the year.
I didn't want to make a judgement on any kind of superiority, or that LLMs simulate brains, or anything of that nature. Just wanted to question why these elements (namely intelligence and reasoning) strike the nerve that they do.
The anthropomorphism argument is case in point, really. It poses the accusation that the other side is imparting human qualities to a machine, without needing to touch on what makes those qualities human or why that matters in the first place. It is, ironically enough, flawed reasoning.
> Just wanted to question why these elements (namely intelligence and reasoning) strike the nerve that they do.
“Just asking questions” is a meme of the unscrupulous.
I think you are unfairly lumping those who believe in human exceptionalism with those cynical of the economics of such claims.
It’s okay, to me, for people to be ignorant of what llms are. What a dismally bland existence if everyone were just llm experts.
What strikes a nerve with me is the people financially incentivized to do so are leveraging the terror, both the awe and fear interpretations, of those ignorant of the tech.
> The anthropomorphism argument is case in point, really. It poses the accusation that the other side is imparting human qualities to a machine, without needing to touch on what makes those qualities human or why that matters in the first place.
This reads as circular reasoning. Those claiming the opposite are also failing to define what those qualities are.
Anthropomorphism is a real thing. I can flinch in pain for the sake of my couch when a friend jumps onto it, but that hardly provides, without me needing to define human pain explicitly, an opportunity for said friend to respond with the absurd claim that human pain is in fact couch based.
I'll concede the just asking questions point, that much is true.
GPT4 appears to give more intelligent responses than GPT3. To describe that, though, perhaps we need to migrate to a term that doesn't step on the toes of those who, though not human exceptionalists, rather just feel that (these particular?) machines don't happen to suit measurement in such human domains as intelligence.
Of cause, the ship has sailed and they're fighting a lost cause. There's little reason to dig for new words. It's the I in AI and it has been for longer than many here have been alive.
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975
Or, they realize that absence cannot be scientifically proven, and that scientists use language loosely, confusing those who take their loose language literally.
The study didn't "find" (discover) what they claim, rather, they didn't find validation that it "can" (the implementation of which varies per observer, sub-perceptually).
If you had to code something like this at work for a different domain, I bet you'd have no problem realizing that a nullable boolean is required to accurately model the problem space.
I recognize your clarification of “discovery” and conclusion from that research, but I do think there is a strong argument that in terms of the stochastic usage of a nonlinear system the “undefined” state of your nullable boolean is itself a falsey state.
You can argue whatever you like, but if the unknown IS actually known, why can't scientists tell us their secrets? How many people would have to be in on the scheme?
And this isn't just a one off, this is a systemic, institutional shortcoming, I encounter several instances of it every day just in my regular social media feeds.
If I am understanding your experience correctly the colloquial wisdom here is to use GIN on static data and GIST on dynamic data.
> In choosing which index type to use, GiST or GIN, consider these performance differences:
> GIN index lookups are about three times faster than GiST
> GIN indexes take about three times longer to build than GiST
> GIN indexes are moderately slower to update than GiST indexes, but about 10 times slower if fast-update support was disabled (see Section 54.3.1 for details)
> GIN indexes are two-to-three times larger than GiST indexes
> As a rule of thumb, GIN indexes are best for static data because lookups are faster. For dynamic data, GiST indexes are faster to update. Specifically, GiST indexes are very good for dynamic data and fast if the number of unique words (lexemes) is under 100,000, while GIN indexes will handle 100,000+ lexemes better but are slower to update.
I think the chief confusion here is that you may be thinking the light arrives at the detector in the same amount of time regardless of spacetime curvature. That is, the the ruler is itself squishing.
But what needs to be considered is the constant speed of light.
This implies that what happens is, in the presence of additional curvature, and constant speed of light, the additional distance traveled will have appeared to slow the light.
In the laser interferometer this registers as interference.
https://m.youtube.com/watch?v=ajZojAwfEbs
It is also worth noting that any claims of detection are thoroughly investigated and confirmed with other detectors.
> It is difficult for a single LIGO detector to confirm a gravitational wave signal on its own. The initial discovery of gravitational waves required that the signal be seen in both detectors (Hanford and Livingston).
https://www.ligo.caltech.edu/page/what-is-ligo