We had this discussion on here recently. It's really puzzling to me. Julia has the most ergonomic array interface I have worked with by far. How did 1 based indexing ever trip you up?
I love Makie but for investigating our datasets Python is overall superior (I am not familiar enough with R), despite Julia having the superior Array Syntax and Makie having the better API. This is simply because of the brilliant library support available in scikit learn and the whole compilation overhead/TTFX issue. For these workflows it's a huge issue that restarting your interactive session takes minutes instead of seconds.
On top of what others have said: In many situations the alternative to Julia isn't Go but C++ (or maybe Rust though its library support is lacking). E.g. if you are writing high-ish* performance algorithmic code that should also run on GPU. Julia also heavily prioritizes composability of libraries. And though that can be a source of bugs at times, there are things we are doing in Julia with one or two PhD students that would be practically impossible (as in require an order of magnitude more implementation work) in any other language.
This has been my optimistic take on the situation for the last two years. My pessimistic take is that social systems have an incredible ability to persist in a state of utter fuckedness much longer than seems reasonably possible.
Well, yes. "Accelerationists" of all philosophies think that heightening the contradictions and breaking the current system will bring about its replacement with a better one. But a new system requires work, while chaos doesn't. It's quite possible to just destroy the current system leaving us without the wherewithal to build a replacement any time soon.
Yeah and like…who knows if what is coming is better. Maybe big labs cartelize and withdraw from the global publication market (which is already unraveling). Maybe we ban theory and demand all papers be empirical, though that will amount to the same thing: seizure of publication by big actors.
As you point out, human systems are machines for making do. There is no guarantee that dramatic pressures produce dramatic change. But I think we’ll see something weird, soon.
This is stupid. Nobody motivated by money is in academia. Academics are motivated by curiosity, but also prestige, vanity and the wish to hire students and collaborators. And on top of human vanity working it's magic, the ideology that everything should be a market and competition is the final form of social organisation, has pervaded academia just as much as everything else.
I agree that the system of publishing papers to gain prestige to gain resources to publish papers was already broken pre AI.
You're right that being a scientist is unlikely to result in personal wealth and so that's not the primary drive for those who seek faculty or research positions. However, it's not just curiosity, prestige and vanity either, because a big factor for promotion and tenure is how much grant money you bring in. That money is what keeps the university's lights on and buys the lab equipment and pays the grad students, so it's still money as a primary driver in the background.
My dad said he stopped being a professor because of that.
He liked the research, and he even liked teaching, but he absolutely hated having to constantly try and find grant money. He said he ended up seeing everyone as "potential funders" and less like "people" because his job kind of depended on it, and it ended up burning him out. He lasted four years and went into engineering.
I don't know that "motivation" is the right word for it, because I don't think professors like having to find grant money all the time. I think most people who get PhDs and try to go to academia do it for a genuine love for the subject, and they find the grant-searching to be a necessary evil part of the job; it's more "survival" than regular motivation, though I am admittedly splitting hairs here.
LLMs can execute searches? You can absolutely send ChatGPT to look for a cheap flight and it will do pretty well. And because I am paying for ChatGPT rather than the advertiser's, I am the customer and not the product.
You may pay to ChatGPT, but sooner or later you will become their product too. All the conversations you had or will have will be turned into signals to match you with products from advertisers, maybe not directly in the conversation with them, but anywhere else. It's not a mater of if, but looking at the pace things are going, and how financially pressured openai is, it's only a matter of time that their conversations with them will be turned into profit in some way or another, they basically have no choice financially.
I disagree. Julia has correctnes issues because it chose maximum composability over specifying interfaces explicitly. And those are not just in immature packages but also in complex packages. Compared to other languages, Julia has no facilities to help structure large complex code bases. And this also leads to bad error messages and bad documentation.
Recently we got the public keyword, but even the PR there says:
"NOTE: This PR is not a complete solution to the "public interfaces are typically not well specified in Julia" problem. We would need to implement much than this to get to that point. Work on that problem is ongoing in Base and packages and contributions are welcome."
What if I want the nth element up to the math element? arr[n:m]. And if I want to split the array into two parts, one until the nth element and the other from the m+1st element arr[1:m] and arr[(m+1):end]. Julia matches how people speak about arrays, including C programmers in their comments. Arrays are (conceptually) not pointer arithmetic. Also for your usecase typically you would just use a 2d array and write a[n,:].
so you just need to overload the syntax of intervals even more to make it work
> arr[0..m], arr[m..]
now `m` refers to different things depending on which side of the interval it's on. less characters doesn't mean nicer
I get it though, I was skeptical about 1-based indexing when I started Julia. By the nature of indices vs length there will always be an off-by-one problem: either you have elements [n, m - 1] with length (m - n) or [n, m] with length (m - n + 1). Unless you're doing a bunch of pointer arithmetic type stuff, I find the symmetry of a inclusive-inclusive interval to be a better default.
Your second point is the main argument for me personally. Numbers in brackets always mean the same thing: the ordinal number of the references object in an ordered collection. In 0 based indexing you can think of the number as refering to the space between the referenced objects. But that is simply an additional mental image on top of the original one.
As a neat bonus, in Julia 1:5 is just the iterator for the numbers 1 to 5. So slicing is typically not some special syntax either. It all works rather nicely.
So if I have a row of 5 apples, I can say "take the second and third apple" or I can say "take the apples between one apple length and three apple lengths from the start".
Which is more natural? The ruler is exactly the right mental image if an array to you is a partitioned region of memory starting at a specific pointer location. If an array to you is an ordered collection of objects, you would never invent 0-based indexing or inclusive-exclusive slicing.
Either way, it's not a big deal. I have lived in both worlds, I have come to think Julia is a bit more natural and easier to teach. But it ls really the silliest bike shedding complaint, given that the language has considerable real trade offs.
This is such a classic example of online discourse in general. There are two options, and folks tribally cling to one or the other without realizing that both are legitimate and well-suited for different situations.
Yes, of course distances are measured starting from 0. But we count discrete things starting at 1. You can do mental gymnastics to enumerate from zero and many programmers are (unfortunately IMO) taught to do so. It's a hard thing to learn that way, so for the folks that have done so, it often becomes a point of pride and a shibboleth.
As a classic example, a four story building has four floors. But you only need to go up three flights to get to the top. You can legitimately call the top floor either 3 or 4, and folks are similarly tribal about their own cultural norms around this one, too.
> There are two options, and folks tribally cling to one or the other without realizing that both are legitimate and well-suited for different situations.
No I disagree entirely. One is simply better.
> It's a hard thing to learn that way, so for the folks that have done so, it often becomes a point of pride and a shibboleth.
It is not hard. It's not better because it's hard-won knowledge. It's better because it leads to simpler, more elegant code. Simple as.
Fully agreed. I first struggled when switching from python to Julia, then ended up finding it slightly better for my use cases (which includes teaching scientists who are not programmers). But it's simply not a big deal either way. I am also reminded of the significant whitespace objections to python in the old days, before python took over everything...
Thanks for correcting; I know that "loss function" is not a good term when it comes to transformer models.
Since I've forgotten every sliver I ever knew about artificial neural networks and related basics, gradient descent, even linear algebra... what's a thorough definition of "next token prediction" though?
The definition of the token space and the probabilities that determine the next token, layers, weights, feedback (or -forward?), I didn't mention any of these terms because I'm unable to define them properly.
I was using the term "loss function" specifically because I was thinking about post-training and reinforcement learning. But to be honest, a less technical term would have been better.
I just meant the general idea of reward or "punishment" considering the idea of an AI black box.
The parent comment probably forgot about the RLHF (reinforcement learning) where predicting the next token from reference text is no longer the goal.
But even regular next token prediction doesn't necessarily preclude it from also learning to give correct and satisfying answers, if that helps it better predict its training data.
I didn't, hence the "first". It's clear that being good at next token prediction forces the models to learn a lot, including giving such answers. But it's not their loss function. Presumably they would be capable of lying and insulting you with the right system prompt just as well. And I doubt RLHF gets rid of this ability.
If you didn't forget about the RLHF, your comment is oddly pedantic, confusing and misleading. "Correct and satisfying answers" is roughly the loss function for RLHF, assuming the humans favor satisfying answers, and using "loss function" loosely, as you yourself do, by gesturing at what the loss function is meant to do rather than formally describing an actual function. The comment you responded to didn't say this was the only loss function during all stages of training. Just that "When your loss function is X", then Y happens.
You could have just acknowledged they are roughly correct about RLHF, but brought up issues caused by pretraining.
> And I doubt RLHF gets rid of this ability.
The commenter you were replying to is worried the RLHF causes lying.
reply