I've nearly exclusively used Julia since 2017. I don't think this is a perverse use of such functions -- long ago I naturally guessed I could use `cumsum!` on the same input and output and it would correctly overwrite the values (which now gives a similar warning in the documentation). However, when I first used it that way I tested if it did what I expected to verify my assumption.
It is good the documentation is now explicit that the behavior is not guaranteed in this case, but even better would be if aliasing were detected and handled (at least for base Julia arrays, so that the warning would only be needed for non-base types).
Still, the lesson is that when using generic functions one should look at what they expect of their input, and if this isn't documented one should at least test what they are giving thoroughly and not assume it just works. I've always worked this way, and never run into surprises like the types of issues reported in the blog post.
Currently there is no documentation on what properties an input to `sum!` must support in the doc string, so one needs to test its correctness when using it outside of base Julia data types (I haven't checked the broader docs for an interface specification, but if there is one it really should be linked in the docstring).
But your use of cumsum!() seems natural; I can see using it that way, and might have done so myself. The use of sum!() under discussion seems weird, though.
The simplest approach is to always read the interface of packages one wants to use, and if one isn't provided look at the code / open an issue to interact with the developers about their input assumptions. One should also make tests to ensure the interface behaves in the expected manner when working with your code.
Using this approach since 2017 I've never really encountered the types of issues mentioned in Yuri's blog post. The biggest issue I've had is if some user-package makes a change that is effectively breaking but they don't flag it the associated release as breaking. But this isn't really a Julia issue so much as a user-space issue, and can happen in any language when relying on others' libraries.
It doesn't happen so frequently in practice, which is maybe why it's not felt equally by everyone in the community. I've followed some discussions on this topic and the current solution is, as you say, to place the onus entirely on the user. There is another faction that wants to support the user through the IDE to warn them of errors they might be making, and others that want to implement formal specifications that catch the errors during specifications. It's not an easy problem to fix and therefore the "simplest approach" as you describe remains the leading solution.
Of course you'd have to use arbitrary-precision arithmetic, that's the easy part. The hard part is knowing what to do with it, if you're someone like me who isn't an expert in ODEs. Do you know of any accessible resources on spectral methods?
(Of course arbitrary precision is far, far worse than double precision in terms of performance, which is why so much work goes into techniques that avoid increasing precision. But sometimes you have few enough instances to solve that you can simply eat the cost.)
I’m all for keeping user freedom to install whatever they want, and wish it was easy to do so on my iPhone. I will point out though that if you pay for YouTube premium it has now incorporated the ability to skip in-video sponsored ads. I use this all the time now on my Apple TV via the YouTube app.
It is actually pretty hypocritical they’re adding such tech to their apps while fighting so hard against people that want to block the ads they serve…
Yeah, and forget about giving skeleton code to students they should fill in; using an AI can quite frequently completely ace a typical undergraduate level assignment. I actually feel bad for people teaching programming courses, as the only real assessment one can now do is in-class testing without computers, but that is a strange way to test students’ ability to write and develop code to solve certain classes of problems…
Hopefully someone is thinking about adapting the assessments. Asking questions that focus on a big picture understanding instead of details on those in-class tests.
I’ve had that happen in notes from an editor telling me that in addition to addressing reviewer comments I should cite a whole bunch of their completely unrelated articles when submitting a revision.
But I have also been a reviewer in a situation where a supposed broad review article for a preeminent journal completely ignores citing a massive amount of relevant prior work (except, conveniently, the authors’ own work, which in many cases was cited over more relevant and important earlier work), including a large number of my own articles.
General scientific computing is pretty good across the Julia ecosystem, from optimization, to ODE and now PDE solver libraries, to various statistics and inference packages, etc. It lacks the deep NN tooling or breadth of ML libraries of Python, and nothing matches R for breadth of stats libraries, but for most other scientific computing it is really great at this point.
I am by no means an expert, but I used Flux.jl for a convolutional neural net in electromagnetics for my latest paper and it was such a breath of fresh air compared to Python and PyTorch. (I'm an EE and not great at programming, so I found a lot of frustration in PyTorch). Even though the Keras library in Python is pretty nice, even then I got myself into some odd pickles when trying to do some custom layers which used FFT processing as it relates to gradient computation. Things are much smoother in Julia, and that doesn't even count how much easier the Plots library is! I'm ashamed to admit that I have no idea how to manipulate the figures and axes in Matplotlib without extensive googling.
You can always (slightly) reduce the DiffEq dependencies by adding OrdinaryDiffEq.jl instead of the meta DifferentialEquations.jl package. But lots of those dependencies arise from supporting modular functionality (changing BLAS, linear solvers, Jacobian calculation methods, in vs. out of place workflows, etc.). That said, the newer extension functionality may let more and more of the dependencies get factored out into optional extensions as time goes on.
And you can always use the "Simple" versions, SimpleDiffEq.jl, SimpleNonlinearSolve.jl. Those libraries were made so that users could use exactly the same syntax but with dumbed down solvers with essentially zero latency. But yes the complete solvers are doing lots of fancy things for fancy cases, but SimpleTsit5 is recommended in the docs for small cases where you don't need all of the extra bells and whistles.
That's a bit different, and an interesting different. Certain types of partial differential equations like the one solved there generally use a form of step splitting (i.e. a finite volume method with a staggard grid). Those don't map cleanly into standard ODE solvers since you generally want to use a different method on one of the equations. It does map into the SplitODEProblem form as a not DynamicalODEProblem, and so there is a way to represent it, but we have not created optimized time stepping methods for that. But I work with those folks so I understand their needs and we'll be kicking off a new project in the MIT Julia Lab in the near future to start developing split step and multi-rate methods specifically for these kinds of PDEs.
It's an interesting space because:
-(a) there aren't really good benchmarks on the full set of options, so a benchmarking paper would be interesting to the field (which then gives a motivation to the software development)
-(b) none of the implementations I have seen used the detailed tricks from standard stiff ODE solvers and so there's some major room for performance improvements
-(c) there's some alternative ways to generate the stable steppers that haven't been explored, and we have some ideas for symbolic-numeric methods that extend the ideas of what people have traditionally done by hand here. That should.
so we do plan to do things in the future. And having Oceananigans is then great because it serves as a speed-of-light baseline: if you auto-generate an ocean model, do you actually get as fast as a real hand-optimized ocean model? That's the goal, and we'll see if we can get there.
We have tons of solvers, but you always need more!
I think it remains to be seen how/if its reputation changes given their new approach. It will take a number of years before its impact can really be assessed.
Right, and that is why the same government bodies need to cap the amount that can be paid towards open access fees or page charges via a grant. If grants only provided $200 per publication that would end the ridiculous open access fees / APCs that currently get charged. There is no reason it should cost thousands of dollars to publish an article given the limited proofing most journals now do, the ability to easily submit works in a template provided by a journal, and that peer-review is uncompensated.
It is good the documentation is now explicit that the behavior is not guaranteed in this case, but even better would be if aliasing were detected and handled (at least for base Julia arrays, so that the warning would only be needed for non-base types).
Still, the lesson is that when using generic functions one should look at what they expect of their input, and if this isn't documented one should at least test what they are giving thoroughly and not assume it just works. I've always worked this way, and never run into surprises like the types of issues reported in the blog post.
Currently there is no documentation on what properties an input to `sum!` must support in the doc string, so one needs to test its correctness when using it outside of base Julia data types (I haven't checked the broader docs for an interface specification, but if there is one it really should be linked in the docstring).