Very common in ML research these days - claim novelty / cite prior work in an obfuscated way and so on.

But they have built a fully “independent” pipeline. Deepseek and others probably trained in gpt4, o1 or whatever data.

Why not use torch.compile()?

They train with ERA5 and observations.

Should be a relevant reference: https://arxiv.org/abs/2111.13587

Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, Bryan Catanzaro

Brussels is quite small. Probably less than Washington.

A lot smaller. The city of Amsterdam employs more bureaucracts than the EU.

Probably unfair as a reaction to the unfair statements in the blog…

PyTorch is developed by multiple companies / stake holders while jax is google only with internal tooling they don’t share with the world. This alone is a major reason not to use jax. Also I think it is more the other way around: with torch.compile the main advantage of jax is disappearing.

It's the old age question in programming: Do you use a highly constrained paradigm that allows easy automatic optimization or do you use a very flexible and more user intuitive paradigm that makes automatic optimization harder?

If the future is going to be better more intelligent compilers, then that settles the question in my opinion.

> with torch.compile the main advantage of jax is disappearing.

Interesting take - I agree here somewhat.

But also, wouldn't you think a framework that has been from the ground-up designed around a specific, mature compiler stack be better able to integrate compilers in a more stable fashion than just shoe-horning static compilers into a very dynamic framework? ;)

Depends. PyTorch on the other hand has a large user base and well defined and tested api. So should be doable; and is already progressing and rapid speed..

So the answer is not Jax?

Because JAX is not designed around a mature compiler stack. The history of Jax is more so that it matured alongside the compiler...

AIFS is transformer based (Graphcast is pure GNN) so different architecture and is already running operationally, see:

https://www.ecmwf.int/en/about/media-centre/aifs-blog/2024/i...

For AI you have to support PyTorch. That works well now on the big AMD GPUs. Consumer gpu capabilities do not matter here.

I think that PyTorch is part of the puzzle and it certainly helps that it is supported by AMD [0]. That said, there is code that needs to run closer to the metal too.

[0] https://pytorch.org/blog/experience-power-pytorch-2.0/