Well, first step is to create bindings to the same libraries Python uses.

The problem is that Numpy is not in fact anything close to a thin wrapper around BLAS/LAPACK like people seem to think it is.

First of all, it contains a ton of custom C code, which to some extent could be extracted to a separate library in theory, but isn't. Second, a lot of that custom code interacts deeply with the Python C API, which historically was very open-ended. Even getting it to work on another implementation of Python was a challenge that took a long time to reach baseline usability.

You could forego Numpy and call out to a library like Eigen, but even then you have a huge amount of work ahead to achieve anything resembling feature parity.

Who singled out Numpy?

Still, your lengthy explanation only confirms how much C and how little Python, that specific case happens to be.