I had to google "differentiable programming" to see if they were really touting automatic differentiation as some futuristic language feature. Honestly the only real response to this should be :trollface:
I've been doing software development in data science, large scale optimization and machine learning for over 15 years... I've needed automatic differentiation in my language .... exactly never. I mean, most of the languages I use regularly are capable of it, and it is a neat trick; it's just not that useful.
The best part of this article is Yann and Soumith twittering they need Lush back (and not because of automatic differentiation). I agree; it's still my all time favorite programming language, and I don't even fool around with Deep Learning.
https://twitter.com/jeremyphoward/status/1097799892167122944
Sure: great -it can be used with other sorts of models as well, but it's really not that big a deal, and can be implemented in a lazy afternoon. I'm pretty sure lack of this as a first class language feature is holding back exactly no DL frameworks or programming languages, and glueing it onto Swift isn't going to make DL people use it. I've never used Swift's repl; I suppose that might actually be something that gets people on board.
Hi, I'm working on making a tensor lib in Rust (think numpy + autodiff) to learn about these topics. There isn't much information online about how projects like numpy and autograd work under the hood.
Do you have any ideas/tips/resources about how it could be done?
Numpy is basically lapack. You'd be hard pressed to replace that with something nearing its performance. For autodiff, I dunno, how about John Mount's explanation?
It had a lot of cranky pieces when it made the 64 bit transition, and afaik it never got GPU support. I'm pretty sure it was ultimately abandoned because students didn't like it.
I've been doing software development in data science, large scale optimization and machine learning for over 15 years... I've needed automatic differentiation in my language .... exactly never. I mean, most of the languages I use regularly are capable of it, and it is a neat trick; it's just not that useful.
The best part of this article is Yann and Soumith twittering they need Lush back (and not because of automatic differentiation). I agree; it's still my all time favorite programming language, and I don't even fool around with Deep Learning. https://twitter.com/jeremyphoward/status/1097799892167122944