People have recently started talking about "differentiable programming". I have listened to some people talk about this at a philosophical level, but I don't see the practical difference between this and just existing Autograd libraries like PyTorch.
I understand that (at least according to some interpretation of "differentiable programming"), what makes it different is that it is an integrated part of a programming language. For example, in Julia, backprop/autograd is apparently a fundamental element of the language.
However, it seems to me that there isn't such a big difference between calling backprop as a library call to PyTorch, vs having it more integrated in the language. The library call to PyTorch is already very simple. So how is "differentiable programming" actually different in a meaningful way from just calling a library like PyTorch or TensorFlow?