2 ms·
You still get an API that encapsulates the behavior. This is not like monkey-patching (directly changing the behavior of libraries), but separating the abstract
by ddragon 6y ago
You still get an API that encapsulates the behavior. This is not like monkey-patching (directly changing the behavior of libraries), but separating the abstraction layers. Every complex enough system will have multiple layers (for example when working on communications, if you're working with the network layer you don't need to focus on the physical layer below or the application layer above). Multiple dispatch allows the library ecosystem to better work in the same way:
For machine learning models we have the layer that handles the low level operation (sums, multiplication), which are swappable (you can have an implementation that runs in the CPU - Julia's Base - and an implementation that run in the GPU - CUDA.jl - and even a TPU - XLA.jl or Torch as backend). Above you have the tracker (the layer responsible for the autodifferentiation logic, which includes Tracker, Zygote, ForwardDiff). And above you have the library with rules for generating gradients (DiffRules, ChainRules), and above you have ML constructs (NNLib), and above ML frameworks (Flux, Knet) and above more specialized libraries like DiffEqFlux.
Whoever writes the ML framework doesn't need to care about the backend, whoever writes the GPU backend doesn't need to care about ML framework. This is not because the person writing the GPU backend patched the ML framework, but because the ML framework legitimately doesn't care about how the low level operations are executed, it doesn't work on that level of abstraction. And the user of the ML library can still see it like a monolith not unlike Pytorch or Tensorflow when he imports a library like Flux, until he wants to extend them and then he will find that they are in fact many independent swappable systems that compose into something more than the sum of it's parts.