14 ms·
Bit late to the party here, but here's a thing that's been bugging me: as a veteran of many deep learning frameworks, I’m 100% convinced of many of the argument
by taliesinb 7y ago
Bit late to the party here, but here's a thing that's been bugging me: as a veteran of many deep learning frameworks, I’m 100% convinced of many of the arguments put forward in "Tensors Considered Harmful" [1].
Basically, I can summarise the argument as: for complex, real-world projects involving multidimensional arrays, having to remember which dimension means which thing, and track these through various tensor transformations, is the 21st century equivalent of how prior to high level programming languages we had to manually track which registers we we were using to store what quantities, and how this changed in different parts of a program.
It was the real dark ages of programming, and we’re in that same place with a lot of complex deep learning models now.
So my question is: has this been discussed at all in the Julia community? Is this on anyones radar? Since one of the core competencies of Julia is array processing and elegant metaprogramming, I would hope that it can gracefully absorb the idea of named array axes in a way that feels natural and complete, and could start showing up in ML frameworks like Zygote and Flux down the line...
[1] http://nlp.seas.harvard.edu/NamedTensor http://nlp.seas.harvard.edu/NamedTensor
- ddragon 7y agoThere are some packages: https://github.com/davidavdav/NamedArrays.jl https://github.com/davidavdav/NamedArrays.jl In Julia, as long as any custom type implements the base methods (through overloading) it will work with anything and with no performance cost (since most base Julia types are written in Julia anyway), including every ML library. For example unitful that keeps track of the physical dimensions that perfectly integrate with the differentialequations library: http://tutorials.juliadiffeq.org/html/type_handling/03-unitful.html http://tutorials.juliadiffeq.org/html/type_handling/03-unitf...
- mcabbott 7y agoThere are a bunch of packages exploring ideas around labelled dimensions, although we are still far from the vision of caring as little as we do about registers: * https://github.com/JuliaArrays/AxisArrays.jl https://github.com/JuliaArrays/AxisArrays.jl * https://github.com/davidavdav/NamedArrays.jl https://github.com/davidavdav/NamedArrays.jl * https://github.com/invenia/NamedDims.jl https://github.com/invenia/NamedDims.jl * https://github.com/rafaqz/DimensionalData.jl https://github.com/rafaqz/DimensionalData.jl * https://github.com/ITensor/ITensors.jl https://github.com/ITensor/ITensors.jl While these don't pass dimension labels along in the type, there are also a number of Einsum-like packages which should work on any storage order, and could be taught to interface with such types: * https://github.com/Jutho/TensorOperations.jl https://github.com/Jutho/TensorOperations.jl * https://github.com/under-Peter/OMEinsum.jl https://github.com/under-Peter/OMEinsum.jl * https://github.com/mcabbott/TensorCast.jl https://github.com/mcabbott/TensorCast.jl