3 ms·
How are sum-types handled in deep learning? E.g., a type that holds "a 3x2 tensor OR a 4x6x2x5 tensor".
by amelius 4y ago
How are sum-types handled in deep learning?
E.g., a type that holds "a 3x2 tensor OR a 4x6x2x5 tensor".
- nicd 4y agoI'm not aware of tensor shape type safety (in general) being widely used in deep learning, let alone sum-types. I believe Pytorch and TensorFlow lack support for tensor shape type checking (looks like there is a Pytorch issue open: https://github.com/pytorch/pytorch/issues/26889 https://github.com/pytorch/pytorch/issues/26889).
- ntr-- 4y agohttps://docs.rs/ndarray/0.15.6/ndarray/ https://docs.rs/ndarray/0.15.6/ndarray/ Rust to the rescue, as it always is for safety. (I don't know if this directly translates for tensors, I used it for manipulating point clouds and iirc it allows for arbitrary dimension containers)
- gpderetta 4y agoHow does this help with typechecking tensor types in python?
- survirtual 4y agoI use this crate extensively and while I have gotten good results, it is not ergonomic and it is not equivalent to numpy ndarray. There are a lot of issues and work still to be done. Just a word of caution for anyone interested in using it. Things like windowing, slicing, views are all “supported” but have many limitations and are non intuitive. In particular, I needed an “axis shift” fn and I ended up spending hours scouring docs just to find a way to shift along an axis. When I mean by shift is a roll in numpy, ala: https://github.com/rust-ndarray/ndarray/issues/281 https://github.com/rust-ndarray/ndarray/issues/281 Answer: you can’t with this crate. I implemented a dynamic n-dim solution myself but it uses views of integer indices that get copied to a new array, which have indexes to another flattened array in order to avoid duplication of possibly massive amounts of n-dimensional data; using the crate alone, copying all the array data would be unavoidable. Ultimately I’ve had to make my own axis shifting and windowing mechanisms. But the crate is still a useful lib and continuing effort. While I don’t mind getting into the weeds, these kinds of side efforts can really impact context focus so it’s just something to be aware of.
- mxkopy 4y agoIn my experience (CV) you typically want 4 dims - (height, width, channels, batch). In Julia you can do element-wise operations on two arrays if their lengths (i.e. size after flattening) are the same. Obviously dimension-wise that sort of thing won't work (what's the dot product of a 3-vector by a 6-vector? Undefined!) But really, the whitepaper should include this sort of detail, and you should be following the white paper.