4 ms·

There is work on rotationally invariant networks, e.g. for identifying galaxies, or cells under a microscope. For example: https://arxiv.org/abs/1612.04642 htt

There is work on rotationally invariant networks, e.g. for identifying galaxies, or cells under a microscope. For example: https://arxiv.org/abs/1612.04642 https://arxiv.org/abs/1612.04642 https://arxiv.org/abs/1805.12301 https://arxiv.org/abs/1805.12301 I haven't looked closely enough to be sure if they literally had complex activations, but this seems like an obvious use. Maybe they would have, if only tensorflow made it easy.

mlthoughts2018 8y ago
Why would you ever want to represent activations directly as complex numbers in that case? I can’t think of any good reason, compared with putting them in rotation matrix form or some other equivalent form that actually maps to the domain modeling problem. Even when working in signal processing problems that require complex arithmetic, the underlying representations are just based on tuples of doubles and operator conventions, and you always need to map to real spaces (real part, imaginary part, angle, or magnitude) for any type of analytical representation that can be human readable. In all these cases, the idea that what we should optimize for is overhead-free easy expression of cutesy math domain verbiage is a bad idea. Writing libraries that expose an API that matches the user’s domain mental model is a great thing. But enforcing a particular abstraction and extensibility hierarchy so those things can be “autogenerated” just by parameterizing over a new type turns out to be actually much worse than just writing that type separately, with helper functions and converters, and customizing its API to be efficient from a domain mental model perspective. A better way, for example, might be to use mixin patterns or decorators and other metaprogramming, while writing a custom data type and its associated methods.
- byt143 8y ago
  Something like this? https://github.com/andyferris/Traitor.jl https://github.com/andyferris/Traitor.jl
  - mlthoughts2018 8y ago
    That is definitely a cool direction to take it!

mlthoughts2018 8y ago

Why would you ever want to represent activations directly as complex numbers in that case? I can’t think of any good reason, compared with putting them in rotation matrix form or some other equivalent form that actually maps to the domain modeling problem. Even when working in signal processing problems that require complex arithmetic, the underlying representations are just based on tuples of doubles and operator conventions, and you always need to map to real spaces (real part, imaginary part, angle, or magnitude) for any type of analytical representation that can be human readable. In all these cases, the idea that what we should optimize for is overhead-free easy expression of cutesy math domain verbiage is a bad idea. Writing libraries that expose an API that matches the user’s domain mental model is a great thing. But enforcing a particular abstraction and extensibility hierarchy so those things can be “autogenerated” just by parameterizing over a new type turns out to be actually much worse than just writing that type separately, with helper functions and converters, and customizing its API to be efficient from a domain mental model perspective. A better way, for example, might be to use mixin patterns or decorators and other metaprogramming, while writing a custom data type and its associated methods.

byt143 8y ago
Something like this? https://github.com/andyferris/Traitor.jl https://github.com/andyferris/Traitor.jl
- mlthoughts2018 8y ago
  That is definitely a cool direction to take it!