6 ms·

Hi everyone, co-creator of Nx here! I have just published an article on Dashbit's blog with a bit more context on Nx and the design decisions behind it: https:

Hi everyone, co-creator of Nx here! I have just published an article on Dashbit's blog with a bit more context on Nx and the design decisions behind it: https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-available https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-... It is hopefully a more in-depth reference than the README. If you have any questions, I will be glad to answer them!

rdtsc 6y ago
Just wanted to say thank you, and how much I appreciate your work for the Elixir and Erlang communities. A common complaint with Elixir and Erlang is that it isn't suitable for numerical computations, well now it is!
nnadams 6y ago
I'm excited to try this out, especially after seeing the XLA support. Where do you see Nx going in the future? I saw there's an MNIST example in the repo. Do you plan including higher-level features for neural networks, similar to Keras/PyTorch?
- josevalim 6y ago
  Sean is looking into higher-level features for NN - it will likely be a separate library on top of Nx. For general direction, I briefly discuss that at the end of the article above. :) If something is unclear, please let me know!
rhizome31 6y ago
As someone working in the weather field, the reference to xarray at the end of the article is very exciting. I imagine that eventuellay it will be possible to read NetCDF files and create dataframes based on Nx.
elcritch 6y ago
Great initiative! One bit that sticks out to me is that the "standard" math functions like `exp` or `ln` aren't imported despite `defn` being quite maths specific. It seems that they could be imported as default functions inside the scope of `defn`. That leads me to a more important syntax question though. Do you have any thoughts or plans on operator syntax for elementwise operations vs matrix level operations? One big "win" for numerical Python was having `@` become an infix operator specifically for matrix dots [0, 1]. The syntax in Julia (which follows Matlab) is to have most math operators also have an element wise version. It's generalized to "broadcast" operations in Julia [2]. R appears to have ``, `%%, `%o%` infix operators for elementwise, inner, and outer matrix multiplication. Really writing any degree of matrix (ahem Tensor) maths and equations becomes very tedious and illegible if you have to use function calls to distinguish elementwise vs matrix operations. The pipe operator really does help significantly, even with simpler stats oriented code (I've tried a few different formats [4]). Still it's still harder to read if you're familiar with "broadcast" syntax from other mathematics oriented languages. 0: https://alysivji.github.io/python-matrix-multiplication-operator.html https://alysivji.github.io/python-matrix-multiplication-oper... 1: https://www.python.org/dev/peps/pep-0465/ https://www.python.org/dev/peps/pep-0465/ 2: https://julialang.org/blog/2018/05/extensible-broadcast-fusion/ https://julialang.org/blog/2018/05/extensible-broadcast-fusi... 3: https://www.statmethods.net/advstats/matrix.html https://www.statmethods.net/advstats/matrix.html 4: https://github.com/elcritch/matrex_numerix/blob/5835a9b477d8ea41bb9b862272a0997fe37c1236/lib/fft.ex#L81 https://github.com/elcritch/matrex_numerix/blob/5835a9b477d8...
- africanboy 6y ago
  Elixir allows overloading the operators or creating custom ones, but only certain symbols are recognized in the parser, to add new operators the Elixir parser needs to be changed. You can see them here https://github.com/elixir-lang/elixir/blob/master/lib/elixir/src/elixir_parser.yrl#L51-L79 https://github.com/elixir-lang/elixir/blob/master/lib/elixir... Not all of them are defined in the language, the list of the unused infix operator includes \\, <-, |, ~>>, <<~, ~>, <~, <~>, <|>, <<<, >>>, |||, &&&, and ^^^ to define a custom infix operator you can use this syntax def left <|> right do IO.puts("#{left} <|> #{right}") end # usage left MyModule.<|> right # or import MyModule left <|> right if the operator is already defined in Kernel, you can exclude it from import and use your own import Kernel, except: [{:+, 2}] import MyModule left + right as you can see @ is not among them, @ it's not a binary operator and is defined as Kernel.@/1
- josevalim 6y ago
  Hi, those are great points and exactly the type of discussions I expect to have with the greater Elixir community over the next months. I will answer them with my current perspective. Given that Nx has its own numerical kernel, we can import whatever we want. :) I have decided to keep the imported keywords to be a subset of the Elixir's built-in Kernel but there is nothing stopping us from changing that. For example, if exp and ln are all automatically imported in Julia, Python, etc, then it is most likely more than enough to justify them being imported in `defn` too. Regarding custom operators, Elixir has a limited set of operators which have historically been expanded on demand. So it is a matter of looking at other communities, find what is the closest thing to a standard, and send a proposal to the language. :) If you are interested in driving this, you are welcome to submit an initial discussion comparing other languages and an initial set of operators on the Nx issues tracker, then we can discuss it upstream. Thank you!
  - elcritch 6y ago
    Thanks for the response! Glad to hear it's on your radar and ready to foster those discussions. :-) > For example, if exp and ln are all automatically imported in Julia, Python, etc, then it is most likely more than enough to justify them being imported in `defn` too. Matlab and Julia import lots of math defaults because that's their primary target. In Python it varies, and it's pretty common to use `np` to avoid cluttering the namespace. Though partly it's because there isn't a good way (AFAICT) to make a scope with default custom namespace like `defn` gives you! It's a bit of taste really though. I'd suggest it's worth considering. > Regarding custom operators, Elixir has a limited set of operators which have historically been expanded on demand. So it is a matter of looking at other communities, find what is the closest thing to a standard, and send a proposal to the language. :) Well now, you give me way too many ideas. ;) I'll at least post some issues on the Nx tracker. My initial thought would be that perhaps a `<>` operator for "dot" operation would fit in with Elixir's other operators and is comparable to R's `%%` operators so there's a similarity and precedence. If it were low risk to add then it'd be a pretty decent option. It wouldn't solve the whole elementwise issue though. but with syntax trees there could be ways to use Einstein notation instead [1]. That'd be fun to play with! > If you are interested in driving this, you are welcome to submit an initial discussion comparing other languages and an initial set of operators on the Nx issues tracker, then we can discuss it upstream. I'll create an issue on Nx project and basically repost my thoughts there. Then see what other people are thinking! 1: https://en.wikipedia.org/wiki/Einstein_notation https://en.wikipedia.org/wiki/Einstein_notation

rdtsc 6y ago

Just wanted to say thank you, and how much I appreciate your work for the Elixir and Erlang communities. A common complaint with Elixir and Erlang is that it isn't suitable for numerical computations, well now it is!

nnadams 6y ago

I'm excited to try this out, especially after seeing the XLA support. Where do you see Nx going in the future? I saw there's an MNIST example in the repo. Do you plan including higher-level features for neural networks, similar to Keras/PyTorch?

josevalim 6y ago
Sean is looking into higher-level features for NN - it will likely be a separate library on top of Nx. For general direction, I briefly discuss that at the end of the article above. :) If something is unclear, please let me know!

rhizome31 6y ago

As someone working in the weather field, the reference to xarray at the end of the article is very exciting. I imagine that eventuellay it will be possible to read NetCDF files and create dataframes based on Nx.

elcritch 6y ago

Great initiative! One bit that sticks out to me is that the "standard" math functions like `exp` or `ln` aren't imported despite `defn` being quite maths specific. It seems that they could be imported as default functions inside the scope of `defn`. That leads me to a more important syntax question though. Do you have any thoughts or plans on operator syntax for elementwise operations vs matrix level operations? One big "win" for numerical Python was having `@` become an infix operator specifically for matrix dots [0, 1]. The syntax in Julia (which follows Matlab) is to have most math operators also have an element wise version. It's generalized to "broadcast" operations in Julia [2]. R appears to have ``, `%%, `%o%` infix operators for elementwise, inner, and outer matrix multiplication. Really writing any degree of matrix (ahem Tensor) maths and equations becomes very tedious and illegible if you have to use function calls to distinguish elementwise vs matrix operations. The pipe operator really does help significantly, even with simpler stats oriented code (I've tried a few different formats [4]). Still it's still harder to read if you're familiar with "broadcast" syntax from other mathematics oriented languages. 0: https://alysivji.github.io/python-matrix-multiplication-operator.html https://alysivji.github.io/python-matrix-multiplication-oper... 1: https://www.python.org/dev/peps/pep-0465/ https://www.python.org/dev/peps/pep-0465/ 2: https://julialang.org/blog/2018/05/extensible-broadcast-fusion/ https://julialang.org/blog/2018/05/extensible-broadcast-fusi... 3: https://www.statmethods.net/advstats/matrix.html https://www.statmethods.net/advstats/matrix.html 4: https://github.com/elcritch/matrex_numerix/blob/5835a9b477d8ea41bb9b862272a0997fe37c1236/lib/fft.ex#L81 https://github.com/elcritch/matrex_numerix/blob/5835a9b477d8...

africanboy 6y ago
Elixir allows overloading the operators or creating custom ones, but only certain symbols are recognized in the parser, to add new operators the Elixir parser needs to be changed. You can see them here https://github.com/elixir-lang/elixir/blob/master/lib/elixir/src/elixir_parser.yrl#L51-L79 https://github.com/elixir-lang/elixir/blob/master/lib/elixir... Not all of them are defined in the language, the list of the unused infix operator includes \\, <-, |, ~>>, <<~, ~>, <~, <~>, <|>, <<<, >>>, |||, &&&, and ^^^ to define a custom infix operator you can use this syntax def left <|> right do IO.puts("#{left} <|> #{right}") end # usage left MyModule.<|> right # or import MyModule left <|> right if the operator is already defined in Kernel, you can exclude it from import and use your own import Kernel, except: [{:+, 2}] import MyModule left + right as you can see @ is not among them, @ it's not a binary operator and is defined as Kernel.@/1
josevalim 6y ago
Hi, those are great points and exactly the type of discussions I expect to have with the greater Elixir community over the next months. I will answer them with my current perspective. Given that Nx has its own numerical kernel, we can import whatever we want. :) I have decided to keep the imported keywords to be a subset of the Elixir's built-in Kernel but there is nothing stopping us from changing that. For example, if exp and ln are all automatically imported in Julia, Python, etc, then it is most likely more than enough to justify them being imported in `defn` too. Regarding custom operators, Elixir has a limited set of operators which have historically been expanded on demand. So it is a matter of looking at other communities, find what is the closest thing to a standard, and send a proposal to the language. :) If you are interested in driving this, you are welcome to submit an initial discussion comparing other languages and an initial set of operators on the Nx issues tracker, then we can discuss it upstream. Thank you!
- elcritch 6y ago
  Thanks for the response! Glad to hear it's on your radar and ready to foster those discussions. :-) > For example, if exp and ln are all automatically imported in Julia, Python, etc, then it is most likely more than enough to justify them being imported in `defn` too. Matlab and Julia import lots of math defaults because that's their primary target. In Python it varies, and it's pretty common to use `np` to avoid cluttering the namespace. Though partly it's because there isn't a good way (AFAICT) to make a scope with default custom namespace like `defn` gives you! It's a bit of taste really though. I'd suggest it's worth considering. > Regarding custom operators, Elixir has a limited set of operators which have historically been expanded on demand. So it is a matter of looking at other communities, find what is the closest thing to a standard, and send a proposal to the language. :) Well now, you give me way too many ideas. ;) I'll at least post some issues on the Nx tracker. My initial thought would be that perhaps a `<>` operator for "dot" operation would fit in with Elixir's other operators and is comparable to R's `%%` operators so there's a similarity and precedence. If it were low risk to add then it'd be a pretty decent option. It wouldn't solve the whole elementwise issue though. but with syntax trees there could be ways to use Einstein notation instead [1]. That'd be fun to play with! > If you are interested in driving this, you are welcome to submit an initial discussion comparing other languages and an initial set of operators on the Nx issues tracker, then we can discuss it upstream. I'll create an issue on Nx project and basically repost my thoughts there. Then see what other people are thinking! 1: https://en.wikipedia.org/wiki/Einstein_notation https://en.wikipedia.org/wiki/Einstein_notation