3 ms·

I find the argument that vectorized code is more difficult to read weird. Having to write out the loops becomes tedious very quickly and does often significantl

I find the argument that vectorized code is more difficult to read weird. Having to write out the loops becomes tedious very quickly and does often significantly decrease readibility. Take for example the element wise multiplication of two 2d square arrays and compare that to the multiplication of an array and a transpose. In the loop the only difference is that the indices are flipped, easy to overlook in a quick read. My biggest annoyance with Julia is however that they decided to follow matlab and do matrix operations by default and even worse use the '.' as the element wise operation modifier. From my own experience and from teaching many students, one of the primary source of issues is getting that wrong. I don't know how many times when debugging some weird matlab issue it was a matter of find the missing '.' they could have chosen any other symbol but instead the opted for the easiest one to overlook.

eigenspace 5y ago
IMO, doing matrix operations by default is the mathematically correct thing to do. E.g. having exp(M::Matrix) give anything other than the matrix exponential is insanity. Same with A::Matrix * B::Matrix giving anything other than matrix multiplication. As far as the ‘.’ for elementwise operations, I don’t think it’s that hard to miss, but I guess my eyes are trained for it by now. There’s also an @. macro to make an entire expression element wise though if needed, e.g. @. h(f(x) + g(y)) is equivalent to h.(f.(x) .+ g.(y)) I think this broadcasting machinery is one of the closet parts of julia because you get to choose at the call site if you want the function to apply element-wise or to see the whole array, and it works on any container, with user customizable behaviour for user defined containers, and has all sorts of goodies like automatic loop fusion.
- NegativeLatency 5y ago
  Having zero experience with this I could see myself getting used to the notation. The dot before the addition sign is strange looking to unfamiliar eyes, is it calling + on the result of f.(x)?
  - eigenspace 5y ago
    Yeah that’s eighth, it’s element wise addition between the results of f.(x) and g.(y) with loop fusion. the syntax for broadcasting a regular function is f.(x) but the syntax for broadcasting an infix function like +, is x .+ y so the dot goes at the start. I think there were some convincing reasons it ended up this way, but I forget what those reasons are. It’s just muscle memory for me at this point. You can also treat any infix function as a regular prefix one by wrapping it in parens, e.g. (+).(x, y)
- bloaf 5y ago
  I personally think a vector library that allowed some form of Einstein notation [1] would be a killer feature for a scientific computing environment. [1] https://en.wikipedia.org/wiki/Einstein_notation https://en.wikipedia.org/wiki/Einstein_notation
  - eigenspace 5y ago
    Ask and ye shall receive (more than ye bargained for): https://github.com/mcabbott/Tullio.jl https://github.com/mcabbott/Tullio.jl https://github.com/Jutho/TensorOperations.jl https://github.com/Jutho/TensorOperations.jl https://github.com/under-Peter/OMEinsum.jl https://github.com/under-Peter/OMEinsum.jl Tullio.jl is the most powerful and interesting to me of the above libraries, but they all have cool strengths.
- cycomanic 5y ago
  I would argue that an array is something else than a matrix. It's a design decision that matlab (and julia) made, regard every multi-element variable as a matrix. I disagree with that decision, because for most of the code I'm exposed to I don't want matrix operations and it causes issues, e.g. when I move from a scalar implementation to an array implementation of a function.
  - eigenspace 5y ago
    Ah sure, yes we did not need to conflate vectors and matrices with arrays and I sometimes wonder if we made the wrong choice there. I guess I'd say that then someone would get to own the syntax [a, b, c] it'd either be a vector of a non-mathematical array and whichever choice was made, someone would be unhappy. I think not too much was lost by having them be the same thing, but the ship has sailed anyways.
moelf 5y ago
1. in Julia you can't write "vecotrize-styled" code by using `@.` without manually adding the dots everywhere and the result will be as fast in numpy "classic use case" 2. Have not-slow for-loops is invaluable precisely because sometimes your workload is not (either too hard, or impossible/bad in terms of RAM usage) suitable for vectorzie-styled code.
- cycomanic 5y ago
  1. I know and pretty much every line of code I have would need to be preceded with a @. I should also add that julia did not follow the matlab insanity of regarding 1D vectors as arrays, but instead throwing errors helps catch errors before they cause weird issues. 2. I agree that this is one of the nice things about Julia.
  - adgjlsfhk1 5y ago
    In general, what you want to do instead of having `@.` in front of every line is write scalar functions and `@.` their application. For example, instead of writing ``` function f1(A,B,C) tmp = A.+B return tmp1 .* C end f1(::Vector,::Vector,::Vector) ``` You can write ``` function f2(a,b,c) tmp = a + b return tmp1 *c end f2.(::Vector,::Vector,::Vector) ```
hpcjoe 5y ago
I think this depends upon the language intrinsic support for vectors/vector operations. I would (generally) prefer to specify operations using a simpler, more concise syntax, and have the compiler map that into the hardware. The problem with this is, most compilers don't do such a good job with that. I like the Matrix/vector operations by default for the reason that it becomes quite a bit simpler to understand the written code. Expressing things the way Numpy forces you to, means that you have to reason about the interface of your code to Numpy, as well as how Numpy will operate. For matrix mult, I agree, writing out loops is tedious. This is why things like M = A * B, where M, A, B are all matrices, is far, far better than writing out the loop representation. I agree as well, messing up the index ordering is annoying (and I still do this 30+ years onward with Fortran, C, etc.) Julia makes this stuff easy to trivial. It makes the interface to using this capability easy to trivial. It makes for good overall performance (I rarely run codes only once).

eigenspace 5y ago

IMO, doing matrix operations by default is the mathematically correct thing to do. E.g. having exp(M::Matrix) give anything other than the matrix exponential is insanity. Same with A::Matrix * B::Matrix giving anything other than matrix multiplication. As far as the ‘.’ for elementwise operations, I don’t think it’s that hard to miss, but I guess my eyes are trained for it by now. There’s also an @. macro to make an entire expression element wise though if needed, e.g. @. h(f(x) + g(y)) is equivalent to h.(f.(x) .+ g.(y)) I think this broadcasting machinery is one of the closet parts of julia because you get to choose at the call site if you want the function to apply element-wise or to see the whole array, and it works on any container, with user customizable behaviour for user defined containers, and has all sorts of goodies like automatic loop fusion.

NegativeLatency 5y ago
Having zero experience with this I could see myself getting used to the notation. The dot before the addition sign is strange looking to unfamiliar eyes, is it calling + on the result of f.(x)?
- eigenspace 5y ago
  Yeah that’s eighth, it’s element wise addition between the results of f.(x) and g.(y) with loop fusion. the syntax for broadcasting a regular function is f.(x) but the syntax for broadcasting an infix function like +, is x .+ y so the dot goes at the start. I think there were some convincing reasons it ended up this way, but I forget what those reasons are. It’s just muscle memory for me at this point. You can also treat any infix function as a regular prefix one by wrapping it in parens, e.g. (+).(x, y)
bloaf 5y ago
I personally think a vector library that allowed some form of Einstein notation [1] would be a killer feature for a scientific computing environment. [1] https://en.wikipedia.org/wiki/Einstein_notation https://en.wikipedia.org/wiki/Einstein_notation
- eigenspace 5y ago
  Ask and ye shall receive (more than ye bargained for): https://github.com/mcabbott/Tullio.jl https://github.com/mcabbott/Tullio.jl https://github.com/Jutho/TensorOperations.jl https://github.com/Jutho/TensorOperations.jl https://github.com/under-Peter/OMEinsum.jl https://github.com/under-Peter/OMEinsum.jl Tullio.jl is the most powerful and interesting to me of the above libraries, but they all have cool strengths.
cycomanic 5y ago
I would argue that an array is something else than a matrix. It's a design decision that matlab (and julia) made, regard every multi-element variable as a matrix. I disagree with that decision, because for most of the code I'm exposed to I don't want matrix operations and it causes issues, e.g. when I move from a scalar implementation to an array implementation of a function.
- eigenspace 5y ago
  Ah sure, yes we did not need to conflate vectors and matrices with arrays and I sometimes wonder if we made the wrong choice there. I guess I'd say that then someone would get to own the syntax [a, b, c] it'd either be a vector of a non-mathematical array and whichever choice was made, someone would be unhappy. I think not too much was lost by having them be the same thing, but the ship has sailed anyways.

moelf 5y ago

1. in Julia you can't write "vecotrize-styled" code by using `@.` without manually adding the dots everywhere and the result will be as fast in numpy "classic use case" 2. Have not-slow for-loops is invaluable precisely because sometimes your workload is not (either too hard, or impossible/bad in terms of RAM usage) suitable for vectorzie-styled code.

cycomanic 5y ago
1. I know and pretty much every line of code I have would need to be preceded with a @. I should also add that julia did not follow the matlab insanity of regarding 1D vectors as arrays, but instead throwing errors helps catch errors before they cause weird issues. 2. I agree that this is one of the nice things about Julia.
- adgjlsfhk1 5y ago
  In general, what you want to do instead of having `@.` in front of every line is write scalar functions and `@.` their application. For example, instead of writing ``` function f1(A,B,C) tmp = A.+B return tmp1 .* C end f1(::Vector,::Vector,::Vector) ``` You can write ``` function f2(a,b,c) tmp = a + b return tmp1 *c end f2.(::Vector,::Vector,::Vector) ```

hpcjoe 5y ago

I think this depends upon the language intrinsic support for vectors/vector operations. I would (generally) prefer to specify operations using a simpler, more concise syntax, and have the compiler map that into the hardware. The problem with this is, most compilers don't do such a good job with that. I like the Matrix/vector operations by default for the reason that it becomes quite a bit simpler to understand the written code. Expressing things the way Numpy forces you to, means that you have to reason about the interface of your code to Numpy, as well as how Numpy will operate. For matrix mult, I agree, writing out loops is tedious. This is why things like M = A * B, where M, A, B are all matrices, is far, far better than writing out the loop representation. I agree as well, messing up the index ordering is annoying (and I still do this 30+ years onward with Fortran, C, etc.) Julia makes this stuff easy to trivial. It makes the interface to using this capability easy to trivial. It makes for good overall performance (I rarely run codes only once).