4 ms·

I've found that PyTorch is the smoothest for math <-> programming (at least when it comes to vector-like calculus). See for example gradient descent in maths (

by stared 6y ago

I've found that PyTorch is the smoothest for math <-> programming (at least when it comes to vector-like calculus). See for example gradient descent in maths (LaTeX) and in PyTorch: https://colab.research.google.com/github/stared/thinking-in-tensors-writing-in-pytorch/blob/master/2%20Gradient%20Descent.ipynb https://colab.research.google.com/github/stared/thinking-in-... For the smoothest math <-> programming

dunefox 6y ago
While I like PyTorch very much I have found Julia + Flux to be even better.
- stared 6y ago
  Could you show some examples? I did try to use Julia quite a few times (including, I don't know 8 years ago), as I loved the philosophy (brief yet fast, types). Sadly, I never considered it readable - a mix of new concepts, legacy MATLAB syntax, and in general disregard for this part (even function names didn't have consistent naming). If it moved a lot with that respect, I would be happy some nice examples.
  - dunefox 6y ago
    I mean, that's a strange thing to say since PyTorch and especially Python are much further from the math than Julia and Flux. For example, if I want to regularise a layer: PyTorch: l2 = layer.pow(2).sum() loss += lambda * l2 Julia: l2 = sum(layer .^ 2) loss += λ * l2 I can just use regular Julia functions whereas in PyTorch I need to use special PyTorch functions as to not detach the tape. This only gets worse and less readable the more complex things I want to do.
    - stared 6y ago
      I guess one core thing is that I don't think that the traditional mathematical notation is particularly consistent. (For a reference, I published over a dozen mathematical papers.) I love when the "data-flow" is consistent, from left to right. The Julia code (and traditional mathematical notation) reads: middle, right, left. Plus, instead of inventing infix notation, I find it cleaner to write custom methods/functions rather than rely on (a fixed, limited, and non-apparent) set of built-in inflix operators. Made-up .^
enriquto 6y ago
I have the contrary impression. Fortran and matlab/octave are great for numeric math. Julia is also very good. The python math stuff (either torch or numpy) is always an unreadable mess, you are always fighting against the language. The matrix product and exponentiation operators look like a bad joke. I do not understand how people can take this stuff seriously.
- stared 6y ago
  To some extent, it is a matter of taste. Personally, I love functional notation (technically, in PyTorch its chaining) e.g.: x.matmul(y).pow(2).sum() This way we can rite a lot of things, and we don't need to make up a new combination of punctuation marks and special characters for an operation. For example, while one can write in Python: torch.sum((x @ y) 2) I consider it less readable. I mean, here maybe it is fine, but once it gets longer, more complicated, or we want to add new operations, it turns into a mess. Vide "style" section in: https://github.com/stared/thinking-in-tensors-writing-in-pytorch https://github.com/stared/thinking-in-tensors-writing-in-pyt...
  - enriquto 6y ago
    Both of your examples are equally ugly and equally unreadable to me. Why cannot it be something like Σ(x*y)^2 ? Why the extraneous words "torch", "matmul" and the ridiculous operator @ ?
    - dunefox 6y ago
      I'm of the same opinion. In Julia it would be sum(A * B) ^ 2 If you define sum as Σ then you can do Σ(A * B) ^ 2
      - enriquto 6y ago
        yes I like Julia very much because it seems to put the natural instincts of mathematicians upfront. When programming in Julia you do not have the impression of fighting against the language continuously. Everything that you want to do is accessible, and it is alright if you implement your computation in the most natural way possible. If I try to do that in python, I always end with an ugly mess of numba, numpy and calls to C code used at different places for different reasons.

by stared 6y ago

dunefox 6y ago

While I like PyTorch very much I have found Julia + Flux to be even better.

stared 6y ago
Could you show some examples? I did try to use Julia quite a few times (including, I don't know 8 years ago), as I loved the philosophy (brief yet fast, types). Sadly, I never considered it readable - a mix of new concepts, legacy MATLAB syntax, and in general disregard for this part (even function names didn't have consistent naming). If it moved a lot with that respect, I would be happy some nice examples.
- dunefox 6y ago
  I mean, that's a strange thing to say since PyTorch and especially Python are much further from the math than Julia and Flux. For example, if I want to regularise a layer: PyTorch: l2 = layer.pow(2).sum() loss += lambda * l2 Julia: l2 = sum(layer .^ 2) loss += λ * l2 I can just use regular Julia functions whereas in PyTorch I need to use special PyTorch functions as to not detach the tape. This only gets worse and less readable the more complex things I want to do.
  - stared 6y ago
    I guess one core thing is that I don't think that the traditional mathematical notation is particularly consistent. (For a reference, I published over a dozen mathematical papers.) I love when the "data-flow" is consistent, from left to right. The Julia code (and traditional mathematical notation) reads: middle, right, left. Plus, instead of inventing infix notation, I find it cleaner to write custom methods/functions rather than rely on (a fixed, limited, and non-apparent) set of built-in inflix operators. Made-up .^

enriquto 6y ago

I have the contrary impression. Fortran and matlab/octave are great for numeric math. Julia is also very good. The python math stuff (either torch or numpy) is always an unreadable mess, you are always fighting against the language. The matrix product and exponentiation operators look like a bad joke. I do not understand how people can take this stuff seriously.

stared 6y ago
To some extent, it is a matter of taste. Personally, I love functional notation (technically, in PyTorch its chaining) e.g.: x.matmul(y).pow(2).sum() This way we can rite a lot of things, and we don't need to make up a new combination of punctuation marks and special characters for an operation. For example, while one can write in Python: torch.sum((x @ y) 2) I consider it less readable. I mean, here maybe it is fine, but once it gets longer, more complicated, or we want to add new operations, it turns into a mess. Vide "style" section in: https://github.com/stared/thinking-in-tensors-writing-in-pytorch https://github.com/stared/thinking-in-tensors-writing-in-pyt...
- enriquto 6y ago
  Both of your examples are equally ugly and equally unreadable to me. Why cannot it be something like Σ(x*y)^2 ? Why the extraneous words "torch", "matmul" and the ridiculous operator @ ?
  - dunefox 6y ago
    I'm of the same opinion. In Julia it would be sum(A * B) ^ 2 If you define sum as Σ then you can do Σ(A * B) ^ 2
    - enriquto 6y ago
      yes I like Julia very much because it seems to put the natural instincts of mathematicians upfront. When programming in Julia you do not have the impression of fighting against the language continuously. Everything that you want to do is accessible, and it is alright if you implement your computation in the most natural way possible. If I try to do that in python, I always end with an ugly mess of numba, numpy and calls to C code used at different places for different reasons.