4 ms·

For me, the issue is that there's too much magic happening, e.g. with macros, but finding or solving the issues seems needlessly convoluted. Haskell, Rust, Pyth

For me, the issue is that there's too much magic happening, e.g. with macros, but finding or solving the issues seems needlessly convoluted. Haskell, Rust, Python, or even C++ feel a lot less magic, and more reasonable despite all being quite different.

wiz21c 3y ago
I like rust a lot but when you get into errors because some deeply generic trait can't be matched, believe me, the compiler's messages are cryptic :-)
- PartiallyTyped 3y ago
  I'd love to see examples :D
anewhnaccount2 3y ago
Certainly there is a lot of magic going on in very popular Python libraries that implement autodiff or JIT compilation like PyTorch or JAX -- although the maturity of the ecosystem does mean that it is often quite well hidden. For the other languages, well... if you don't think C++'s templates tricks, Haskell's extension potpourri or indeed Rust's own combination of traits and type-level programming techniques can get convoluted, I'm not sure what to say.
- PartiallyTyped 3y ago
  I really don't feel that there is magic in PyTorch or jax, but that may be because I have written my own autograd libs. In PyTorch you have a graph that is created on runtime by connecting the operations together in a transparent manner. Jax may feel a bit magic, but all that's done is sending / splitting tracers and recording the operations and compiling; by limiting the language, you have controlled branching with the proper semantics. --- The main reason I have had such with Julia is simply because of how early / soon I ended up needing to use them whereas with the other languages, you can get away without getting into the very messed up things.
  - kristofferc 3y ago
    What did you feel you needed to use macros for?
    - PartiallyTyped 3y ago
      I can’t remember, it was some statistical tools, running a few simulations, it was 2 years ago.
  - freilanzer 3y ago
    > The main reason I have had such with Julia Such what? Fun? ;)
  - mathisfun123 3y ago
    >In PyTorch you have a graph that is created on runtime by connecting the operations together in a transparent manner. You've jumped the shark here mate because autodiff in PyTorch is implemented using compile-time generated code in libtorch - it's not only the very definition of opaque but also pretty close in spirit to a macro.
    - PartiallyTyped 3y ago
      Transparent in this case means the average user does not need to interact with the underlying system, it is there, it does stuff, but the user does not see it or interact directly. What is opaque is jax's tracer objects in the sense that the system exists, is there, the user is aware that it is there, and you can't peek inside it without actively going out of your way. With PyTorch you can implement your own functions with their own forward and backward passes; whether the underlying operations use libtorch is irrelevant, that is simply a backend. The computation graph __is__ created at runtime, whether the scaffolding for this is done in libtorch or directly in python is not relevant.
      - mathisfun123 3y ago
        well i mean if we redefine all the words so they mean whatever we want them to mean then sure blob code that isn't visible even in the source tree can be considered transparent and conversely, perfectly accessible script code (jax's interpreters) can be considered opaque what do i know i'm just a contrib on both projects shrug lots of your other claims are just complete misunderstandings e.g. whether the graph is created at runtime - it's not and you can read adam's original paper to find that out if you don't believe me - and the idea that libtorch is just a backend - it's not - it is, for all intents and purposes, pytorch.

wiz21c 3y ago

I like rust a lot but when you get into errors because some deeply generic trait can't be matched, believe me, the compiler's messages are cryptic :-)

PartiallyTyped 3y ago
I'd love to see examples :D

anewhnaccount2 3y ago

Certainly there is a lot of magic going on in very popular Python libraries that implement autodiff or JIT compilation like PyTorch or JAX -- although the maturity of the ecosystem does mean that it is often quite well hidden. For the other languages, well... if you don't think C++'s templates tricks, Haskell's extension potpourri or indeed Rust's own combination of traits and type-level programming techniques can get convoluted, I'm not sure what to say.

PartiallyTyped 3y ago
I really don't feel that there is magic in PyTorch or jax, but that may be because I have written my own autograd libs. In PyTorch you have a graph that is created on runtime by connecting the operations together in a transparent manner. Jax may feel a bit magic, but all that's done is sending / splitting tracers and recording the operations and compiling; by limiting the language, you have controlled branching with the proper semantics. --- The main reason I have had such with Julia is simply because of how early / soon I ended up needing to use them whereas with the other languages, you can get away without getting into the very messed up things.
- kristofferc 3y ago
  What did you feel you needed to use macros for?
  - PartiallyTyped 3y ago
    I can’t remember, it was some statistical tools, running a few simulations, it was 2 years ago.
- freilanzer 3y ago
  > The main reason I have had such with Julia Such what? Fun? ;)
- mathisfun123 3y ago
  >In PyTorch you have a graph that is created on runtime by connecting the operations together in a transparent manner. You've jumped the shark here mate because autodiff in PyTorch is implemented using compile-time generated code in libtorch - it's not only the very definition of opaque but also pretty close in spirit to a macro.
  - PartiallyTyped 3y ago
    Transparent in this case means the average user does not need to interact with the underlying system, it is there, it does stuff, but the user does not see it or interact directly. What is opaque is jax's tracer objects in the sense that the system exists, is there, the user is aware that it is there, and you can't peek inside it without actively going out of your way. With PyTorch you can implement your own functions with their own forward and backward passes; whether the underlying operations use libtorch is irrelevant, that is simply a backend. The computation graph __is__ created at runtime, whether the scaffolding for this is done in libtorch or directly in python is not relevant.
    - mathisfun123 3y ago
      well i mean if we redefine all the words so they mean whatever we want them to mean then sure blob code that isn't visible even in the source tree can be considered transparent and conversely, perfectly accessible script code (jax's interpreters) can be considered opaque what do i know i'm just a contrib on both projects shrug lots of your other claims are just complete misunderstandings e.g. whether the graph is created at runtime - it's not and you can read adam's original paper to find that out if you don't believe me - and the idea that libtorch is just a backend - it's not - it is, for all intents and purposes, pytorch.