19 ms·

Sounds like exactly the problem of Julia language. On the surface, Julia seems slick in code and fast in running. But then you discover that 1. there are no ob

Sounds like exactly the problem of Julia language. On the surface, Julia seems slick in code and fast in running. But then you discover that 1. there are no objects, so when you switch from Python, instead of chains of methods, like `my_dataframe.groupby(...).aggregate(...).groupby(...).aggregate(...)` you need either (a) a lot of poorly readable nested calls, or (b) ugly "pipes", where the left side is implicit first param of the function on the right side: ` my_df |> groupby(...) |> aggregate(...) |> groupby(...) |> aggregate(...)`. (Writing them, you feel insecure of what is happening -- in this case, what gets passed to the function.) 2. You can't make it a CLI script, because it compiles the whole code WITH dependencies every time, and as soon as you import some serious libraries, compile times will skyrocket. I quickly hit 40 seconds of compilation with just geospatial lib and dataframes. These 40 seconds turned out to be A LOT when you develop interactively. And you can't build everything in a Jupyter Notebook, because eventually you'll have to switch to a CLI script. I think I could put up with pipes, but the compilation times were the final argument against Julia.

troupo 3y ago
> (b) ugly "pipes", where the left side is implicit first param of the function on the right side: ` my_df |> groupby(...) |> aggregate(...) |> groupby(...) |> aggregate(...)`. (Writing them, you feel insecure of what is happening -- in this case, what gets passed to the function.) These are not ugly pipes. These are a very common feature in a lot of functional languages. Once you've used them, you want them everywhere :) And of course you know exactly what is passed to the function. You wrote it yourself: left side is passed as the first param of the function on the right side.
- thumbuddy 3y ago
  I think it's not so clear a lot of times which function is called. Say you write a function that accepts your type as an argument. Cool, you compile it everything is good. Now you import a library, just so happens it also has a function that has the same name, and accepts an argument of the same parent type. Which function gets called? If you're reading the code and not intimately familiar with how imports work, all the imports in scope, and the function your coworker defined on like 5,000 in a random file you might not be sure unless you start debugging. Pipes are fine, but methods and traits with a strong type system reduce any ambiguity there. Doesn't mean you can't use pipes reasonably! But in Julia that seems like it could be harder to unpack.
  - StackOverlord 3y ago
    > Say you write a function that accepts your type as an argument. Cool, you compile it everything is good. Now you import a library, just so happens it also has a function that has the same name, This has nothing to do with pipes, this is a matter of namespacing.
    - thumbuddy 3y ago
      Well when methods by definition associate a type instance to a function the name spacing is obvious...
      - sodapopcan 3y ago
        This is why I generally dislike imports. It's common practice in Elixir (and I believe some other functional languages) to generally fully qualify function calls. " hello world " |> String.trim() |> String.split() |> Enum.map(fn char -> String.uppercase(char) end) |> Enum.join() #=> "HELLO WORLD" I'm sure some will think that is too verbose but I love it. Otherwise pipes are far more flexible than method chaining just because you aren't constrained to one "type". You don't always want to do this, but as per the example above, it's pretty handy. You also don't have to rely on a method you don't own returning an instance of itself.
        em-bee 3y ago
        well, it depends on how import works. in python you can't use a module unless you import it, (though you can import it in a way that the full path to the function is still needed) but for example in pike import is only useful as a shortcut, and not needed if you use fully qualified function calls. and like you i never saw the point of import. using the fully qualified function calls makes the code more readable because i can quickly recognize library functions from custom code.
        sodapopcan 3y ago
        True enough about Python but I was responding more about pipes in general which Python doesn't have. I'm more confused about the original assertion that pipes give you lower confidence as to what is happening. But then I looked up how they work in Julia and saw there are two different ways they are implemented so I agree that that is maybe a little bit unfortunate, though I can't really say for sure as I've never written a line of Julia.
- culebron21 3y ago
  Pipe is useful, of course. I personally just dislike the symbols selected for it, they're harder to type. And no, in many cases, it's absolutely not clear what gets passed, because you may have a vector of scalars, and the function accepts... what? Is it restricted in input types at all? If it takes a scalar and then gets passed a vec of them (but you don't call it's vectorized equivalent), what happens? Yes, I can go check the function, check if I have a vec and check if I call the function the proper (scalar or vectored) way,.. but this escalates the complexity and choices you should make, towards Rust language. And still it can run and try doing something implicitly. When I coded it, I wasn't sure I can't mess things up somewhere in the middle of the pipe. Don't get me wrong, I liked many features of Julia. I made one project in several languages just to compare, and Julia was easy to learn and worked pretty well. The main issue that stopped me from using it was the above mentioned cold start time.
  - em-bee 3y ago
    i haven't used a lanuage with this feature before so i don't know anything, but i don't understand the problem. how is the return value before the pipe being the first argument any different than normally putting it into the argument list in the first position. or is there an implementation where a pipe takes a vector and applies it to the next function as multiple arguments?
    - culebron21 3y ago
      Dataframes are vectors of vectors. And functions on them may return either vec<scalar>, or vec<vec<scalar>>, and in a chain of such operations it's easy to lose track of what you have.

troupo 3y ago

> (b) ugly "pipes", where the left side is implicit first param of the function on the right side: ` my_df |> groupby(...) |> aggregate(...) |> groupby(...) |> aggregate(...)`. (Writing them, you feel insecure of what is happening -- in this case, what gets passed to the function.) These are not ugly pipes. These are a very common feature in a lot of functional languages. Once you've used them, you want them everywhere :) And of course you know exactly what is passed to the function. You wrote it yourself: left side is passed as the first param of the function on the right side.

thumbuddy 3y ago
I think it's not so clear a lot of times which function is called. Say you write a function that accepts your type as an argument. Cool, you compile it everything is good. Now you import a library, just so happens it also has a function that has the same name, and accepts an argument of the same parent type. Which function gets called? If you're reading the code and not intimately familiar with how imports work, all the imports in scope, and the function your coworker defined on like 5,000 in a random file you might not be sure unless you start debugging. Pipes are fine, but methods and traits with a strong type system reduce any ambiguity there. Doesn't mean you can't use pipes reasonably! But in Julia that seems like it could be harder to unpack.
- StackOverlord 3y ago
  > Say you write a function that accepts your type as an argument. Cool, you compile it everything is good. Now you import a library, just so happens it also has a function that has the same name, This has nothing to do with pipes, this is a matter of namespacing.
  - thumbuddy 3y ago
    Well when methods by definition associate a type instance to a function the name spacing is obvious...
    - sodapopcan 3y ago
      This is why I generally dislike imports. It's common practice in Elixir (and I believe some other functional languages) to generally fully qualify function calls. " hello world " |> String.trim() |> String.split() |> Enum.map(fn char -> String.uppercase(char) end) |> Enum.join() #=> "HELLO WORLD" I'm sure some will think that is too verbose but I love it. Otherwise pipes are far more flexible than method chaining just because you aren't constrained to one "type". You don't always want to do this, but as per the example above, it's pretty handy. You also don't have to rely on a method you don't own returning an instance of itself.
      - em-bee 3y ago
        well, it depends on how import works. in python you can't use a module unless you import it, (though you can import it in a way that the full path to the function is still needed) but for example in pike import is only useful as a shortcut, and not needed if you use fully qualified function calls. and like you i never saw the point of import. using the fully qualified function calls makes the code more readable because i can quickly recognize library functions from custom code.
        sodapopcan 3y ago
        True enough about Python but I was responding more about pipes in general which Python doesn't have. I'm more confused about the original assertion that pipes give you lower confidence as to what is happening. But then I looked up how they work in Julia and saw there are two different ways they are implemented so I agree that that is maybe a little bit unfortunate, though I can't really say for sure as I've never written a line of Julia.
culebron21 3y ago
Pipe is useful, of course. I personally just dislike the symbols selected for it, they're harder to type. And no, in many cases, it's absolutely not clear what gets passed, because you may have a vector of scalars, and the function accepts... what? Is it restricted in input types at all? If it takes a scalar and then gets passed a vec of them (but you don't call it's vectorized equivalent), what happens? Yes, I can go check the function, check if I have a vec and check if I call the function the proper (scalar or vectored) way,.. but this escalates the complexity and choices you should make, towards Rust language. And still it can run and try doing something implicitly. When I coded it, I wasn't sure I can't mess things up somewhere in the middle of the pipe. Don't get me wrong, I liked many features of Julia. I made one project in several languages just to compare, and Julia was easy to learn and worked pretty well. The main issue that stopped me from using it was the above mentioned cold start time.
- em-bee 3y ago
  i haven't used a lanuage with this feature before so i don't know anything, but i don't understand the problem. how is the return value before the pipe being the first argument any different than normally putting it into the argument list in the first position. or is there an implementation where a pipe takes a vector and applies it to the next function as multiple arguments?
  - culebron21 3y ago
    Dataframes are vectors of vectors. And functions on them may return either vec<scalar>, or vec<vec<scalar>>, and in a chain of such operations it's easy to lose track of what you have.