7 ms·

Now, all we need for crystal to succeed is numpy, scipy, something pandas-like, and of course matplotlib (not plots, matplotlib). By the way, how are we on dat

Now, all we need for crystal to succeed is numpy, scipy, something pandas-like, and of course matplotlib (not plots, matplotlib). By the way, how are we on data-science friendly IDEs? Debugging? Automatic thorough documentation generator? Tooling in general? Is the time to first plot fast? Where I work, we desperately need a fast python.

mumblemumble 4y ago
I've tried a few times now to do data science in a statically typed language, and I just haven't enjoyed the developer experience. In general I tend to think the advantages of dynamic typing are overblown, but this is one problem domain where it seems to be indispensable. ML and data engineering is a different story, of course. Also, I wouldn't be surprised if something clever could be done with a structural type system.
- siproprio 4y ago
  What languages have you tried, and what in them did not work? What’s wrong with casting variables to mutate their type?
- wudangmonk 4y ago
  Static typing is like salt, sprinkle a little bit and it enhances and gives flavor to your food. Too much of it and it completely ruins the dish.
  - keyle 4y ago
    Nicely said! Let me try... Types are like guard rails, you don't think you need them till you fly off the road.
  - dunefox 4y ago
    It only "ruins the dish" if you don't want your dish to be correct.
    - wudangmonk 4y ago
      Its either tasty or salty, never heard of a correct dish.
  - galangalalgol 4y ago
    Why would you want "a little" static typing? What does that mean? I used to write python in vim and hated it, made tons of mistakes that were time consuming to find. Type hinting religioussly and using pylance in vscode sped me up a great deal. But I still make those mistakes sometimes, and they are still troublesome to find. Sometimes they pass unit tests because the types are off in the test too. I think a compiled strongly and statically typed language with excellent type inference is the best of both worlds, freedom with a safety net. The problem with interpreted or jit languages is that even with strong typing, inference won't save you until runtime. I'm enjoying julia, but finding type errors at runtime is a downside.
kimburgess 4y ago
Data science is a domain crystal could excel in. Having both an approachable syntax and the feedback that the compiler and type system provides makes it an excellent companion for that work. There's some interesting things you can do with the type system too, like capturing dimensionality in the type: https://git.sr.ht/~kb/matrix/tree/main/item/src/matrix.cr https://git.sr.ht/~kb/matrix/tree/main/item/src/matrix.cr One missing item though is SIMD support: https://github.com/crystal-lang/crystal/issues/3057 https://github.com/crystal-lang/crystal/issues/3057 Re docgen - that's built into the compiler: https://crystal-lang.org/reference/1.5/syntax_and_semantics/documenting_code.html https://crystal-lang.org/reference/1.5/syntax_and_semantics/...
db65edfc7996 4y ago
You would also need a Notebook equivalent REPL for fast iteration.
- kimburgess 4y ago
  https://crystal-lang.org/2021/12/29/crystal-i.html https://crystal-lang.org/2021/12/29/crystal-i.html
e12e 4y ago
>... numpy, scipy, something pandas-like, and of course matplotlib... Where I work, we desperately need a fast python. It sounds like Julia would be a better fit than Crystal? Ironically they have similar problems: compile/start-up time (though crystal has working ahead of time compilation - AFAIK Julia developers are still working on speeding up "first run" in various ways).
- siproprio 4y ago
  At first I thought Julia could fill role, but it has many problems with correctness (bugs), proper documentation, tooling…

mumblemumble 4y ago

I've tried a few times now to do data science in a statically typed language, and I just haven't enjoyed the developer experience. In general I tend to think the advantages of dynamic typing are overblown, but this is one problem domain where it seems to be indispensable. ML and data engineering is a different story, of course. Also, I wouldn't be surprised if something clever could be done with a structural type system.

siproprio 4y ago
What languages have you tried, and what in them did not work? What’s wrong with casting variables to mutate their type?
wudangmonk 4y ago
Static typing is like salt, sprinkle a little bit and it enhances and gives flavor to your food. Too much of it and it completely ruins the dish.
- keyle 4y ago
  Nicely said! Let me try... Types are like guard rails, you don't think you need them till you fly off the road.
- dunefox 4y ago
  It only "ruins the dish" if you don't want your dish to be correct.
  - wudangmonk 4y ago
    Its either tasty or salty, never heard of a correct dish.
- galangalalgol 4y ago
  Why would you want "a little" static typing? What does that mean? I used to write python in vim and hated it, made tons of mistakes that were time consuming to find. Type hinting religioussly and using pylance in vscode sped me up a great deal. But I still make those mistakes sometimes, and they are still troublesome to find. Sometimes they pass unit tests because the types are off in the test too. I think a compiled strongly and statically typed language with excellent type inference is the best of both worlds, freedom with a safety net. The problem with interpreted or jit languages is that even with strong typing, inference won't save you until runtime. I'm enjoying julia, but finding type errors at runtime is a downside.

kimburgess 4y ago

Data science is a domain crystal could excel in. Having both an approachable syntax and the feedback that the compiler and type system provides makes it an excellent companion for that work. There's some interesting things you can do with the type system too, like capturing dimensionality in the type: https://git.sr.ht/~kb/matrix/tree/main/item/src/matrix.cr https://git.sr.ht/~kb/matrix/tree/main/item/src/matrix.cr One missing item though is SIMD support: https://github.com/crystal-lang/crystal/issues/3057 https://github.com/crystal-lang/crystal/issues/3057 Re docgen - that's built into the compiler: https://crystal-lang.org/reference/1.5/syntax_and_semantics/documenting_code.html https://crystal-lang.org/reference/1.5/syntax_and_semantics/...

db65edfc7996 4y ago

You would also need a Notebook equivalent REPL for fast iteration.

kimburgess 4y ago
https://crystal-lang.org/2021/12/29/crystal-i.html https://crystal-lang.org/2021/12/29/crystal-i.html

e12e 4y ago

>... numpy, scipy, something pandas-like, and of course matplotlib... Where I work, we desperately need a fast python. It sounds like Julia would be a better fit than Crystal? Ironically they have similar problems: compile/start-up time (though crystal has working ahead of time compilation - AFAIK Julia developers are still working on speeding up "first run" in various ways).

siproprio 4y ago
At first I thought Julia could fill role, but it has many problems with correctness (bugs), proper documentation, tooling…