3 ms·

If they’re data centric, would you not be better off with vanilla python? If you have numpy do the heavy lifting, it’ll be as fast plus all the syntactic/ecosys

If they’re data centric, would you not be better off with vanilla python? If you have numpy do the heavy lifting, it’ll be as fast plus all the syntactic/ecosystem benefits

nkozyra 6y ago
Yeah, I like nim but it isn't going to offer and advantage over python in this regard.
ca_parody 6y ago
The main reasons I see for nim in scientific computing over python are: (a) you need performance for applications outside of numpy / pandas & you want to avoid writing a C extension to do so (nim is also actually a good choice for python extensions via nimpy - if you are still using python as the main language of your project/system) (a2) you want good concurrency (some would say python's async/await is good - but it feels too little too late to me) & you at least want the option of parallelism (like this article takes advantage of) (b) you want / need the powerful macro system for dsls / custom (compile-type-checkable) language extensions. There is a good future for nim in scientific computing (frankly, there is a good present) - but for it to be better, people need to be willing to forgo some of the ecosystem (when possible) and to help build that future. https://github.com/nim-lang/needed-libraries/issues/77 https://github.com/nim-lang/needed-libraries/issues/77
logicchains 6y ago
The OP said Scala-like. A big advantage of Scala/Spark over Python is more compile-time checking: the code is much less likely to fail at runtime. Particulary helpful for production code (e.g. pipelines). For people coming from an engineering background (as opposed to data science), Python's lack of any static type checking for data processing code can be a big productivity killer. In that sense, Nim might be able to offer similar advantages to Scala.
nimmer 6y ago
Numpy is nice but pure-Nim applications appear to outperform it easily: https://narimiran.github.io/2018/05/10/python-numpy-nim.html https://narimiran.github.io/2018/05/10/python-numpy-nim.html https://github.com/mratsim/Arraymancer https://github.com/mratsim/Arraymancer
- qeternity 6y ago
  Interesting but given the low total execution time, the majority of time in the numpy implementation is going to be spent initializing CPython. I think if this were on the order of minutes/hours you would see numpy outperform.

If they’re data centric, would you not be better off with vanilla python? If you have numpy do the heavy lifting, it’ll be as fast plus all the syntactic/ecosystem benefits

nkozyra 6y ago

Yeah, I like nim but it isn't going to offer and advantage over python in this regard.

ca_parody 6y ago

The main reasons I see for nim in scientific computing over python are: (a) you need performance for applications outside of numpy / pandas & you want to avoid writing a C extension to do so (nim is also actually a good choice for python extensions via nimpy - if you are still using python as the main language of your project/system) (a2) you want good concurrency (some would say python's async/await is good - but it feels too little too late to me) & you at least want the option of parallelism (like this article takes advantage of) (b) you want / need the powerful macro system for dsls / custom (compile-type-checkable) language extensions. There is a good future for nim in scientific computing (frankly, there is a good present) - but for it to be better, people need to be willing to forgo some of the ecosystem (when possible) and to help build that future. https://github.com/nim-lang/needed-libraries/issues/77 https://github.com/nim-lang/needed-libraries/issues/77

logicchains 6y ago

The OP said Scala-like. A big advantage of Scala/Spark over Python is more compile-time checking: the code is much less likely to fail at runtime. Particulary helpful for production code (e.g. pipelines). For people coming from an engineering background (as opposed to data science), Python's lack of any static type checking for data processing code can be a big productivity killer. In that sense, Nim might be able to offer similar advantages to Scala.

nimmer 6y ago

Numpy is nice but pure-Nim applications appear to outperform it easily: https://narimiran.github.io/2018/05/10/python-numpy-nim.html https://narimiran.github.io/2018/05/10/python-numpy-nim.html https://github.com/mratsim/Arraymancer https://github.com/mratsim/Arraymancer

qeternity 6y ago
Interesting but given the low total execution time, the majority of time in the numpy implementation is going to be spent initializing CPython. I think if this were on the order of minutes/hours you would see numpy outperform.