7 ms·

I second this. Python is actually starting to get significant traction in the scientific community. Depending on the field, R, Fortran and Matlab (and even C++)

• oefrha 6y ago
  I don’t have much insight on the scientific computing landscape in general, but here’s one notable data point: I worked on the CMS experiment of LHC (Large Hadron Collider) for a while, which is one of the highest profile experiments in experimental physics. The majority of CMS code is C++, which you can check for yourself at https://github.com/cms-sw/cmssw https://github.com/cms-sw/cmssw (yes, much/most? of the code is open source). What I worked on specifically was prototyped in Python, then ported to C++ and plugged into the massive data processing pipeline where performance is critical due to the sheer amount of data. So I probably wouldn’t put C++ in parentheses.
  • pansa2 6y ago
    > prototyped in Python, then ported to C++ This need to rewrite, of course, is what Julia is trying to avoid. My workflow is exactly the same, and I’d love to be able to write code in a high-level language like Python and then use that directly instead of having to rewrite. However, in my case the reason for rewriting isn’t just performance, but also to be able to build compiled binaries. Julia aims to be as high-level as Python but faster - is there a language that’s as high-level as Python but AOT-compiled?
    • oefrha 6y ago
      Nim? I know it has Python-like syntax and aims to be performant, but don’t know much beyond that.
      • 0-_-0 6y ago
        Indeed, the Julia autodiff implementation linked above would look very similar in Nim as well.
    • mlthoughts2018 6y ago
      Cython - in fact I think in 2021 if you want to write a pure C or pure C++ program, Cython is the best way to go, and just disable use of CPython. The “need to rewrite” is actually a sort of advantage with Cython. You only target small pieces of your program to be compiled to C or C++ for optimization, and the rest where runtime is already fast enough or otherwise doesn’t matter, you seamlessly write in plain Python. Using extension modules is just a time-tested, highly organized, modular, robust design pattern. Julia and others do themselves a disservice by trying to make “the whole language automatically optimized” which counter-intuitively is worse than make the language overall optimized for flexibility instead of speed, yet with an easy system to patch optimization modules anywhere they are needed.
      • szemet 6y ago
        > Using extension modules is just a time-tested, highly organized, modular, robust design pattern. I really don't get this. I'am fully on the side that limitations may increase design quality. E.g I accept the argument that Haskell immutability often leads to good design, I also believe the same true for Rust ownership rules (it often forces a design where components have a well defined responsibility: this component only manages resource X starting from { until }.) But having a performance boundary between components, why would that help? E.g. This algorithm will be fast with floats but will be slow with complex numbers. Or: You can provide X,Y as callback function to our component, it will be blessed and fast, but providing your custom function Z it will be slow. So you should implement support for callback Z in a different layer but not for callback X,Y, and you should rewrite your algorithm in a lower level layer just to support complex numbers. Will this really lead to a better design?
        • mlthoughts2018 6y ago
          > “But having a performance boundary between components, why would that help?” It helps precisely so you don’t pay premature abstraction costs to over-generalize the performance patterns. One of my biggest complaints with Julia is that zealots for the language insist these permeating abstractions are costless, but they totally aren’t. Sometimes I’m way better off if not everything up the entire language stack is differentiable and carries baggage with it needed for that underlying architecture. But Julia hasn’t given me the choice of this little piece that does benefit from it vs that little piece that, by virtue of being built on top of the same differentiability, is just bloat or premature optimization. > “you should rewrite your algorithm in a lower level layer just to support complex numbers.” Yes, precisely. This maximally avoids premature abstraction and premature extensibility. And if, like in Cython, the process of “rewriting” the algorithm is essentially instantaneous, easy, pleasant to work with, then the cost is even lower. This is why you have such a spectrum in Python. 1. Create restricted computation domains (eg numpy API, pandas API, tensorflow API) 2. Allow each to pursue optimization independently, with clear boundaries and API constraints if you want to hook in 3. When possible, automate large classes of transpilation from outside the separate restricted computation domains to inside them (eg JITs like numba), but never seek a pan-everything JIT that destroys the clear boundaries 4. For everything else (eg cases where you deliberately don’t want a JIT auto-optimizing because you need to restrict the scope or you need finer control), use Cython and write your Python modules seamlessly with some optimization-targeting patches in C/C++ and the rest in just normal, easy to use Python.
    • pjmlp 6y ago
      > is there a language that’s as high-level as Python but AOT-compiled? Common Lisp, Ocaml for example.
  • bsdubernerd 6y ago
    Just for reference, my experience is mostly computational genomics. R is king of analysis, and most of the actual "meat" is implemented in C++. But I work with other teams as well, so the experience is a bit more varied if you look across different areas.