15 ms·

Python vs. Rust for Neural Networks

290 points by rencire 7y ago

tanilama 7y ago
Nobody writing NN in Python, they are just describing it. For NN or DL in general, the correctness doesn't really lie too much on the code quality level, like ownership Rust people love to talk about. It is more about Numeric stability under/overflow and such. Choice of programming language offers limited help here. I don't think Rust has a killer app for ML/DL community to offer as of now, the focus is vastly different.
- kuzehanka 7y ago
  I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a component that's idle 99% of the time. The bulk of the load in an DL workflow is CUDA code and sits on the GPU. Even the intermediate libraries like cublas would only see marginal-to-none benefits of being reimplemented in rust. This is a cool project, but it has no chance to displace or even complement Python in the data science space.
  - wokwokwok 7y ago
    > comparatively tedious languages like Rust will never attract data science practitioners. Well, fast.ai is using swift now. ... I think it's fair to say 'never say never'. You're probably right, rust isn't really the sweet spot for this stuff, but its also a case that python has some down sides that are pretty severe, and well acknowledged.
    - melling 7y ago
      I wouldn’t call Swift a tedious language. With type inference, immutability, etc, Swift is far from tedious: http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=strings http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=stri... http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=dictionaries http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=dict... It’s not quite as nice as Python but it’s an enjoyable language.
      - frutiger 7y ago
        Rust has those too, GGGP still considered it tedious.
        antisemiotic 7y ago
        I think the tedious part would be the borrow checker. Also, Rust doesn't have top-level type inference (by design).
        skohan 7y ago
        Yeah there is some tedium in general with Rust syntax. Semicolons, for example, feel old fashioned, and there's a lot of verbosity in things like unwrapping. It's a fine language and I like working with it, but there's a lot of details involved in Rust development which don't make sense for data scientists to worry about.
      - wokwokwok 7y ago
        > They don't care about the kind of safety it brings, they don't care about improving performance in a component that's idle 99% of the time. https://www.fast.ai/2019/03/06/fastai-swift/ https://www.fast.ai/2019/03/06/fastai-swift/: > Because Swift for TensorFlow is the first serious effort I’ve seen to incorporate differentiable programming deep in to the heart of a widely used language that is designed from the ground up for performance. > But Python is not designed to be fast, and it is not designed to be safe. Instead, it is designed to be easy, and flexible. To work around the performance problems of using “pure Python” code, we instead have to use libraries written in other languages (generally C and C++), like numpy, PyTorch, and TensorFlow, which provide Python wrappers. To work around the problem of a lack of type safety, recent versions of Python have added type annotations that optionally allow the programmer to specify the types used in a program. However, Python’s type system is not capable of expressing many types and type relationships, does not do any automated typing, and can not reliably check all types at compile time. Therefore, using types in Python requires a lot of extra code, but falls far short of the level of type safety that other languages can provide. ie. My point: OP's point is categorically false. ...not that swift is tedious. I <3 swift.
    - chillee 7y ago
      Just because fast.ai has some investment in Swift does not mean that S4TF has attracted mind share. The bulk of fast.ai is still taught on Pytorch.
      - wokwokwok 7y ago
        The parent comment literally said data science practitioners don't care about speed or safety because the GPU is where all the real work happens; that's false, I've provided an example of it being false from a respected party. What do you want me to say? eh, I give up. Believe whatever you want to believe.
        chillee 7y ago
        I'm not saying that no data science practitioners care about speed and safety, and its easy for me to come up with cases where they should/do care. My point is that even fast.ai still views S4TF as somewhat niche, and that data science practitioners as a whole still don't care.
        MiroF 7y ago
        I am a practitioner, I care, but everyone on this thread seems to be distracted by the red herring of speed and efficiency. None of these languages are going to be speedier in the GPU. But python has serious design problems that manifest itself when developing a large project. You can only go so far with dynamic typing
  - xxxpupugo 7y ago
    CUDA is an important part of the story. I think the industry is moving to 'MLIR' solution (Yes, there is a Google project called exactly that, but I am referring to the general idea here), where the network is defined and trained in one place, then the weights are exported, delegated to optimized runtime to be executed. If such trend furthers down, then there will be very little reason to replace Python as the glue layer here. Instead it will become that everything is training in Python -> exported to shared format -> executed in optimized runtime, kind of flow. Rust's opportunity could be to replace C++ in this case. But do mind that this is also a competitive business, where the computation is pushed further down to the hardware implementations, like TPUv1 and T4 chips etc.
    - skohan 7y ago
      Personally I hope for alternatives because I don't find Python that nice to work with compared to other languages. It's not awful, but I really miss having type and nullability errors detected by the compiler and reported to me with meaningful error messages. Also there are a few weird things with the Python workflow, like dealing with Python2 vs Python3, and generally having to run everything in the context of a virtual environment, and the fact that it doesn't seem to really have a single "right way" to deal with dependency management. It's a bit like the story with imports in Javascript: those problems were never really solved, and the popular solutions are just sort of bolted on to the language. It's amazing how many great tools there are available in Python, but it does sometimes seem like it's an under-powered tool which has been hacked to serve bigger problems than it was intended for.
      - ImNotTheNSA 7y ago
        > like dealing with Python2 vs Python3, and generally having to run everything in the context of a virtual environment, and the fact that it doesn't seem to really have a single "right way" to deal with dependency management. Not to mention, this problem seems to be getting worse, not better. People are moving off of python 2.7, which was kind of the de-facto LTS release of python... leaving (currently) no LTS version of python and no clear path for the community to establish a new LTS version that the community will still support — there are so many releases with so many breaking changes in python 3 within the last few years that there is seemingly no consensus and no way to compromise. > It's amazing how many great tools there are available in Python, but it does sometimes seem like it's an under-powered tool which has been hacked to serve bigger problems than it was intended for. This is becoming more and more clear with every release of python IMO. The language is evolving too quickly for it to be used on large developments, but it’s still being used for that. We have an entire test framework and libraries for high performance embedded systems level testing which is written entirely in python. The amount of high speed operations (timing tests, measurements, etc) is very obviously overwhelming for the intention of the language and libraries, yet the company keeps pushing ahead with this stuff. In order to mitigate the issue, we are developing more high speed embedded systems to offload the work from the test framework and report measurements to the framework. I think it’s quickly becoming extremely expensive and will only become more expensive — the framework is extremely “pythonic” to the point of being unreadable, using a number of hacks to break through the python interpreter. Jobs much better allocated to readable C++ libraries are being implemented in unreadable python spaghetti code with no clear architecture - just quick-and-dirty whatever-it-takes python. I love python but I think it’s a quick-and-dirty language for a reason. What python does well cannot be beat by other languages (for example, prototyping), but I think it is often misused, since people can get something up and running quickly and cleanly in python, but it eventually has diminishing (and even negative) returns.
  - pjmlp 7y ago
    Rust might not be it. But AOT/JIT compiled languages that can naturally talk to the GPGPU, without 2nd language syndrome, like Julia, Swift, Java and .NET will certainly be more attractive to data science practitioners. I can already envision those life science guys that migrate to VB.NET when they have outgrown their Excel/VBA code, to start playing with ML.NET.
    - kuzehanka 7y ago
      Python already gets JIT compiled to CUDA[1] and there's an entire funded ecosystem built around python+gpgpu called RAPIDS[2] which is the future of the ML space by most indicators. I don't see any other language even making a dent in the Python ecosystem without some kind of new killer feature that can't be quickly replicated in Python. [1] https://numba.pydata.org https://numba.pydata.org [2] https://rapids.ai https://rapids.ai
      - disgruntledphd2 7y ago
        Thank you for the cite to rapids.ai, that looks extremely interesting :)
      - MiroF 7y ago
        Swift and static type checking and compiler analysis might be that language and feature combo.
      - hsaliak 7y ago
        You beat me to making this point. I don't see anything in the article's python code that the numba's jit decorator cannot handle. When numba works (it's rapidly improving), it's seriously impressive. For this particular case, you should be able to get really good performance without sacrificing readability. Also Jax - https://github.com/google/jax https://github.com/google/jax
    - MiroF 7y ago
      Your last paragraph is my nightmare
  - skohan 7y ago
    This is why I think Swift is a much better choice to replace or at least compliment Python than Rust. It has a modern, powerful type system and all the quality of life advantages which come with it, but it manages this with a lot more usability than Rust. A well written swift framework almost becomes a DSL for the problem domain, which is a great property for a data science tool to have.
    - echelon 7y ago
      Swift is not fun on non-Mac platforms. It feels a lot like Google's Dart in terms of how it was positioned and advocated. Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors. The compiler is incredibly helpful, non-lexical lifetimes are a thing, and unless you're doing a lot of sharing and parallelism, you can avoid many borrow checker problems until you learn RAII.
      - skohan 7y ago
        It's a fair criticism of Swift that it is not well supported across platforms. The story is getting better, but it is still miles behind Rust in this regard. I am a huge fan of Swift as a language, but I am very critical of the tooling. > Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors. I beg to differ. I've been programming professionally for over a decade, and I have shipped projects in a variety of languages, and I can safely say that Rust has a steeper learning curve and requires more cognitive overhead to use than many other languages. I find it relatively nice to work with rust in spite of this because the tooling is so great, but it's undeniable that Rust has made tradeoffs which sacrifice ease of use in favor of safety and performance.
        MiroF 7y ago
        The tooling for Swift 4 TF is not anywhere near satisfactory. I can't even seem to get it on my computer without installing XCode (and that's on an apple computer) I should be able to pull a docker image and have s4tf immediately at my fingertips
  - danielscrubs 7y ago
    I am a data scientist and I care. The time when you could just do proof of concepts or a PowerPoint presentation is long behind us. So now we have to start to take it into production, which means we get the exact same problems as SE has always had. Iff Rust helps us take it into production we will use it. But it’s a lot of land to cover to reach Pythons libraries so I’m not holding my breath. That said, Pythons performance is slow even when shuffling to Numpy.
    - kuzehanka 7y ago
      I must be missing something. Modern data science workloads involve fanning out data and code across dozens to hundreds of nodes. The bottlenecks, in order, are: inter-node comms, gpu/compute, on-disk shuffling, serialisation, pipeline starvation, and finally the runtime. Why worry about optimising the very top of the perf pyramid which will make the least difference? Why worry if you spent 1ms pushing data to numpy when that data just spent 2500ms on the wire? And why are you even pushing from python runtime to numpy instead of using arrow?
      - corndoge 7y ago
        Good lord, hopefully latency isn't 2.5 seconds!
        sjwright 7y ago
        I can’t even. How could you ever get 2500 msec on transit? That’s like circling the globe ten times.
        justinclift 7y ago
        Maybe a bunch of SSL cert exchanges through some very low bandwidth connections? ;) Still, it's more likely a figure used for exaggeration, for effect.
        deleted 7y ago
        [deleted]
  - Nullabillity 7y ago
    I'm by no means a specialized data scientist, but I've done some (very surface-level) text crunching with both the TF/Numpy stack and Rust. To me, the nice thing about switching to Rust for that kind of stuff was that it dramatically raised the bar of what I could do before reaching for those hyper-optimized descriptive libraries. Want to calculate the levenshtein distances of the cross product of 100k strings? Sure, just load it into a Vec, find a levenshtein library on crates.io, and it'll probably be fast enough. Could it be done in Python? Sure, but with Rust I didn't have to think about how to do it in a viable amount of time. Does that mean that Rust is going to take over the DS world? Probably not in the short term, Rust currently can't compete with Python's DS ecosystem. But if I'm doing something that I don't know will fit into an existing Python mold (that I know about) then I'll strongly consider using it.
    - smallnamespace 7y ago
      > But if I'm doing something that I don't know will fit into an existing Python mold (that I know about) then I'll strongly consider using it. The thing is, for anything performance intensive and scientific, you're almost guaranteed to find a Python binding. It has all these bindings because scientists are almost always writing either Python, C++, or Fortran (with a smattering of R or Octave on the side). Want to do linear algebra? NumPy is always there, otherwise you can go lower level: https://docs.scipy.org/doc/scipy/reference/linalg.blas.html https://docs.scipy.org/doc/scipy/reference/linalg.blas.html Old-school optimization? https://www.roguewave.com/sites/rw/files/attachments/PyIMSLStudioFeatures.pdf https://www.roguewave.com/sites/rw/files/attachments/PyIMSLS... Computer vision? https://pypi.org/project/opencv-python/ https://pypi.org/project/opencv-python/ In fact, I'd basically argue against using most language-native implementations of algorithms where performance is at stake, because most implementations don't have all the algorithmic optimizations.
      - danielscrubs 7y ago
        And they will cover a small part of what is found in research papers. Python has the best ecosystem, but Rust was made by a competent team so we will root for it.
  - echelon 7y ago
    I think this project has immediate merits when it comes to productionizing small, sparse networks for running on the CPU. I do research and prototyping in Python, but I have to deploy on mobile devices. I was going to roll my own implementation, but now that this exists, it's something I'm going to look into.
    - MiroF 7y ago
      Load the model from torch script? In C++? Don't see why Rust made this possible.
      - echelon 7y ago
        Ideally we'd have native Rust loading of tensorflow, pytorch, etc. models. I've hand-coded a few, but it was pretty monotonous.
- atoav 7y ago
  As somebody who programs in both Python and Rust (and likes both languages) I think Rust's place would be parts of the code that have to be fast, and that you want to get right. Calling Python code from Rust or Rust from Python is totally doable, and there is in my view no reason why you shouldn't use both in the use cases that suit them. And the speed part is serious. Some guy once asked for the fastest tokenizer in any given language and my naive implementation came second place in his benchmark right after an stripped down and optimized C variant. So using Rust for speed critical modules and interfacing them from easy to use Python libraries isn't exactly irrational.
  - gridlockd 7y ago
    Calling C (or C++) code from Python is not only totally doable, it's what all the popular libraries do. Furthermore, most ML stuff runs on the GPU, which also is C-like code. Hence, Rust offers no performance benefit that isn't already there. It really only offers safety and modern language features, at the cost of being tedious to use.
    - atoav 7y ago
      A bit snarky isn’t it? I explained why it isn’t irrational to use Rust — nowhere did I say you have to use Rust or that Rust is the only solution to that problem (which it obviously isn’t). Assuming the person opposite doesn’t know things is an extra unpleasant move on your part and I am not sure why you feel attacked by any of what I wrote. For the record, I also use C and I know that expert level C can beat naive Rust at any time (expert level Rust should allow you to do literally the same thing you do in C, just wrapped in a “I can handle this”-unsafe block. Your response tells more about yourself than it does about the subject at hand. Rust isn’t stealing your cookies and nor am I.
      - ImNotTheNSA 7y ago
        Sounds like you’re over reading into what was essentially a two-point comment... there was no snarkiness in that comment at all - just a statement that rust is not really offering much benefit for this circumstance. Yeesh... projection
- drewm1980 7y ago
  You're right, and... I'm working professionally using DL for computer vision, for robotics. We spend the majority of our time writing the business logic around the learned parts. To get all of that code bug free and fast is way harder in python that I expect it would be in Rust. Rust's ndarray ecosystem is immature, but Python's static type checking ecosystem is too (no stable mypy stubs for numpy ndarrays), as is Julia's AOT compilation story.
- fulafel 7y ago
  Numerical stability sounds like floating point artifacts are a problem, could a specification/verification language with more exact arithmetic semantics help?
  - sharpneli 7y ago
    Its not about floating point artifacts. Especially as full IEEE floats are completely deterministic and have exact semantics. It’s something more fundamental. Similar issues happens elsewhere in numerical computation. See https://en.m.wikipedia.org/wiki/Numerical_stability https://en.m.wikipedia.org/wiki/Numerical_stability
    - DougBTX 7y ago
      I’d phrase it as, “It’s not _just_ about floating point artefacts.” Rounding is a source of error inherent to storing arbitrary precision numbers in finite space, floating point numbers are just a very common way to do that. Numerical stability is basically about information loss, typically when adding a very small number to a very large number causing information about the small number to be rounded out. It happens with ints too, but it is so much more obvious than with floats that no one would be surprised. Repeating i += 0.1 a million times will produce the same value as i += 0.2 a million times if i is an int, since the rounding will make both effectively i += 0.
- deleted 7y ago
  [deleted]
- bigred100 7y ago
  There’s plenty of issues beyond that. For example, many people aren’t gonna use a library that doesn’t have a good AS engine.
- jimbokun 7y ago
  Not a data scientist, but isn’t Julia better positioned to challenge Python for ML workflows than Rust?
  - ddragon 7y ago
    Definitely, since Julia has the same approach as Python of allowing quick and dirty solutions for data analysis/modelling, but with a much larger scope even when you don't have complete library support. In a few hours you can make a functional pytorch clone (and just using special GPU arrays you get it running on GPUs) with similar performance [1], and within a day (given a very good understanding of the language) a method that compiles the gradient directly from unmodified Julia code [2]. Plus native matlab-like goodness such as multi-dimensional arrays, so you don't have a separate library for fast operations and you can just use normal loops or whatever you want. But while Julia targets Python fast and concise (while not compromising speed or power), it does not target the slower but more correct (though there is a culture of testing, which is quite important for math-oriented problems since the type system will not catch the most subtle and troublesome problems). There is space for a language to do exploratory/research work that can be quickly deployed in a fast iterative cycle and another for the new Spark/Flink or critical production areas that needs to take the extra effort (like self-driving cars), which could be Rust (or Scala, or Haskell, or Swift or stay with C++/Fortran). [1] https://github.com/MikeInnes/diff-zoo https://github.com/MikeInnes/diff-zoo [2] http://blog.rogerluo.me/2019/07/27/yassad/ http://blog.rogerluo.me/2019/07/27/yassad/
  - blub 7y ago
    Rust isn't challenging anything for ML workflows, let's be serious.
- jkoudys 7y ago
  Rust people will talk about ownership, because when you work with the language you learn to appreciate what it implies. Rust code for ml can be fairly effortlessly compiled to any target - including wasm. Not so for python running over c-compiled libs. Rust has the potential to NN over a bunch of phones and laptops all collaborating on the same thing together. Huge potential, esp around dapps that need a little ml in their lives.
sgillen 7y ago
>> Is rust suitable for data science workflows? >> Right now I have to say that the answer is “not yet”. I’ll definitely reach for rust in the future when I need to write optimized low-level code with minimal dependencies. However using it as a full replacement for python or C++ will require a more stabilized and well-developed ecosystem of packages. I'm not sure rust is really aiming to be something used for data science workflows. I'm not sure the community will be putting much effort into making this a reality.
- adrianN 7y ago
  There is this https://www.arewelearningyet.com/ https://www.arewelearningyet.com/
- s_Hogg 7y ago
  Fine, but it seems reasonable for someone to check. It's much easier to make a decision about whether a programming language is what you want if you've got evidence from someone who has tried something similar to you, as opposed to just hearing people say "Language X is awesome because of unimaginably low-level (from my point of view) feature Y". Rust seems great, to be honest, just not universally so. Nothing wrong with defining the boundaries.
- drewm1980 7y ago
  There are people working on language features that will get Rust closer to parity with C++ for numerical computing, most prominently "const generics", which will make it more ergonomic to write numeric libraries (see C++'s Eigen) that use static array sizes. This will ultimately be important for how aggressively the compiler can optimize the code, via eliminating bounds checks, etc.
- ImNotTheNSA 7y ago
  > I'm not sure rust is really aiming to be something used for data science workflows I’m not totally convinced rust knows what it’s aiming to be, other than replacing c++...
archgoon 7y ago
This seems to be comparing a hand implemented neural network in python (and numpy) and one in rust. Even in this simple case, the author discovers that in the python case, most of the time is spent in non-python linear algebra libraries. Most of the major deep learning frameworks for python (tensorflow, keras, torch, mxnet, etc) will not normally be spending the majority of their time in python. Typically, the strategy is to use python to declare the overall structure of the net, and where to load data from, and then the actual heavy lifting will be done in optimized libraries written probably in C++ (or fortran, I seem to recall BLAS used fortran).
- sgillen 7y ago
  I think BLAS is a spec rather than one library. So your version may or may not be fortran. I do think the original "reference implementation" was written in fortran, which is sometimes called "The BLAS library" but I think most BLAS you see in the wild are not that.
  - the_svd_doctor 7y ago
    Yes. BLAS was originally specified in Fortran. But many BLAS implementation (like cuBLAS, the CUDA/Nvidia version for GPU's) don't use Fortran at all.
  - archgoon 7y ago
    > I think BLAS is a spec rather than one library. Yep; poor wording on my part. Thanks. :)
- axaxs 7y ago
  This is the strength of python, though, and one you cannot ignore. Python is old, and has fast C ops for everything you may want to accomplish. There's no shame in that method in benchmarks.
longemen3000 7y ago
Nice to see, a Julia implementation should be fun to do, but the point of the article is true:when you work with linear algebra, most of the time is spent in BLAS.
ageofwant 7y ago
This approach I think is missing the point. You will write highly optimized libraries in Rust, and then use those in Python. This is why Python has eaten the world. Not because its the best at any one thing, except bringing all those things together - at which it is unparalleled, and is unlikely to be surpassed anytime soon. numpy, scipy, pandas, tensorflow all those have very little actual Python code, its c++ and even fortran here and there. This whole Python vs Bla thing is just silly nonsense. I know Python and some Bla, and so should you. Tonight someone will release SuperFantasticNewThing implemented in Bla, tomorrow someone else will wrap that in Python, and tomorrow night the rest of us will use PySuperFantasticNewThing, and that's exactly how it should be.
- pjmlp 7y ago
  Tensorflow is now having Swift support and then there is XLA. Thanks to ML.NET, I can enjoy .NET JIT/AOT compilers performance, while binding to the same Tensorflow C++ libraries that are wrapped for Python. Swift and Kotlin enjoy similar bindings on their respective mobile platforms. Then there is Julia. Python might have gotten there first, but its kingdom is already getting slowly eroded.
  - longemen3000 7y ago
    For God's sake, there are some folks starting to implement BLAS in Julia, so I have high hopes for the language
- dmos62 7y ago
  I think you're being unnecessarily dismissive of discussions surrounding python's fitness, and I find your remark "that's exactly how it should be" confusing. Python is an ok interface language, in that it's script-like, dynamically typed and simple to comprehend. It's popular, which makes on-boarding efficient due to the sheer volume of tutorials online. And, it has built up a large ecosystem, because of the last two points. That said, it's naive to suppose that python is the currently-ideal or future-ideal interface language. It's just ok, plus it's popular.
  - 7y ago
psv1 7y ago
Neural network libraries (Tensorflow, Pytorch) have a C++ backend and a Python interface. Which is great - you get a performant compiled language as the backend and a flexible user-friendly language as the interface. Rust vs Python is a weird question because in reality no one writes their own neural network with numpy, and no one expects Rust to act like an interpreted language suitable for data science workflows. It would be more apt to compare Rust and C++.
- ianamartin 7y ago
  Yeah, this is the real point. Python is an interface to C, C++, and FORTRAN for a lot of stuff. There are even crossover libs for running R. This is like comparing apples and steaks.
  - TheRealKing 7y ago
    "Fortran", since 1990 (FORTRAN refers to F77 :-)
- seanmcdirmid 7y ago
  Or maybe Rust can be compared to Julia if Julia is used for custom machine learning rather than just calling into pre compiled CUDA kernels? Also, does Rust have a GPU/CUDA backend yet?
marmaduke 7y ago
Rust seems more suitable for implementing the next OpenBLAS. While Julia's single language mantra is great, as long as things like Python exist, there will be a need for C/C++/Rust.
- asdjlkadsjklads 7y ago
  Yea, this whole discussion feels weird to me. Different use cases. I love Rust (and dislike Py lol), but from everything i hear a highly dynamic frontend (like Py) has little downsides to authors of ML/etc. All of the hotpaths are in other already because Python is so slow. The only downside i've seen is sometimes the programmer will want more safety. In such a scenario Rust for the "frontend" would be very useful. So we have two concerns, frontend and backend. For the backend Rust would perfectly acceptable, but i'm not sure it is fixing a safety issue/etc in other (C/etc) languages - aka, perhaps little value in the backend. For the frontend it only has value in some areas. Regardless, i love Rust and would totally welcome any tooling to keep me in Rust. However i'm not an ML person hah.
  - danielscrubs 7y ago
    Novel feature engineering is becoming a must. Do you reach for Python or C++? Do you want multiple languages if one was enough?
  - MiroF 7y ago
    A dynamic language can be super frustrating to develop with because you have to keep tensor dimensions memorized in your head or in comments
    - marmaduke 7y ago
      How is à statically typed language going to help with tensor dimensions?
      - MiroF 7y ago
        Named/typed axes checking - ie. I shouldn't be able to contract along two dimensions of different types (like batch dimension contracted with time dimension)
roadbeats 7y ago
I’m a newbie in NN topic and feel surprised to hear that noone uses Numpy in actual NN implementations, although it’s written in C++ and highly optimized. Why is that ? And, how about Gonum (Go equivalent) ? Finally, I’m currently going through the deeplearning.ai program. I got one week left, and will experiment with building some apps. Which technical stack should I choose ?
- jimmy_dean 7y ago
  The main reason numpy isn't used in NN implementations is that it does not, natively speaking, have GPU support. Tensor structures in PyTorch and TensorFlow have the most solid backend support for GPUs (TPUs) and have a good amount of numpy's ndarray capabilities. There is recent work to put numpy on the same footing for deep learning. Check https://github.com/google/jax https://github.com/google/jax
- csande17 7y ago
  Most software (including Python and Numpy and Go and pretty much every Rust program) runs on your computer's CPU. The CPU is good at running programs with a lot of different instructions and if-statements and loops and stuff. But for neural networks, people often prefer to use special hardware like graphics cards, since graphics cards are really good at doing relatively simple math on many pieces of data at once. So they create special libraries like TensorFlow that can send commands to the graphics card instead of doing the math on the CPU. (And they don't use Numpy because even though it's highly optimized, it's highly optimized for CPUs, and graphics cards are a lot faster than CPUs at running neural networks.)
- PeterisP 7y ago
  Numpy is at a too low level for applied NN implementations. If I'm not doing research on new methods but want to build a model for a particular problem using well-known best practices, then all the custom code that my app needs and what I need to write is about the transformation and representation and structure of my particular dataset and task; but things like, for example, optimized backpropagation for a stack of bidirectional LSTM layers are not custom for my app, they're generic - why would I need or want to reimplement them except as a learning exercise? That'd be like reinventing a bicycle, for generic things like that I'd want to call a library where that code is well-tested and well-optimized (including for GPU usage) by someone else, and that library isn't numpy. Numpy works at the granularity of matrix multiplication ops, but applied ML works at the granularity of whole layers such as self-attention or LSTM or CNN; which perhaps are not that complex conceptually, but do require some attention to implement properly in an optimized way; you can implement them in numpy but you probably shouldn't (unless as a learning exercise).
bayesian_horse 7y ago
It's going to be tough to beat Numpy, Cython and Numba on speed with Rust. And this kind of workload really isn't the problem Rust wants to solve.
pilooch 7y ago
vs C++-14 ? Indeed most DL is in fact C++. The Pytorch recent C++ API is a must. As professionals in this industry, my colleagues and I have switched to full C++. I'd be interested in advantages of Rust vs C++ instead of Python (which truely in terms of performances is C in the background).
timClicks 7y ago
I think it's quite impressive actually that someone can pick up Rust and manage to out-perform Numpy in their first project. BLAS implementations are decades-long exercises in optimization. In my own experience, Rust has been excellent for the more boring side of data science - churning through TBs of input data.
- archgoon 7y ago
  > I think it's quite impressive actually that someone can pick up Rust and manage to out-perform Numpy in their first project. They didn't. At the end of the article they discuss this. "In fact it’s worse than that. One of the exercises in the book is to rewrite the Python code to use vectorized matrix multiplication. In this approach the backpropagation for all of the samples in each mini-batch happens in a single set of vectorized matrix multiplication operations. This requires the ability to matrix multiplication between 3D and 2D arrays. Since each matrix multiplication operation happens using a larger amount of data than the non-vectorized case, OpenBLAS is able to more efficiently occupy CPU caches and registers, ultimately better using the available CPU resources on my laptop. The rewritten Python version ends up faster than the Rust version, again by a factor of two or so." The original python code was written to be easily understood; not performant. Shifting more of the work to the libraries improved the performance.
  - mplanchard 7y ago
    That is a different algorithm, though, right? They didn’t rewrite the rust code to use matrix multiplication, so it’s no longer a direct comparison, and is instead comparing unoptimized rust with a less efficient algorithm to super optimized C++ with a more efficient algorithm (being called from python). In which case, IMO, it’s fairly surprising that the rust implementation is only 2x slower.
- MiroF 7y ago
  This is a common phenomenon (as anyone who has tried to rewrite the standard library buffer cache can tell you) - the reason is that these algorithms are oftentimes optimized to perform decently on the very very worst cases which means that they'll be slightly slower overall
xiaodai 7y ago
"The first step here was to figure out how to load the data. That ended up being fiddly enough that I decided to break that off into its own post." this is exactly why we have R and pandas!! Julia is doing an increasingly excellent job of it as well.
acollins1331 7y ago
They already have a language to blow python out of the water in ML and it's called Julia.
tammysmith 7y ago
I have been seeing some strange calls from my husband phone and yet am unable to locate his phone or even get the number that have been calling him. i was so lucky enough that i went online to hire a hacker who was gonna help me in tracking my husband phone and also giving me access to his phone. while i was searching i saw jamesscotthacker@gmail.com as the best and i hired him this hacker was so good that he delivered me with my husband call logs,whatsapp,facebook, pictures, deleted call logs, all calls where recorded from last year till date i also had access to his text messages and also deleted messages. all this i had access right on my phone just with a very cheap price you can also text him :1(323)4214332
lucasbonatto 7y ago
tldr "Right now I have to say that the answer is “not yet”. I’ll definitely reach for rust in the future when I need to write optimized low-level code with minimal dependencies. However using it as a full replacement for python or C++ will require a more stabilized and well-developed ecosystem of packages."
daenz 7y ago
tldr: The bottleneck is in the linear algebra libraries. Unoptimized Rust is 2x faster than unoptimized Python. Optimized Python is 2x faster than unoptimized Rust because it can vectorize matrix multiplications, but Rust can't. My takeaway: stick to a GPU where everything is more parallel.
mrtweetyhack 7y ago
I appreciate the nice write-up
qaq 7y ago
Given work Chris Lattner is doing with Swift it has better chance of becoming a contender in this space vs Rust.
HelloNurse 7y ago
In the end, hypothetical future good Rust libraries for linear algebra will just be turned into Python extensions and embraced into the mainstream. They'll also, equally well, serve the needs of Rust programs which need serious number crunching (presumably a small niche); there is no "versus" in the comparison.
ahurmazda 7y ago
I could see how Rust could be a good option for writing the prediction server. In my use case, DL models are trained offline. It does not matter if it’s faster by 30 mins in training (maybe when I R&D-ing). On the other hand, you typically have to reach for something like c++ for a low latency, high thruput environment. It can be a bear to write a server. Anything more ergonomic would be welcome
ram_rar 7y ago
I dont think, rust is competing with python in a similar space. The biggest win for Rust would be to have someone in the community rewrite Cython in Rust and get rid of GIL. Its a win-win for everyone.
adsharma 7y ago
https://medium.com/@konchunas/transpiling-python-to-rust-766459b6ab8f https://medium.com/@konchunas/transpiling-python-to-rust-766... Its possible to transpile. But the tech is not mature yet
sdinsn 7y ago
Rust isn't going to be used until it has better CUDA support.

290 points by rencire 7y ago

Nobody writing NN in Python, they are just describing it. For NN or DL in general, the correctness doesn't really lie too much on the code quality level, like ownership Rust people love to talk about. It is more about Numeric stability under/overflow and such. Choice of programming language offers limited help here. I don't think Rust has a killer app for ML/DL community to offer as of now, the focus is vastly different.

kuzehanka 7y ago
I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a component that's idle 99% of the time. The bulk of the load in an DL workflow is CUDA code and sits on the GPU. Even the intermediate libraries like cublas would only see marginal-to-none benefits of being reimplemented in rust. This is a cool project, but it has no chance to displace or even complement Python in the data science space.
- wokwokwok 7y ago
  > comparatively tedious languages like Rust will never attract data science practitioners. Well, fast.ai is using swift now. ... I think it's fair to say 'never say never'. You're probably right, rust isn't really the sweet spot for this stuff, but its also a case that python has some down sides that are pretty severe, and well acknowledged.
  - melling 7y ago
    I wouldn’t call Swift a tedious language. With type inference, immutability, etc, Swift is far from tedious: http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=strings http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=stri... http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=dictionaries http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=dict... It’s not quite as nice as Python but it’s an enjoyable language.
    - frutiger 7y ago
      Rust has those too, GGGP still considered it tedious.
      - antisemiotic 7y ago
        I think the tedious part would be the borrow checker. Also, Rust doesn't have top-level type inference (by design).
        skohan 7y ago
        Yeah there is some tedium in general with Rust syntax. Semicolons, for example, feel old fashioned, and there's a lot of verbosity in things like unwrapping. It's a fine language and I like working with it, but there's a lot of details involved in Rust development which don't make sense for data scientists to worry about.
    - wokwokwok 7y ago
      > They don't care about the kind of safety it brings, they don't care about improving performance in a component that's idle 99% of the time. https://www.fast.ai/2019/03/06/fastai-swift/ https://www.fast.ai/2019/03/06/fastai-swift/: > Because Swift for TensorFlow is the first serious effort I’ve seen to incorporate differentiable programming deep in to the heart of a widely used language that is designed from the ground up for performance. > But Python is not designed to be fast, and it is not designed to be safe. Instead, it is designed to be easy, and flexible. To work around the performance problems of using “pure Python” code, we instead have to use libraries written in other languages (generally C and C++), like numpy, PyTorch, and TensorFlow, which provide Python wrappers. To work around the problem of a lack of type safety, recent versions of Python have added type annotations that optionally allow the programmer to specify the types used in a program. However, Python’s type system is not capable of expressing many types and type relationships, does not do any automated typing, and can not reliably check all types at compile time. Therefore, using types in Python requires a lot of extra code, but falls far short of the level of type safety that other languages can provide. ie. My point: OP's point is categorically false. ...not that swift is tedious. I <3 swift.
  - chillee 7y ago
    Just because fast.ai has some investment in Swift does not mean that S4TF has attracted mind share. The bulk of fast.ai is still taught on Pytorch.
    - wokwokwok 7y ago
      The parent comment literally said data science practitioners don't care about speed or safety because the GPU is where all the real work happens; that's false, I've provided an example of it being false from a respected party. What do you want me to say? eh, I give up. Believe whatever you want to believe.
      - chillee 7y ago
        I'm not saying that no data science practitioners care about speed and safety, and its easy for me to come up with cases where they should/do care. My point is that even fast.ai still views S4TF as somewhat niche, and that data science practitioners as a whole still don't care.
        MiroF 7y ago
        I am a practitioner, I care, but everyone on this thread seems to be distracted by the red herring of speed and efficiency. None of these languages are going to be speedier in the GPU. But python has serious design problems that manifest itself when developing a large project. You can only go so far with dynamic typing
- xxxpupugo 7y ago
  CUDA is an important part of the story. I think the industry is moving to 'MLIR' solution (Yes, there is a Google project called exactly that, but I am referring to the general idea here), where the network is defined and trained in one place, then the weights are exported, delegated to optimized runtime to be executed. If such trend furthers down, then there will be very little reason to replace Python as the glue layer here. Instead it will become that everything is training in Python -> exported to shared format -> executed in optimized runtime, kind of flow. Rust's opportunity could be to replace C++ in this case. But do mind that this is also a competitive business, where the computation is pushed further down to the hardware implementations, like TPUv1 and T4 chips etc.
  - skohan 7y ago
    Personally I hope for alternatives because I don't find Python that nice to work with compared to other languages. It's not awful, but I really miss having type and nullability errors detected by the compiler and reported to me with meaningful error messages. Also there are a few weird things with the Python workflow, like dealing with Python2 vs Python3, and generally having to run everything in the context of a virtual environment, and the fact that it doesn't seem to really have a single "right way" to deal with dependency management. It's a bit like the story with imports in Javascript: those problems were never really solved, and the popular solutions are just sort of bolted on to the language. It's amazing how many great tools there are available in Python, but it does sometimes seem like it's an under-powered tool which has been hacked to serve bigger problems than it was intended for.
    - ImNotTheNSA 7y ago
      > like dealing with Python2 vs Python3, and generally having to run everything in the context of a virtual environment, and the fact that it doesn't seem to really have a single "right way" to deal with dependency management. Not to mention, this problem seems to be getting worse, not better. People are moving off of python 2.7, which was kind of the de-facto LTS release of python... leaving (currently) no LTS version of python and no clear path for the community to establish a new LTS version that the community will still support — there are so many releases with so many breaking changes in python 3 within the last few years that there is seemingly no consensus and no way to compromise. > It's amazing how many great tools there are available in Python, but it does sometimes seem like it's an under-powered tool which has been hacked to serve bigger problems than it was intended for. This is becoming more and more clear with every release of python IMO. The language is evolving too quickly for it to be used on large developments, but it’s still being used for that. We have an entire test framework and libraries for high performance embedded systems level testing which is written entirely in python. The amount of high speed operations (timing tests, measurements, etc) is very obviously overwhelming for the intention of the language and libraries, yet the company keeps pushing ahead with this stuff. In order to mitigate the issue, we are developing more high speed embedded systems to offload the work from the test framework and report measurements to the framework. I think it’s quickly becoming extremely expensive and will only become more expensive — the framework is extremely “pythonic” to the point of being unreadable, using a number of hacks to break through the python interpreter. Jobs much better allocated to readable C++ libraries are being implemented in unreadable python spaghetti code with no clear architecture - just quick-and-dirty whatever-it-takes python. I love python but I think it’s a quick-and-dirty language for a reason. What python does well cannot be beat by other languages (for example, prototyping), but I think it is often misused, since people can get something up and running quickly and cleanly in python, but it eventually has diminishing (and even negative) returns.
- pjmlp 7y ago
  Rust might not be it. But AOT/JIT compiled languages that can naturally talk to the GPGPU, without 2nd language syndrome, like Julia, Swift, Java and .NET will certainly be more attractive to data science practitioners. I can already envision those life science guys that migrate to VB.NET when they have outgrown their Excel/VBA code, to start playing with ML.NET.
  - kuzehanka 7y ago
    Python already gets JIT compiled to CUDA[1] and there's an entire funded ecosystem built around python+gpgpu called RAPIDS[2] which is the future of the ML space by most indicators. I don't see any other language even making a dent in the Python ecosystem without some kind of new killer feature that can't be quickly replicated in Python. [1] https://numba.pydata.org https://numba.pydata.org [2] https://rapids.ai https://rapids.ai
    - disgruntledphd2 7y ago
      Thank you for the cite to rapids.ai, that looks extremely interesting :)
    - MiroF 7y ago
      Swift and static type checking and compiler analysis might be that language and feature combo.
    - hsaliak 7y ago
      You beat me to making this point. I don't see anything in the article's python code that the numba's jit decorator cannot handle. When numba works (it's rapidly improving), it's seriously impressive. For this particular case, you should be able to get really good performance without sacrificing readability. Also Jax - https://github.com/google/jax https://github.com/google/jax
  - MiroF 7y ago
    Your last paragraph is my nightmare
- skohan 7y ago
  This is why I think Swift is a much better choice to replace or at least compliment Python than Rust. It has a modern, powerful type system and all the quality of life advantages which come with it, but it manages this with a lot more usability than Rust. A well written swift framework almost becomes a DSL for the problem domain, which is a great property for a data science tool to have.
  - echelon 7y ago
    Swift is not fun on non-Mac platforms. It feels a lot like Google's Dart in terms of how it was positioned and advocated. Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors. The compiler is incredibly helpful, non-lexical lifetimes are a thing, and unless you're doing a lot of sharing and parallelism, you can avoid many borrow checker problems until you learn RAII.
    - skohan 7y ago
      It's a fair criticism of Swift that it is not well supported across platforms. The story is getting better, but it is still miles behind Rust in this regard. I am a huge fan of Swift as a language, but I am very critical of the tooling. > Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors. I beg to differ. I've been programming professionally for over a decade, and I have shipped projects in a variety of languages, and I can safely say that Rust has a steeper learning curve and requires more cognitive overhead to use than many other languages. I find it relatively nice to work with rust in spite of this because the tooling is so great, but it's undeniable that Rust has made tradeoffs which sacrifice ease of use in favor of safety and performance.
      - MiroF 7y ago
        The tooling for Swift 4 TF is not anywhere near satisfactory. I can't even seem to get it on my computer without installing XCode (and that's on an apple computer) I should be able to pull a docker image and have s4tf immediately at my fingertips
- danielscrubs 7y ago
  I am a data scientist and I care. The time when you could just do proof of concepts or a PowerPoint presentation is long behind us. So now we have to start to take it into production, which means we get the exact same problems as SE has always had. Iff Rust helps us take it into production we will use it. But it’s a lot of land to cover to reach Pythons libraries so I’m not holding my breath. That said, Pythons performance is slow even when shuffling to Numpy.
  - kuzehanka 7y ago
    I must be missing something. Modern data science workloads involve fanning out data and code across dozens to hundreds of nodes. The bottlenecks, in order, are: inter-node comms, gpu/compute, on-disk shuffling, serialisation, pipeline starvation, and finally the runtime. Why worry about optimising the very top of the perf pyramid which will make the least difference? Why worry if you spent 1ms pushing data to numpy when that data just spent 2500ms on the wire? And why are you even pushing from python runtime to numpy instead of using arrow?
    - corndoge 7y ago
      Good lord, hopefully latency isn't 2.5 seconds!
      - sjwright 7y ago
        I can’t even. How could you ever get 2500 msec on transit? That’s like circling the globe ten times.
        justinclift 7y ago
        Maybe a bunch of SSL cert exchanges through some very low bandwidth connections? ;) Still, it's more likely a figure used for exaggeration, for effect.
        deleted 7y ago
        [deleted]
- Nullabillity 7y ago
  I'm by no means a specialized data scientist, but I've done some (very surface-level) text crunching with both the TF/Numpy stack and Rust. To me, the nice thing about switching to Rust for that kind of stuff was that it dramatically raised the bar of what I could do before reaching for those hyper-optimized descriptive libraries. Want to calculate the levenshtein distances of the cross product of 100k strings? Sure, just load it into a Vec, find a levenshtein library on crates.io, and it'll probably be fast enough. Could it be done in Python? Sure, but with Rust I didn't have to think about how to do it in a viable amount of time. Does that mean that Rust is going to take over the DS world? Probably not in the short term, Rust currently can't compete with Python's DS ecosystem. But if I'm doing something that I don't know will fit into an existing Python mold (that I know about) then I'll strongly consider using it.
  - smallnamespace 7y ago
    > But if I'm doing something that I don't know will fit into an existing Python mold (that I know about) then I'll strongly consider using it. The thing is, for anything performance intensive and scientific, you're almost guaranteed to find a Python binding. It has all these bindings because scientists are almost always writing either Python, C++, or Fortran (with a smattering of R or Octave on the side). Want to do linear algebra? NumPy is always there, otherwise you can go lower level: https://docs.scipy.org/doc/scipy/reference/linalg.blas.html https://docs.scipy.org/doc/scipy/reference/linalg.blas.html Old-school optimization? https://www.roguewave.com/sites/rw/files/attachments/PyIMSLStudioFeatures.pdf https://www.roguewave.com/sites/rw/files/attachments/PyIMSLS... Computer vision? https://pypi.org/project/opencv-python/ https://pypi.org/project/opencv-python/ In fact, I'd basically argue against using most language-native implementations of algorithms where performance is at stake, because most implementations don't have all the algorithmic optimizations.
    - danielscrubs 7y ago
      And they will cover a small part of what is found in research papers. Python has the best ecosystem, but Rust was made by a competent team so we will root for it.
- echelon 7y ago
  I think this project has immediate merits when it comes to productionizing small, sparse networks for running on the CPU. I do research and prototyping in Python, but I have to deploy on mobile devices. I was going to roll my own implementation, but now that this exists, it's something I'm going to look into.
  - MiroF 7y ago
    Load the model from torch script? In C++? Don't see why Rust made this possible.
    - echelon 7y ago
      Ideally we'd have native Rust loading of tensorflow, pytorch, etc. models. I've hand-coded a few, but it was pretty monotonous.
atoav 7y ago
As somebody who programs in both Python and Rust (and likes both languages) I think Rust's place would be parts of the code that have to be fast, and that you want to get right. Calling Python code from Rust or Rust from Python is totally doable, and there is in my view no reason why you shouldn't use both in the use cases that suit them. And the speed part is serious. Some guy once asked for the fastest tokenizer in any given language and my naive implementation came second place in his benchmark right after an stripped down and optimized C variant. So using Rust for speed critical modules and interfacing them from easy to use Python libraries isn't exactly irrational.
- gridlockd 7y ago
  Calling C (or C++) code from Python is not only totally doable, it's what all the popular libraries do. Furthermore, most ML stuff runs on the GPU, which also is C-like code. Hence, Rust offers no performance benefit that isn't already there. It really only offers safety and modern language features, at the cost of being tedious to use.
  - atoav 7y ago
    A bit snarky isn’t it? I explained why it isn’t irrational to use Rust — nowhere did I say you have to use Rust or that Rust is the only solution to that problem (which it obviously isn’t). Assuming the person opposite doesn’t know things is an extra unpleasant move on your part and I am not sure why you feel attacked by any of what I wrote. For the record, I also use C and I know that expert level C can beat naive Rust at any time (expert level Rust should allow you to do literally the same thing you do in C, just wrapped in a “I can handle this”-unsafe block. Your response tells more about yourself than it does about the subject at hand. Rust isn’t stealing your cookies and nor am I.
    - ImNotTheNSA 7y ago
      Sounds like you’re over reading into what was essentially a two-point comment... there was no snarkiness in that comment at all - just a statement that rust is not really offering much benefit for this circumstance. Yeesh... projection
drewm1980 7y ago
You're right, and... I'm working professionally using DL for computer vision, for robotics. We spend the majority of our time writing the business logic around the learned parts. To get all of that code bug free and fast is way harder in python that I expect it would be in Rust. Rust's ndarray ecosystem is immature, but Python's static type checking ecosystem is too (no stable mypy stubs for numpy ndarrays), as is Julia's AOT compilation story.
fulafel 7y ago
Numerical stability sounds like floating point artifacts are a problem, could a specification/verification language with more exact arithmetic semantics help?
- sharpneli 7y ago
  Its not about floating point artifacts. Especially as full IEEE floats are completely deterministic and have exact semantics. It’s something more fundamental. Similar issues happens elsewhere in numerical computation. See https://en.m.wikipedia.org/wiki/Numerical_stability https://en.m.wikipedia.org/wiki/Numerical_stability
  - DougBTX 7y ago
    I’d phrase it as, “It’s not _just_ about floating point artefacts.” Rounding is a source of error inherent to storing arbitrary precision numbers in finite space, floating point numbers are just a very common way to do that. Numerical stability is basically about information loss, typically when adding a very small number to a very large number causing information about the small number to be rounded out. It happens with ints too, but it is so much more obvious than with floats that no one would be surprised. Repeating i += 0.1 a million times will produce the same value as i += 0.2 a million times if i is an int, since the rounding will make both effectively i += 0.
deleted 7y ago
[deleted]
bigred100 7y ago
There’s plenty of issues beyond that. For example, many people aren’t gonna use a library that doesn’t have a good AS engine.
jimbokun 7y ago
Not a data scientist, but isn’t Julia better positioned to challenge Python for ML workflows than Rust?
- ddragon 7y ago
  Definitely, since Julia has the same approach as Python of allowing quick and dirty solutions for data analysis/modelling, but with a much larger scope even when you don't have complete library support. In a few hours you can make a functional pytorch clone (and just using special GPU arrays you get it running on GPUs) with similar performance [1], and within a day (given a very good understanding of the language) a method that compiles the gradient directly from unmodified Julia code [2]. Plus native matlab-like goodness such as multi-dimensional arrays, so you don't have a separate library for fast operations and you can just use normal loops or whatever you want. But while Julia targets Python fast and concise (while not compromising speed or power), it does not target the slower but more correct (though there is a culture of testing, which is quite important for math-oriented problems since the type system will not catch the most subtle and troublesome problems). There is space for a language to do exploratory/research work that can be quickly deployed in a fast iterative cycle and another for the new Spark/Flink or critical production areas that needs to take the extra effort (like self-driving cars), which could be Rust (or Scala, or Haskell, or Swift or stay with C++/Fortran). [1] https://github.com/MikeInnes/diff-zoo https://github.com/MikeInnes/diff-zoo [2] http://blog.rogerluo.me/2019/07/27/yassad/ http://blog.rogerluo.me/2019/07/27/yassad/
- blub 7y ago
  Rust isn't challenging anything for ML workflows, let's be serious.
jkoudys 7y ago
Rust people will talk about ownership, because when you work with the language you learn to appreciate what it implies. Rust code for ml can be fairly effortlessly compiled to any target - including wasm. Not so for python running over c-compiled libs. Rust has the potential to NN over a bunch of phones and laptops all collaborating on the same thing together. Huge potential, esp around dapps that need a little ml in their lives.

sgillen 7y ago

>> Is rust suitable for data science workflows? >> Right now I have to say that the answer is “not yet”. I’ll definitely reach for rust in the future when I need to write optimized low-level code with minimal dependencies. However using it as a full replacement for python or C++ will require a more stabilized and well-developed ecosystem of packages. I'm not sure rust is really aiming to be something used for data science workflows. I'm not sure the community will be putting much effort into making this a reality.

adrianN 7y ago
There is this https://www.arewelearningyet.com/ https://www.arewelearningyet.com/
s_Hogg 7y ago
Fine, but it seems reasonable for someone to check. It's much easier to make a decision about whether a programming language is what you want if you've got evidence from someone who has tried something similar to you, as opposed to just hearing people say "Language X is awesome because of unimaginably low-level (from my point of view) feature Y". Rust seems great, to be honest, just not universally so. Nothing wrong with defining the boundaries.
drewm1980 7y ago
There are people working on language features that will get Rust closer to parity with C++ for numerical computing, most prominently "const generics", which will make it more ergonomic to write numeric libraries (see C++'s Eigen) that use static array sizes. This will ultimately be important for how aggressively the compiler can optimize the code, via eliminating bounds checks, etc.
ImNotTheNSA 7y ago
> I'm not sure rust is really aiming to be something used for data science workflows I’m not totally convinced rust knows what it’s aiming to be, other than replacing c++...

archgoon 7y ago

This seems to be comparing a hand implemented neural network in python (and numpy) and one in rust. Even in this simple case, the author discovers that in the python case, most of the time is spent in non-python linear algebra libraries. Most of the major deep learning frameworks for python (tensorflow, keras, torch, mxnet, etc) will not normally be spending the majority of their time in python. Typically, the strategy is to use python to declare the overall structure of the net, and where to load data from, and then the actual heavy lifting will be done in optimized libraries written probably in C++ (or fortran, I seem to recall BLAS used fortran).

sgillen 7y ago
I think BLAS is a spec rather than one library. So your version may or may not be fortran. I do think the original "reference implementation" was written in fortran, which is sometimes called "The BLAS library" but I think most BLAS you see in the wild are not that.
- the_svd_doctor 7y ago
  Yes. BLAS was originally specified in Fortran. But many BLAS implementation (like cuBLAS, the CUDA/Nvidia version for GPU's) don't use Fortran at all.
- archgoon 7y ago
  > I think BLAS is a spec rather than one library. Yep; poor wording on my part. Thanks. :)
axaxs 7y ago
This is the strength of python, though, and one you cannot ignore. Python is old, and has fast C ops for everything you may want to accomplish. There's no shame in that method in benchmarks.

longemen3000 7y ago

Nice to see, a Julia implementation should be fun to do, but the point of the article is true:when you work with linear algebra, most of the time is spent in BLAS.

ageofwant 7y ago

This approach I think is missing the point. You will write highly optimized libraries in Rust, and then use those in Python. This is why Python has eaten the world. Not because its the best at any one thing, except bringing all those things together - at which it is unparalleled, and is unlikely to be surpassed anytime soon. numpy, scipy, pandas, tensorflow all those have very little actual Python code, its c++ and even fortran here and there. This whole Python vs Bla thing is just silly nonsense. I know Python and some Bla, and so should you. Tonight someone will release SuperFantasticNewThing implemented in Bla, tomorrow someone else will wrap that in Python, and tomorrow night the rest of us will use PySuperFantasticNewThing, and that's exactly how it should be.

pjmlp 7y ago
Tensorflow is now having Swift support and then there is XLA. Thanks to ML.NET, I can enjoy .NET JIT/AOT compilers performance, while binding to the same Tensorflow C++ libraries that are wrapped for Python. Swift and Kotlin enjoy similar bindings on their respective mobile platforms. Then there is Julia. Python might have gotten there first, but its kingdom is already getting slowly eroded.
- longemen3000 7y ago
  For God's sake, there are some folks starting to implement BLAS in Julia, so I have high hopes for the language
dmos62 7y ago
I think you're being unnecessarily dismissive of discussions surrounding python's fitness, and I find your remark "that's exactly how it should be" confusing. Python is an ok interface language, in that it's script-like, dynamically typed and simple to comprehend. It's popular, which makes on-boarding efficient due to the sheer volume of tutorials online. And, it has built up a large ecosystem, because of the last two points. That said, it's naive to suppose that python is the currently-ideal or future-ideal interface language. It's just ok, plus it's popular.
- 7y ago

psv1 7y ago

Neural network libraries (Tensorflow, Pytorch) have a C++ backend and a Python interface. Which is great - you get a performant compiled language as the backend and a flexible user-friendly language as the interface. Rust vs Python is a weird question because in reality no one writes their own neural network with numpy, and no one expects Rust to act like an interpreted language suitable for data science workflows. It would be more apt to compare Rust and C++.

ianamartin 7y ago
Yeah, this is the real point. Python is an interface to C, C++, and FORTRAN for a lot of stuff. There are even crossover libs for running R. This is like comparing apples and steaks.
- TheRealKing 7y ago
  "Fortran", since 1990 (FORTRAN refers to F77 :-)
seanmcdirmid 7y ago
Or maybe Rust can be compared to Julia if Julia is used for custom machine learning rather than just calling into pre compiled CUDA kernels? Also, does Rust have a GPU/CUDA backend yet?

marmaduke 7y ago

Rust seems more suitable for implementing the next OpenBLAS. While Julia's single language mantra is great, as long as things like Python exist, there will be a need for C/C++/Rust.

asdjlkadsjklads 7y ago
Yea, this whole discussion feels weird to me. Different use cases. I love Rust (and dislike Py lol), but from everything i hear a highly dynamic frontend (like Py) has little downsides to authors of ML/etc. All of the hotpaths are in other already because Python is so slow. The only downside i've seen is sometimes the programmer will want more safety. In such a scenario Rust for the "frontend" would be very useful. So we have two concerns, frontend and backend. For the backend Rust would perfectly acceptable, but i'm not sure it is fixing a safety issue/etc in other (C/etc) languages - aka, perhaps little value in the backend. For the frontend it only has value in some areas. Regardless, i love Rust and would totally welcome any tooling to keep me in Rust. However i'm not an ML person hah.
- danielscrubs 7y ago
  Novel feature engineering is becoming a must. Do you reach for Python or C++? Do you want multiple languages if one was enough?
- MiroF 7y ago
  A dynamic language can be super frustrating to develop with because you have to keep tensor dimensions memorized in your head or in comments
  - marmaduke 7y ago
    How is à statically typed language going to help with tensor dimensions?
    - MiroF 7y ago
      Named/typed axes checking - ie. I shouldn't be able to contract along two dimensions of different types (like batch dimension contracted with time dimension)

roadbeats 7y ago

I’m a newbie in NN topic and feel surprised to hear that noone uses Numpy in actual NN implementations, although it’s written in C++ and highly optimized. Why is that ? And, how about Gonum (Go equivalent) ? Finally, I’m currently going through the deeplearning.ai program. I got one week left, and will experiment with building some apps. Which technical stack should I choose ?

jimmy_dean 7y ago
The main reason numpy isn't used in NN implementations is that it does not, natively speaking, have GPU support. Tensor structures in PyTorch and TensorFlow have the most solid backend support for GPUs (TPUs) and have a good amount of numpy's ndarray capabilities. There is recent work to put numpy on the same footing for deep learning. Check https://github.com/google/jax https://github.com/google/jax
csande17 7y ago
Most software (including Python and Numpy and Go and pretty much every Rust program) runs on your computer's CPU. The CPU is good at running programs with a lot of different instructions and if-statements and loops and stuff. But for neural networks, people often prefer to use special hardware like graphics cards, since graphics cards are really good at doing relatively simple math on many pieces of data at once. So they create special libraries like TensorFlow that can send commands to the graphics card instead of doing the math on the CPU. (And they don't use Numpy because even though it's highly optimized, it's highly optimized for CPUs, and graphics cards are a lot faster than CPUs at running neural networks.)
PeterisP 7y ago
Numpy is at a too low level for applied NN implementations. If I'm not doing research on new methods but want to build a model for a particular problem using well-known best practices, then all the custom code that my app needs and what I need to write is about the transformation and representation and structure of my particular dataset and task; but things like, for example, optimized backpropagation for a stack of bidirectional LSTM layers are not custom for my app, they're generic - why would I need or want to reimplement them except as a learning exercise? That'd be like reinventing a bicycle, for generic things like that I'd want to call a library where that code is well-tested and well-optimized (including for GPU usage) by someone else, and that library isn't numpy. Numpy works at the granularity of matrix multiplication ops, but applied ML works at the granularity of whole layers such as self-attention or LSTM or CNN; which perhaps are not that complex conceptually, but do require some attention to implement properly in an optimized way; you can implement them in numpy but you probably shouldn't (unless as a learning exercise).

bayesian_horse 7y ago

It's going to be tough to beat Numpy, Cython and Numba on speed with Rust. And this kind of workload really isn't the problem Rust wants to solve.

pilooch 7y ago

vs C++-14 ? Indeed most DL is in fact C++. The Pytorch recent C++ API is a must. As professionals in this industry, my colleagues and I have switched to full C++. I'd be interested in advantages of Rust vs C++ instead of Python (which truely in terms of performances is C in the background).

timClicks 7y ago

I think it's quite impressive actually that someone can pick up Rust and manage to out-perform Numpy in their first project. BLAS implementations are decades-long exercises in optimization. In my own experience, Rust has been excellent for the more boring side of data science - churning through TBs of input data.

archgoon 7y ago
> I think it's quite impressive actually that someone can pick up Rust and manage to out-perform Numpy in their first project. They didn't. At the end of the article they discuss this. "In fact it’s worse than that. One of the exercises in the book is to rewrite the Python code to use vectorized matrix multiplication. In this approach the backpropagation for all of the samples in each mini-batch happens in a single set of vectorized matrix multiplication operations. This requires the ability to matrix multiplication between 3D and 2D arrays. Since each matrix multiplication operation happens using a larger amount of data than the non-vectorized case, OpenBLAS is able to more efficiently occupy CPU caches and registers, ultimately better using the available CPU resources on my laptop. The rewritten Python version ends up faster than the Rust version, again by a factor of two or so." The original python code was written to be easily understood; not performant. Shifting more of the work to the libraries improved the performance.
- mplanchard 7y ago
  That is a different algorithm, though, right? They didn’t rewrite the rust code to use matrix multiplication, so it’s no longer a direct comparison, and is instead comparing unoptimized rust with a less efficient algorithm to super optimized C++ with a more efficient algorithm (being called from python). In which case, IMO, it’s fairly surprising that the rust implementation is only 2x slower.
MiroF 7y ago
This is a common phenomenon (as anyone who has tried to rewrite the standard library buffer cache can tell you) - the reason is that these algorithms are oftentimes optimized to perform decently on the very very worst cases which means that they'll be slightly slower overall

xiaodai 7y ago

"The first step here was to figure out how to load the data. That ended up being fiddly enough that I decided to break that off into its own post." this is exactly why we have R and pandas!! Julia is doing an increasingly excellent job of it as well.

acollins1331 7y ago

They already have a language to blow python out of the water in ML and it's called Julia.

tammysmith 7y ago

I have been seeing some strange calls from my husband phone and yet am unable to locate his phone or even get the number that have been calling him. i was so lucky enough that i went online to hire a hacker who was gonna help me in tracking my husband phone and also giving me access to his phone. while i was searching i saw jamesscotthacker@gmail.com as the best and i hired him this hacker was so good that he delivered me with my husband call logs,whatsapp,facebook, pictures, deleted call logs, all calls where recorded from last year till date i also had access to his text messages and also deleted messages. all this i had access right on my phone just with a very cheap price you can also text him :1(323)4214332

lucasbonatto 7y ago

tldr "Right now I have to say that the answer is “not yet”. I’ll definitely reach for rust in the future when I need to write optimized low-level code with minimal dependencies. However using it as a full replacement for python or C++ will require a more stabilized and well-developed ecosystem of packages."

daenz 7y ago

tldr: The bottleneck is in the linear algebra libraries. Unoptimized Rust is 2x faster than unoptimized Python. Optimized Python is 2x faster than unoptimized Rust because it can vectorize matrix multiplications, but Rust can't. My takeaway: stick to a GPU where everything is more parallel.

mrtweetyhack 7y ago

I appreciate the nice write-up

qaq 7y ago

Given work Chris Lattner is doing with Swift it has better chance of becoming a contender in this space vs Rust.

HelloNurse 7y ago

In the end, hypothetical future good Rust libraries for linear algebra will just be turned into Python extensions and embraced into the mainstream. They'll also, equally well, serve the needs of Rust programs which need serious number crunching (presumably a small niche); there is no "versus" in the comparison.

ahurmazda 7y ago

I could see how Rust could be a good option for writing the prediction server. In my use case, DL models are trained offline. It does not matter if it’s faster by 30 mins in training (maybe when I R&D-ing). On the other hand, you typically have to reach for something like c++ for a low latency, high thruput environment. It can be a bear to write a server. Anything more ergonomic would be welcome

ram_rar 7y ago

I dont think, rust is competing with python in a similar space. The biggest win for Rust would be to have someone in the community rewrite Cython in Rust and get rid of GIL. Its a win-win for everyone.

adsharma 7y ago

https://medium.com/@konchunas/transpiling-python-to-rust-766459b6ab8f https://medium.com/@konchunas/transpiling-python-to-rust-766... Its possible to transpile. But the tech is not mature yet

sdinsn 7y ago

Rust isn't going to be used until it has better CUDA support.