6 ms·

Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” …it’s: > And simply removing Python from production workloads. Python can rea

Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” …it’s: > And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches. You know, those hugging face guys kindaaaaa do know what they’re talking about. Like, really. Python is a pain in the ass in production environments.

ravenstine 3y ago
> You know, those hugging face guys kindaaaaa do know what they’re talking about. > Like, really. Trust me, bro?
fooblaster 3y ago
Pytorch has multiple production paths for deployment models without the python interpreter already, like torchscript and libtorch, or more annoying paths like onnx export and onnx runtime. So there is no need to have rust to solve this problem.
- malcolmgreaves 3y ago
  Those are great technologies for making the model inference independent of Python. However, there's always a good deal of pre and post processing, or other business logic, that needs to go around model inference. This sort of thing needs to be done in a general programming language. It's in these situations where Python is often used (because the supporting model code is often written by the same person, and that code is probably going to be Python because your model training and eval code is most likely going to be Python too). This is where having a non-Python PL, such as Rust, can make a huge difference for both ease of production deployment/maintenance as well as having truely efficient production inference.
  - fooblaster 3y ago
    I've found that there is usually very little code to be shared within training and inference when most of the pre and post processing is pulled into the model itself with custom operations. Relatively few people are aware that you can write training code in C++ today with torch to share a common language with inference and training, but very few take advantage of this in practice.
devjab 3y ago
> Python is a pain in the ass in production environments. I really dislike blanket statements like that one. Because any programming language can be a pain in production environments. One of the disadvantages of Python is also one of it's advantages. Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points is so mature. With those in place Python becomes great in production. Yes it'll require your organization to take some serious CI culture decisions and also enforce them. The thing is, however, while you can get a long way without doing that with some programming languages, you're frankly always going to want them by the time you reach enterprise size. So it's just so much easier if you get it done right away, and you will, if you're serious about using Python. I'd argue that if it's a pain to work in your production environments then the issue is not the technology but the process.
- coldtea 3y ago
  >Because any programming language can be a pain in production environments. Except one with memory safety and static builds. Like Rust can offer.
  - oivey 3y ago
    I wish my bugs were mostly preventable via static typing and memory safety checks. For most ML and data work algorithm design and logic errors are the primary source of bugs. Type errors are usually the trivial things you fix in minutes.
    - coldtea 3y ago
      The topic wasn't bugs though - and nobody mentioned static typing. The grandparent said "Python is a pain in the ass in production environments". And the parent said: "any programming language can be a pain in production environments". My point was how Rust is better for backend services for fewer problems in production, as it wont have memory bugs (which have long plagued backend services and have resulted in many exploits), and it also offers static builds (so deployment can be dead easy, and wont be having the kind of environment issues some languages have). I understood the grandparent to mean something about how Python has issues with dependencies, and isolation, and paths, and updates, and so on. And the parent to mean that you can have deployment issues with any language. And to obviously neither to refer to design and logic errors, since the former wouldn't be singling out Python, and the latter would be saying a tautology. And neither would make sense to talk about those being an issue in production specifically.
alex_lav 3y ago
This comment seems abnormally toxic for what's being discussed. > Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” This was never said. > You know, those hugging face guys kindaaaaa do know what they’re talking about. This was never challenged. > Python is a pain in the ass in production environments. This is "I don't like python", but phrased adversarially. Language flamewars suck. Some people don't like certain languages in certain contexts. I see Rust's syntax (having written a small to medium amount of Rust) and feel pretty confident I wouldn't task my data science team with writing or maintaining Rust codebases. We can squabble over "But eventually you'll get it", but ultimately that's more of a priorities comment than a language feature.
- nmstoker 3y ago
  I'm not getting a sense of toxicity from this discussion. Sure, people are disagreeing and have distinct positions but the manner of discussion seems constructive. >> This was never said. Quotes don't have to used solely to indicate somebody explicitly said something.
- maximilianburke 3y ago
  > This is "I don't like python", but phrased adversarially. But it is an incredible pain in the ass in production environments. Environment drift is something that _always_ seems to happen, no matter how diligent people are about using venvs or dependency managers or w/e. One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code. Recently we had mypy stop checking one of our internal packages because of a Dockerfile refactoring. The package could still be found at runtime but the type information just wasn't found by mypy. We didn't notice it until we started getting errors at runtime, which made the fix a lot more difficult to diagnose. There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started. For a language which states "There should be one -- and preferably only one -- obvious way to do it" as part of its guiding principles, that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".

ravenstine 3y ago

> You know, those hugging face guys kindaaaaa do know what they’re talking about. > Like, really. Trust me, bro?

fooblaster 3y ago

Pytorch has multiple production paths for deployment models without the python interpreter already, like torchscript and libtorch, or more annoying paths like onnx export and onnx runtime. So there is no need to have rust to solve this problem.

malcolmgreaves 3y ago
Those are great technologies for making the model inference independent of Python. However, there's always a good deal of pre and post processing, or other business logic, that needs to go around model inference. This sort of thing needs to be done in a general programming language. It's in these situations where Python is often used (because the supporting model code is often written by the same person, and that code is probably going to be Python because your model training and eval code is most likely going to be Python too). This is where having a non-Python PL, such as Rust, can make a huge difference for both ease of production deployment/maintenance as well as having truely efficient production inference.
- fooblaster 3y ago
  I've found that there is usually very little code to be shared within training and inference when most of the pre and post processing is pulled into the model itself with custom operations. Relatively few people are aware that you can write training code in C++ today with torch to share a common language with inference and training, but very few take advantage of this in practice.

devjab 3y ago

> Python is a pain in the ass in production environments. I really dislike blanket statements like that one. Because any programming language can be a pain in production environments. One of the disadvantages of Python is also one of it's advantages. Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points is so mature. With those in place Python becomes great in production. Yes it'll require your organization to take some serious CI culture decisions and also enforce them. The thing is, however, while you can get a long way without doing that with some programming languages, you're frankly always going to want them by the time you reach enterprise size. So it's just so much easier if you get it done right away, and you will, if you're serious about using Python. I'd argue that if it's a pain to work in your production environments then the issue is not the technology but the process.

coldtea 3y ago
>Because any programming language can be a pain in production environments. Except one with memory safety and static builds. Like Rust can offer.
- oivey 3y ago
  I wish my bugs were mostly preventable via static typing and memory safety checks. For most ML and data work algorithm design and logic errors are the primary source of bugs. Type errors are usually the trivial things you fix in minutes.
  - coldtea 3y ago
    The topic wasn't bugs though - and nobody mentioned static typing. The grandparent said "Python is a pain in the ass in production environments". And the parent said: "any programming language can be a pain in production environments". My point was how Rust is better for backend services for fewer problems in production, as it wont have memory bugs (which have long plagued backend services and have resulted in many exploits), and it also offers static builds (so deployment can be dead easy, and wont be having the kind of environment issues some languages have). I understood the grandparent to mean something about how Python has issues with dependencies, and isolation, and paths, and updates, and so on. And the parent to mean that you can have deployment issues with any language. And to obviously neither to refer to design and logic errors, since the former wouldn't be singling out Python, and the latter would be saying a tautology. And neither would make sense to talk about those being an issue in production specifically.

alex_lav 3y ago

This comment seems abnormally toxic for what's being discussed. > Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” This was never said. > You know, those hugging face guys kindaaaaa do know what they’re talking about. This was never challenged. > Python is a pain in the ass in production environments. This is "I don't like python", but phrased adversarially. Language flamewars suck. Some people don't like certain languages in certain contexts. I see Rust's syntax (having written a small to medium amount of Rust) and feel pretty confident I wouldn't task my data science team with writing or maintaining Rust codebases. We can squabble over "But eventually you'll get it", but ultimately that's more of a priorities comment than a language feature.

nmstoker 3y ago
I'm not getting a sense of toxicity from this discussion. Sure, people are disagreeing and have distinct positions but the manner of discussion seems constructive. >> This was never said. Quotes don't have to used solely to indicate somebody explicitly said something.
maximilianburke 3y ago
> This is "I don't like python", but phrased adversarially. But it is an incredible pain in the ass in production environments. Environment drift is something that _always_ seems to happen, no matter how diligent people are about using venvs or dependency managers or w/e. One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code. Recently we had mypy stop checking one of our internal packages because of a Dockerfile refactoring. The package could still be found at runtime but the type information just wasn't found by mypy. We didn't notice it until we started getting errors at runtime, which made the fix a lot more difficult to diagnose. There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started. For a language which states "There should be one -- and preferably only one -- obvious way to do it" as part of its guiding principles, that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".