3 ms·

>Shark was born in an attempt to fix some python issues I have spotted with the years: Python is slow I constantly hear this, but are there some actual use cas

>Shark was born in an attempt to fix some python issues I have spotted with the years: Python is slow I constantly hear this, but are there some actual use cases where this commonly matters? Yes, yes, looping is slow in Python, but whenever someone in real life complains about Python being slow execution of the program still takes so little time that even if Python has 10 times slower it wouldn't make much of a difference. Or have I just misunderstood this completely and python actually is unusably slow for common things?

sixstringtheory 4y ago
Throughput. Of course it won't be obvious if you aren't running parallel implementations of your backend in python and C++/Rust/etc to compare your cloud billing, but 10x slower means 10x more compute requirements for the same service quality, doesn't it? And you have to pay for that. Maybe price doesn't increase linearly, but I'm sure it does increase.
- nextlevelwizard 4y ago
  Parellelism is a good reason to not use Python since GIL is what it is. But I would argue whatever you gain back from lowered hosting expenses probably aren't worth the extra development time & costs that come with lower level languages, but yes that depends on what you are writing.
  - nordsieck 4y ago
    > But I would argue whatever you gain back from lowered hosting expenses probably aren't worth the extra development time & costs that come with lower level languages, but yes that depends on what you are writing. I mean... I've used both python and go. And in my opinion, python doesn't really have much higher developer velocity. I'd never do web dev in C++, but Rust seems like a mixed bag. It is a bit notorious for having a steep learning curve. But once people are productive, it's arguably a higher level language than python.
    - Yoric 4y ago
      > I'd never do web dev in C++, but Rust seems like a mixed bag. It is a bit notorious for having a steep learning curve. But once people are productive, it's arguably a higher level language than python. Having done back-end in Python, TS and Rust, my experience confirms this: Rust is a very high language once you're past the steep learning curve.
  - Yoric 4y ago
    Possibly. However, the times are changing. There is an energy and components crisis and there are plenty of reasons to believe that it's going to grow much worse / more global. I suspect that at some point in the very near future, we'll need to relearn to code for a finite set of resources instead of just assuming that "the cloud" will solve it for a few more dollars.
xchkr1337 4y ago
I've used lots of obscure cli tools and the ones written in python are always so slow to the point where it's just annoying. Maybe python is meant to glue things together but many people just ignore that and write everything in python anyway.
- nextlevelwizard 4y ago
  Could you give examples? Because I know that there is subsection of people who simply refuse to use a CLI tool if they hear it was written in python, but I really want to get my hands on one so I can see for myself.
  - smabie 4y ago
    Try conda: its dependency resolution is hilariously slow
qsort 4y ago
> Or have I just misunderstood this completely and python actually is unusably slow for common things? I use python for some side-projects at work and for most of my personal stuff and it's mostly fine, but its general slowness/big memory footprint implies that you can't just write stuff the "easy" way, you have to use the "fast" API or else it's terrible, and that's infuriating. It's as though the power of your machine is partially hidden from you and you have to go around that limitation. It also feels like it "should" be faster than it is, which compounds the frustration. OTOH many performance losses come from using the wrong algorithm, often implicitly (wrong SQL query, wrong caching policy, wrong data structure), and Python is generally fine in that respect, much more than many other high-level languages in fact: the pythonic way is usually also algorithmically efficient.
- nextlevelwizard 4y ago
  I write a lot of python code at work. Mostly tools, but sometimes protocol libraries for our testing framework and while I've sometimes gotten complaints about performance of some of my tools usually when I end up profiling the tool it spend most of its time outside of python. Which is why I am a bit curious why many people think python is so slow it is not worth even using.
MathMonkeyMan 4y ago
Datadog's "agent," a monitoring daemon that customers install on their hosts, used to be mostly Python but is now mostly Go for performance reasons. I wasn't around when that happened, so I don't know if it was mostly a CPU/memory thing or mostly a GIL/throughput thing.
- influx 4y ago
  I love Python, but even if Go wasn't faster, trying to deploy Python dependencies on customer hosts sounds like a nightmare compared to compiling and distributing a single binary.
DeathArrow 4y ago
> but are there some actual use cases where this commonly matters? > Or have I just misunderstood this completely and python actually is unusably slow for common things? I don't know what you mean by common, but common sense tells me "common" things shouldn't be slow. Python was a glue language, ok for small scripts. And it's good at what was designed to do. Trying to use Python for everything is like trying to use Javascript for everything: a bad judgment. And JS is way faster than Python. Python is Visual Basic rehashed, the greatest thing for a zillion wannabe programmers because they are too lazy to learn something else.
- xavdid 4y ago
  > Python is Visual Basic rehashed, the greatest thing for a zillion wannabe programmers because they are too lazy to learn something else. This is needlessly inflammatory - characterizing Python programmers as lazy, stupid, or "not real programmers" has no place here. Many (very successful) businesses have been built using Python and, while they could have been written in other languages, there are always tradeoffs (developer experience, memory safety, package ecosystem, etc).
- nextlevelwizard 4y ago
  >Python is Visual Basic rehashed, the greatest thing for a zillion wannabe programmers because they are too lazy to learn something else. This is what I think most python haters actually hate. For some reason there is hate when someone makes things work faster (production time) than you with what you feel is inferior language. I learned to program with C++ and Java and I've dabbled with multiple languages. Currently I write everything I can with Python. I seriously do not see the benefit of making things run 10 or 100 or even 1000 times faster when everything already is fast enough for real life. I even coded my last embedded project with CircuitPython instead of C and just to check it out and I really can't feel the difference between the end products. >Javascript for everything: a bad judgment. There is inherently nothing wrong with writing most things in JavaScript the reason why we hate most JavaScript applications is Electrum - meaning the fact that it requires entire extra browser with it. Of course there will always be somethings where performance is key, but these are usually specific things and require purpose made solutions - and you know what? If you write that solution in C (and probably Rust) you can literally just call that library from cPython. >And JS is way faster than Python. I feel suspicious of this. What JavaScript on V8 has going for it is JIT, but I don't know if the difference between node and pypy is significant enough.
porcoda 4y ago
Your notion of “common things” may not be universal. In the sciences where Python is extremely popular, many common things are quite computationally demanding and feel the slowness of Python quite acutely and the slowdown certainly does make a difference. Looking at this shark thing, they appear to be concerned with games and interfaces, which have their own demands and likely have similar problems where the performance difference is very tangible.
Yoric 4y ago
Not sure if this answers your question, but the lack of (real) threads in Python is a big blocker if you are running any kind of performance-related code, e.g. a back-end. Yes, you can kinda emulate threads with processes, but suddenly, your code becomes much more complicated and all the readability benefits of using Python disappear. For a very long time, OCaml had the same problem. Now that OCaml has finally entered the multicore era, I will see if I can get back in touch with the OCaml magic :)

sixstringtheory 4y ago

Throughput. Of course it won't be obvious if you aren't running parallel implementations of your backend in python and C++/Rust/etc to compare your cloud billing, but 10x slower means 10x more compute requirements for the same service quality, doesn't it? And you have to pay for that. Maybe price doesn't increase linearly, but I'm sure it does increase.

nextlevelwizard 4y ago
Parellelism is a good reason to not use Python since GIL is what it is. But I would argue whatever you gain back from lowered hosting expenses probably aren't worth the extra development time & costs that come with lower level languages, but yes that depends on what you are writing.
- nordsieck 4y ago
  > But I would argue whatever you gain back from lowered hosting expenses probably aren't worth the extra development time & costs that come with lower level languages, but yes that depends on what you are writing. I mean... I've used both python and go. And in my opinion, python doesn't really have much higher developer velocity. I'd never do web dev in C++, but Rust seems like a mixed bag. It is a bit notorious for having a steep learning curve. But once people are productive, it's arguably a higher level language than python.
  - Yoric 4y ago
    > I'd never do web dev in C++, but Rust seems like a mixed bag. It is a bit notorious for having a steep learning curve. But once people are productive, it's arguably a higher level language than python. Having done back-end in Python, TS and Rust, my experience confirms this: Rust is a very high language once you're past the steep learning curve.
- Yoric 4y ago
  Possibly. However, the times are changing. There is an energy and components crisis and there are plenty of reasons to believe that it's going to grow much worse / more global. I suspect that at some point in the very near future, we'll need to relearn to code for a finite set of resources instead of just assuming that "the cloud" will solve it for a few more dollars.

xchkr1337 4y ago

I've used lots of obscure cli tools and the ones written in python are always so slow to the point where it's just annoying. Maybe python is meant to glue things together but many people just ignore that and write everything in python anyway.

nextlevelwizard 4y ago
Could you give examples? Because I know that there is subsection of people who simply refuse to use a CLI tool if they hear it was written in python, but I really want to get my hands on one so I can see for myself.
- smabie 4y ago
  Try conda: its dependency resolution is hilariously slow

qsort 4y ago

> Or have I just misunderstood this completely and python actually is unusably slow for common things? I use python for some side-projects at work and for most of my personal stuff and it's mostly fine, but its general slowness/big memory footprint implies that you can't just write stuff the "easy" way, you have to use the "fast" API or else it's terrible, and that's infuriating. It's as though the power of your machine is partially hidden from you and you have to go around that limitation. It also feels like it "should" be faster than it is, which compounds the frustration. OTOH many performance losses come from using the wrong algorithm, often implicitly (wrong SQL query, wrong caching policy, wrong data structure), and Python is generally fine in that respect, much more than many other high-level languages in fact: the pythonic way is usually also algorithmically efficient.

nextlevelwizard 4y ago
I write a lot of python code at work. Mostly tools, but sometimes protocol libraries for our testing framework and while I've sometimes gotten complaints about performance of some of my tools usually when I end up profiling the tool it spend most of its time outside of python. Which is why I am a bit curious why many people think python is so slow it is not worth even using.

MathMonkeyMan 4y ago

Datadog's "agent," a monitoring daemon that customers install on their hosts, used to be mostly Python but is now mostly Go for performance reasons. I wasn't around when that happened, so I don't know if it was mostly a CPU/memory thing or mostly a GIL/throughput thing.

influx 4y ago
I love Python, but even if Go wasn't faster, trying to deploy Python dependencies on customer hosts sounds like a nightmare compared to compiling and distributing a single binary.

DeathArrow 4y ago

> but are there some actual use cases where this commonly matters? > Or have I just misunderstood this completely and python actually is unusably slow for common things? I don't know what you mean by common, but common sense tells me "common" things shouldn't be slow. Python was a glue language, ok for small scripts. And it's good at what was designed to do. Trying to use Python for everything is like trying to use Javascript for everything: a bad judgment. And JS is way faster than Python. Python is Visual Basic rehashed, the greatest thing for a zillion wannabe programmers because they are too lazy to learn something else.

xavdid 4y ago
> Python is Visual Basic rehashed, the greatest thing for a zillion wannabe programmers because they are too lazy to learn something else. This is needlessly inflammatory - characterizing Python programmers as lazy, stupid, or "not real programmers" has no place here. Many (very successful) businesses have been built using Python and, while they could have been written in other languages, there are always tradeoffs (developer experience, memory safety, package ecosystem, etc).
nextlevelwizard 4y ago
>Python is Visual Basic rehashed, the greatest thing for a zillion wannabe programmers because they are too lazy to learn something else. This is what I think most python haters actually hate. For some reason there is hate when someone makes things work faster (production time) than you with what you feel is inferior language. I learned to program with C++ and Java and I've dabbled with multiple languages. Currently I write everything I can with Python. I seriously do not see the benefit of making things run 10 or 100 or even 1000 times faster when everything already is fast enough for real life. I even coded my last embedded project with CircuitPython instead of C and just to check it out and I really can't feel the difference between the end products. >Javascript for everything: a bad judgment. There is inherently nothing wrong with writing most things in JavaScript the reason why we hate most JavaScript applications is Electrum - meaning the fact that it requires entire extra browser with it. Of course there will always be somethings where performance is key, but these are usually specific things and require purpose made solutions - and you know what? If you write that solution in C (and probably Rust) you can literally just call that library from cPython. >And JS is way faster than Python. I feel suspicious of this. What JavaScript on V8 has going for it is JIT, but I don't know if the difference between node and pypy is significant enough.

porcoda 4y ago

Your notion of “common things” may not be universal. In the sciences where Python is extremely popular, many common things are quite computationally demanding and feel the slowness of Python quite acutely and the slowdown certainly does make a difference. Looking at this shark thing, they appear to be concerned with games and interfaces, which have their own demands and likely have similar problems where the performance difference is very tangible.

Yoric 4y ago

Not sure if this answers your question, but the lack of (real) threads in Python is a big blocker if you are running any kind of performance-related code, e.g. a back-end. Yes, you can kinda emulate threads with processes, but suddenly, your code becomes much more complicated and all the readability benefits of using Python disappear. For a very long time, OCaml had the same problem. Now that OCaml has finally entered the multicore era, I will see if I can get back in touch with the OCaml magic :)