3 ms·

I have a half million users a day streaming website coded in Python. It does machine learning, uploading, encoding, dynamic load balancing, live search, comment

I have a half million users a day streaming website coded in Python. It does machine learning, uploading, encoding, dynamic load balancing, live search, comments, votes and most of the code I had to write was in django or rq endpoints. On first load, not only the page is ready, but the video has started to auto play (including buffering), under 3 seconds. Less than 500ms on a reload. The whole project has been running for 10 years on 7 servers. Hosting costs under $1K/mo. And even better, the code base is a mere 11K LoC, so a single dev can manage it. And make a living out of it. That's because: - Python is almost never the bottleneck in a website - Python wraps compiled languages for slow stuff. That's the whole point. - your archi matters much more than the language you chose. Because good db providers, proper caching and optimized data fetching will be the pareto moves, unless you are google size. - you will not code most things anyway. You are not going to code the video encoding, you will use ffmpeg. You will not recode a database, you will use an existing one. You will not code a server, you will use nginx or something else. What you will spend time on is creating logic for your clients. So I think your are extrapolating this rare use case where your POV makes sense, and ignore the 99% of the case were, indeed, a slow language doesn't matter. In fact, if electron success tells anything, it's that while I do value speed a lot, the market has voted.

rixed 5y ago
> I have a half million users a day streaming website coded in Python. It does machine learning, uploading, encoding, dynamic load balancing, live search, comments, votes and most of the code I had to write was in django or rq endpoints. Sounds much more likely that you wrote in python the code that bundle together the libraries that peform the actual network streaming, machine learning, encoding, search, etc, and that those libraries were not written in python.
- BiteCode_dev 5y ago
  Yes, that's my whole point. Where in the stack do you see that Python performances matter? Only one software out of a 1000 will actually need it. The others will mostly do plumbing or automation. And scripting languages excel at both. If you don't fall in that category, then yes, do use a fast language. If you are coding a chat server, use Erlang. If you want to do a rendering engine, use Rust. But if you most likely want to code your company CRUD software and glue it to some external services and providers, use Python.
  - rixed 5y ago
    > Where in the stack do you see that Python performances matter? As we just saw, most of the code forming your stack is not written in python because its performance would not allow it. > Only one software out of a 1000 will actually need it. Not what I see from your example. It seems to me that you are overinflating the volume of the python code that you wrote and are blind to the huge volume of performance critical code you just take for granted. You probably mean: most of the code that most of the programmer write is disposable code assembling reusable libraries. This code performance does not matter therefore python is fine. I agree on this. The problem I see with this reasoning though is that since python is so slow and error prone compared to compiled alternatives that it is unfit when scaled up from small disposable programs to largish permanent software, in a world where all programs trend to grow and stick in production for longer than expected.
hsn915 5y ago
3 seconds to load and $1000/mo hosting costs is way over the unacceptability threshold for me. I don't know what is the website you're running or how many people use it or how much money you make. I know you can make a lot of money doing stuff in python or electron. Does that mean the market has voted that performance doesn't matter? I don't think so. Jira makes a lot of money but almost everyone hates it. I know a lot of other software that is very slow but still makes tons of money. > you will not code most things anyway You say that like it's a good thing! It's a bad thing. You may say it's because you don't need to. But the flip side is that you can't, even if you wanted to. You can't program a database or a video encoding or a web server in this language you are using to write your program. So what is the point of this language? You can't even use it to cache things in memory. That's why everyone uses redis or something similar.

rixed 5y ago

> I have a half million users a day streaming website coded in Python. It does machine learning, uploading, encoding, dynamic load balancing, live search, comments, votes and most of the code I had to write was in django or rq endpoints. Sounds much more likely that you wrote in python the code that bundle together the libraries that peform the actual network streaming, machine learning, encoding, search, etc, and that those libraries were not written in python.

BiteCode_dev 5y ago
Yes, that's my whole point. Where in the stack do you see that Python performances matter? Only one software out of a 1000 will actually need it. The others will mostly do plumbing or automation. And scripting languages excel at both. If you don't fall in that category, then yes, do use a fast language. If you are coding a chat server, use Erlang. If you want to do a rendering engine, use Rust. But if you most likely want to code your company CRUD software and glue it to some external services and providers, use Python.
- rixed 5y ago
  > Where in the stack do you see that Python performances matter? As we just saw, most of the code forming your stack is not written in python because its performance would not allow it. > Only one software out of a 1000 will actually need it. Not what I see from your example. It seems to me that you are overinflating the volume of the python code that you wrote and are blind to the huge volume of performance critical code you just take for granted. You probably mean: most of the code that most of the programmer write is disposable code assembling reusable libraries. This code performance does not matter therefore python is fine. I agree on this. The problem I see with this reasoning though is that since python is so slow and error prone compared to compiled alternatives that it is unfit when scaled up from small disposable programs to largish permanent software, in a world where all programs trend to grow and stick in production for longer than expected.

hsn915 5y ago

3 seconds to load and $1000/mo hosting costs is way over the unacceptability threshold for me. I don't know what is the website you're running or how many people use it or how much money you make. I know you can make a lot of money doing stuff in python or electron. Does that mean the market has voted that performance doesn't matter? I don't think so. Jira makes a lot of money but almost everyone hates it. I know a lot of other software that is very slow but still makes tons of money. > you will not code most things anyway You say that like it's a good thing! It's a bad thing. You may say it's because you don't need to. But the flip side is that you can't, even if you wanted to. You can't program a database or a video encoding or a web server in this language you are using to write your program. So what is the point of this language? You can't even use it to cache things in memory. That's why everyone uses redis or something similar.