3 ms·

It's written in Python so as soon as they have enough users it will be as slow as any webpage with tons of JS. Thankfully, that CPU load won't be on the client

by hmnom 7y ago

It's written in Python so as soon as they have enough users it will be as slow as any webpage with tons of JS. Thankfully, that CPU load won't be on the client side... so it's still an improvement

kees99 7y ago
Are you sure? Looks like static pre-generated page to me: $ curl -s https://sourcehut.org/|egrep 'meta.*gen' <meta name="generator" content="Hugo 0.57.2" /> [0] https://gohugo.io/ https://gohugo.io/
- psychrometer 7y ago
  The front end of the site it static but the backend of the service seems to mostly by python. https://git.sr.ht/~sircmpwn/meta.sr.ht/tree/master/metasrht https://git.sr.ht/~sircmpwn/meta.sr.ht/tree/master/metasrht https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht edit: It is using flask according to this blog. https://drewdevault.com/2019/01/30/Why-I-built-sr.ht-with-Flask.html https://drewdevault.com/2019/01/30/Why-I-built-sr.ht-with-Fl...
- helb 7y ago
  The website/blog (https://sourcehut.org https://sourcehut.org) is static, the Sourcehut app itself (https://sr.ht https://sr.ht) is Python-based. https://git.sr.ht/~sircmpwn/?search=sr.ht https://git.sr.ht/~sircmpwn/?search=sr.ht
kdmccormick 7y ago
I recommend you relax your convictions about Python performance a bit. Server-side bottlenecks are more often than not database reads/writes and other IO. And the few CPU-intensive operations can be delegated to libraries written in C. Pure Python is slow for CPU-intensive tasks, but that doesn't mean that a Python webserver is necessarily slow.
- hmnom 7y ago
  So what you're saying is that the author of Sourcehat is expected to rewrite Flask, Jinja2, etc in C? As an example, https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht takes 600-800 ms to generate, and it's probably heavily cached already. What will happen when they get more users? The site will be unbearably slow, unless the guy starts spending thousands in servers.
  - kdmccormick 7y ago
    I imagine that the CPU-intensive parts of Flask and Jinja2 are already written in C. Much of Python's standard library is. 800ms is a reasonable response time, and if they scale up according to their userbase, they will hopefully maintain that time. (Also, we don't know how much of that 800ms is Python vs. IO)
    - hmnom 7y ago
      https://github.com/pallets/flask https://github.com/pallets/flask https://github.com/pallets/jinja https://github.com/pallets/jinja 0% C Also 800 ms is NOT a reasonable response time to generate what basically is a bunch of text, that is absurd, but I guess this is the baseline in 2019. I trust all IO is cached. The author can confirm it. This is just how slow Python is.
      - wyldfire 7y ago
        Switching to PyPy is likely to improve the overall performance if it's not database or I/O bound.
      - ignaloidas 7y ago
        Almost none of the IO of that page is cached actually[0]. The only thing I'm sure that is cached is the templates themselves. I'm pretty sure that neither git lookups, nor DB accesses are cached, where you can save time. And mind you, this is served from a single data center in USA, and the latency can already eat up a lot of that. I in Europe have 300ms ping to it, so it might be that you are simply far away from the physical location. [0] https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht/blueprints/repo.py#L136 https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht/bl...
sirn 7y ago
I used to run an image board written in Python with around 15m PV/month on a Pentium 4 machine in 2004 or so. The first bottleneck I found was the database. Some query optimization and caching fixed it very quickly. It goes a really long way until Python become a bottleneck, of which can also be fixed relatively easily (image processing in my case, which is fixed by moving the whole thing to a worker).
bluedino 7y ago
Python isn't slow. Running a lot of Python is slow.
stjohnswarts 7y ago
If it shards well and you have the capital to buy hardware, there's nothing wrong with using python

by hmnom 7y ago

It's written in Python so as soon as they have enough users it will be as slow as any webpage with tons of JS. Thankfully, that CPU load won't be on the client side... so it's still an improvement

kees99 7y ago

Are you sure? Looks like static pre-generated page to me: $ curl -s https://sourcehut.org/|egrep 'meta.*gen' <meta name="generator" content="Hugo 0.57.2" /> [0] https://gohugo.io/ https://gohugo.io/

psychrometer 7y ago
The front end of the site it static but the backend of the service seems to mostly by python. https://git.sr.ht/~sircmpwn/meta.sr.ht/tree/master/metasrht https://git.sr.ht/~sircmpwn/meta.sr.ht/tree/master/metasrht https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht edit: It is using flask according to this blog. https://drewdevault.com/2019/01/30/Why-I-built-sr.ht-with-Flask.html https://drewdevault.com/2019/01/30/Why-I-built-sr.ht-with-Fl...
helb 7y ago
The website/blog (https://sourcehut.org https://sourcehut.org) is static, the Sourcehut app itself (https://sr.ht https://sr.ht) is Python-based. https://git.sr.ht/~sircmpwn/?search=sr.ht https://git.sr.ht/~sircmpwn/?search=sr.ht

kdmccormick 7y ago

I recommend you relax your convictions about Python performance a bit. Server-side bottlenecks are more often than not database reads/writes and other IO. And the few CPU-intensive operations can be delegated to libraries written in C. Pure Python is slow for CPU-intensive tasks, but that doesn't mean that a Python webserver is necessarily slow.

hmnom 7y ago
So what you're saying is that the author of Sourcehat is expected to rewrite Flask, Jinja2, etc in C? As an example, https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht takes 600-800 ms to generate, and it's probably heavily cached already. What will happen when they get more users? The site will be unbearably slow, unless the guy starts spending thousands in servers.
- kdmccormick 7y ago
  I imagine that the CPU-intensive parts of Flask and Jinja2 are already written in C. Much of Python's standard library is. 800ms is a reasonable response time, and if they scale up according to their userbase, they will hopefully maintain that time. (Also, we don't know how much of that 800ms is Python vs. IO)
  - hmnom 7y ago
    https://github.com/pallets/flask https://github.com/pallets/flask https://github.com/pallets/jinja https://github.com/pallets/jinja 0% C Also 800 ms is NOT a reasonable response time to generate what basically is a bunch of text, that is absurd, but I guess this is the baseline in 2019. I trust all IO is cached. The author can confirm it. This is just how slow Python is.
    - wyldfire 7y ago
      Switching to PyPy is likely to improve the overall performance if it's not database or I/O bound.
    - ignaloidas 7y ago
      Almost none of the IO of that page is cached actually[0]. The only thing I'm sure that is cached is the templates themselves. I'm pretty sure that neither git lookups, nor DB accesses are cached, where you can save time. And mind you, this is served from a single data center in USA, and the latency can already eat up a lot of that. I in Europe have 300ms ping to it, so it might be that you are simply far away from the physical location. [0] https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht/blueprints/repo.py#L136 https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/gitsrht/bl...

sirn 7y ago

I used to run an image board written in Python with around 15m PV/month on a Pentium 4 machine in 2004 or so. The first bottleneck I found was the database. Some query optimization and caching fixed it very quickly. It goes a really long way until Python become a bottleneck, of which can also be fixed relatively easily (image processing in my case, which is fixed by moving the whole thing to a worker).

bluedino 7y ago

Python isn't slow. Running a lot of Python is slow.

stjohnswarts 7y ago

If it shards well and you have the capital to buy hardware, there's nothing wrong with using python