5 ms·

Needed when executing 1 million API requests for example. Use the full force of your machine by utilising async IO via the excellent library Trio. It has been

by joshlk 6y ago

Needed when executing 1 million API requests for example. Use the full force of your machine by utilising async IO via the excellent library Trio. It has been tested in the wild with more than 10 million requests. Automatically handles errors and executes retries. Also provides helper functions for executing embarrassingly parallel async coroutines.

glic3rinu 6y ago
when you say parallel you mean the work is distributed across multiple cores? e.g. multiple python worker processes. Or is all work running single threaded in a single process (concurrent IO, but not true parallelism)?
- joshlk 6y ago
  All running in a single thread - but the work is executed asynchronously. This is preferred for I/O heavy throughput such as many HTTP requests.
  - nurettin 6y ago
    Why not distribute the concurrent work over multiple processes to get the most out of multiple cores? More event loops, better performance, and maybe more text preprocessing.
    - joshlk 6y ago
      That would be a good idea, but it's non-trivial in Python as multi-processing async support is patchy. If you have to choose multi-processing vs async for HTTP requests, from experience, async is much faster (as your not CPU bound but IO bound) and easier to use.
      - nurettin 6y ago
        I do both when I'm scaling to millions of requests per day. I also do some cpu bound preprocessing after the io bound async functions end.
        genidoi 6y ago
        Look into Go, specifically the colly pkg. Learnt Go just so I could use this library after wrestling with highly parralelized Python web scraping. The 1k/requests/sec/core figure is true and frankly deadly, and Go borrows a lot of useful stuff from Python like array slicing. It was pretty remarkable to go from 32 cores being maxed for 100 concurrent workers to... A 10-15% usage spike on a 4 core XPS :) https://github.com/gocolly/colly https://github.com/gocolly/colly
  - deleted 6y ago
    [deleted]
throwaway189262 6y ago
Really it's better to use a fast language like Go or Java for huge loads like this. Our typical CRUD Spring backends can handle ~15,000 requests a second with no optimization. I appreciate all the effort that's gone into making Python faster but I think it's still the wrong tool for the job at this point if heavy load is involved
- rovr138 6y ago
  If the rest of their stack is in python, not having to manage another language means that everyone else can switch to this and quickly keep going. There is a space for go and other languages, but also for libraries like these.
hansvm 6y ago
Nice job! Is vanilla Python networking is still dog slow, on the order of 1k requests per second per core on mid-range consumer hardware no matter how you wire it all together?

by joshlk 6y ago

glic3rinu 6y ago

when you say parallel you mean the work is distributed across multiple cores? e.g. multiple python worker processes. Or is all work running single threaded in a single process (concurrent IO, but not true parallelism)?

joshlk 6y ago
All running in a single thread - but the work is executed asynchronously. This is preferred for I/O heavy throughput such as many HTTP requests.
- nurettin 6y ago
  Why not distribute the concurrent work over multiple processes to get the most out of multiple cores? More event loops, better performance, and maybe more text preprocessing.
  - joshlk 6y ago
    That would be a good idea, but it's non-trivial in Python as multi-processing async support is patchy. If you have to choose multi-processing vs async for HTTP requests, from experience, async is much faster (as your not CPU bound but IO bound) and easier to use.
    - nurettin 6y ago
      I do both when I'm scaling to millions of requests per day. I also do some cpu bound preprocessing after the io bound async functions end.
      - genidoi 6y ago
        Look into Go, specifically the colly pkg. Learnt Go just so I could use this library after wrestling with highly parralelized Python web scraping. The 1k/requests/sec/core figure is true and frankly deadly, and Go borrows a lot of useful stuff from Python like array slicing. It was pretty remarkable to go from 32 cores being maxed for 100 concurrent workers to... A 10-15% usage spike on a 4 core XPS :) https://github.com/gocolly/colly https://github.com/gocolly/colly
- deleted 6y ago
  [deleted]

throwaway189262 6y ago

Really it's better to use a fast language like Go or Java for huge loads like this. Our typical CRUD Spring backends can handle ~15,000 requests a second with no optimization. I appreciate all the effort that's gone into making Python faster but I think it's still the wrong tool for the job at this point if heavy load is involved

rovr138 6y ago
If the rest of their stack is in python, not having to manage another language means that everyone else can switch to this and quickly keep going. There is a space for go and other languages, but also for libraries like these.

hansvm 6y ago

Nice job! Is vanilla Python networking is still dog slow, on the order of 1k requests per second per core on mid-range consumer hardware no matter how you wire it all together?