4 ms·
Stress Testing HTTP with Twisted Python and Treq
- twakefield 14y agoSpeaking on using this with multiprocessing. AFAIK Twisted does not work well with multiprocessing – the details are not on top of my head. So the simple way to make this work on a multi-core machine is to wrap the whole thing (including imports) into a function and spawn a pool of processes BEFORE you import Twisted, initialize the reactor, etc.
- lvh 14y agoAs long as you mean the multiprocessing module, that's correct. Alternatively, you can use the reactor's spawnProcess method for creating processes, or Ampoule if you want a process pool (which is nice because it works remote, too). Perspective Broker and it's object-capability successor, Foolscap, are more full-blown distributed computing tools, but they would work fine for this as well if distributed requests is what you want :) Of course, all of this mumbo jumbo is probably only required if you can't manage to saturate that network link with a single core :)
- lvh 14y agoIn case anyone was wondering, this is wrapped in a cooperator for a reason. At every yield, the reactor is given a chance to act. That's important, because otherwise the reporting function would only ever run after you've made all the requests, which is probably not quite what you want :) Also, you might want to replace the automatic rescheduling with twisted.internet.task.LoopingCall.