5 ms·
I wrote a fast tcp port scanner using Python 3.5 and concurrent.futures.ThreadPoolExecutor. See lines 123-124: https://github.com/jftuga/universe/blob/master/
by jftuga 10y ago
I wrote a fast tcp port scanner using Python 3.5 and concurrent.futures.ThreadPoolExecutor.
See lines 123-124:
https://github.com/jftuga/universe/blob/master/tcpscan.py https://github.com/jftuga/universe/blob/master/tcpscan.py
I have used this under Linux, OSX and Windows. It's cool to add the Thread Count field in Task Manager and then see something I wrote use so many threads! I am more of a sys admin, so this code could be better - but it seems to work very well. :-)
- orf 10y agoWhatever works for you, but changing 20 lines of code in this to use asyncio will send your throughput through the roof.
- revelation 10y agoThe limiting factor for a fast TCP scanner is not your favorite async toolkit.
- orf 10y agoWell, no, of course not. Don't be stupid. The default ThreadPoolExecutor size is the number of cores you have on your laptop. In my case, 8. So that's 8 concurrent connections at once. That's, frankly, shit. Use asyncio (or roll your own toolkit, or just spawn 'nmap') and you get 20000 concurrent connections. Which is better?
- rantanplan 10y agoYou're conflating concurrency with parallelism. Your 20000 concurrent connections with asyncio are not parallel. You can have 20000 threads if you want.
- orf 10y agoThe connections are actually handled by the OS in parallel, it's the callbacks to Python that are not. Why do you need a whole OS thread to handle sending a SYN/ACK? As you well know 20000 threads is not a great way to do anything. Especially in Python.
- the_mitsuhiko 10y agoI agree that threads in Python are not great but 20000 threads is otherwise not an issue.
- fzzzy 10y agoAre you serious? Have you ever tried starting 20k threads?
- fzzzy 10y agoAre you sure about that? Have you ever tried starting 20,000 threads? Try it.
- njharman 10y agoNot using something to deserialize all that blocking I/O waiting on the internet is THE limiting factor.
- hueving 10y agoquick github tip: if you click the line number on the left you can get a link directly to the line you are referring to. extra tip: after clicking the line, press 'y' on your keyboard and you'll get a link to the file in it's state at the current commit so future commits won't break your old hyperlinks.
- jftuga 10y agocool, thanks