11 ms·
ZMap: Internet scanner maps all of IPv4 in 45 minutes
- dylangs1030 13y agoThis is very cool, but I'm curious as to why they stuck with ZMap. Nmap's graphical interface, Zenmap, sounds very similar and has a huge following among security researchers. For the technically inclined, a good white paper describing the advantages of IPv4-wide scanning for security reconaissance and the advantages of ZMap vs other tools like NMap can be found here: https://www.usenix.org/system/files/conference/usenixsecurity13/sec13-paper_durumeric.pdf https://www.usenix.org/system/files/conference/usenixsecurit...
- dadrian 13y agoProbably because the author's first name is Zakir.
- mrb 13y agoWhen they say "scanning the whole Internet in 45 min" they mean scanning only one port of every IP address (for example sending a short GET request to port 80/tcp) over a Gigabit link: 2^32 (IP addresses) * 1 (port per IP) * 80 (bytes per packet) * 8 (bits per byte) / 1e9 (throughput in bit/sec) / 60 (sec per min) = 46 minutes (note: excluding multicast space, RFC 1918 space, etc, scanning time would be reduced down to ~35 min) That's equivalent to "scanning all 65,535 ports of a /16 subnet in 45 min" which does sound less impressive...
- qwerty_asdf 13y agoscanning all 65,535 ports of a /16 subnet in 45 min ...or in other words: scanning all ports in the reserved Class C range, from 192.168.0.0 to 192.168.255.255, in 45 min
- jlgaddis 13y ago> ... "class C" ... I realize lots of people are simply in the habit of saying "Class C" when what they really mean is a /24, "Class B" for a /16, etc. but classless routing[0] has been around for 20 years now and these terms need to go away. [0]: http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing
- cwb71 13y agoExcept that qwerty_asdf wasn't referring to a /24 subnet, (s)he was talking about one of the three address spaces defined in RFC 1918 and described thusly: "Note that (in pre-CIDR notation) the first block is nothing but a single class A network number, while the second block is a set of 16 contiguous class B network numbers, and third block is a set of 256 contiguous class C network numbers." So it is common for crusty old network engineers and sysadmins to refer to 192.168/16 as "the class C" private block, even when they understand that you can subnet it however you'd like.
- jlgaddis 13y ago> ... (s)he was talking about one of the three address spaces defined ... Right, I realized that when s/he said "reserved Class C range". It was more of a general observation. I always forget I have to be extremely specific here on HN.
- Dylan16807 13y agoWhy should the terms go away? Obviously the networks have been widely broken up and shuffled around, but I see nothing wrong with calling 11.5.0.0/16 a class B network.
- jlgaddis 13y agoBecause classful networking went out the window 20 years ago. Case in point: 11.5.0.0/16 is not a Class B network and never was. (11.0.0.0/8, however, was/"is" a Class A network). /16 != Class B network.
- mobiplayer 13y agoIn order to send a GET request you may want to first establish a TCP connection, so it's going to be at least three packets. Otherwise you're not going to receive any response. Also, for these scans is quite common to just send a SYN packet and wait for the SYN/ACK to decide if the port is "open" or not.
- xyzzy123 13y ago> That's equivalent to "scanning all 65,535 ports of a /16 subnet in 45 min" which does sound less impressive... Actually, the above is a much harder problem. Scanning a limited subnet requires congestion control in a way that scanning the whole Internet does not.
- iotakodali 13y agogonna try it over university network
- jacquesm 13y agoMake sure to do a write-up when you regain internet access what it was like to be expelled for 'running just a script'.
- unimpressive 13y agoNow I'm curious. It's not safe to run this at home, on any sort of paid hosting, on a university network, where is it safe to run this program? If you work for an ISP? If you ask the University nicely?
- jacquesm 13y agoOn a colocation facility that you own using a net port that you pay for yourself? As soon as you expose someone downstream to stuff like this you're asking for being disconnected. If you ask your University nicely they'll likely refuse unless you state a goal you wish to achieve. If using a script to download documents qualifies as hacking then hitting all of the internet with a portscanner is likely going to get you network administrator attention of the entirely wrong kind. And that's because they in turn will get some flak from the outside world.
- ingenium 13y agoYeah our school basically had one big LAN (everything got a publicly routable IPv4 address, but nothing within the university was firewalled from anything else). There were networked printers though in the computer labs, some of these labs being on the first floor of the dorms. If you needed to print slides before class (and professors always posted the slides only like an hour before class), you had to go to the lab, hit print, and wait 15 minutes for the printers to get to your job in the queue and the staff to actually fetch it and lay it out. Definitely added to the time it took to get to class. Since I was already accessing my dorm computer via Samba in the labs (I know, dumb idea in hindsight, even with a password, but this was 2004), I decided to figure out a way to print directly from my room and then just grab it on the way to class. Long story short I ran a port scan on the computer lab to find the printer IPs and had my network port turned off within minutes (I had the IPs though!). Ended up having to go to some office and explain what I was doing. Got turned back on a few days later. The upside was that I eventually was able to print from my room as long as I converted whatever it was to postscript first. The downside was that I didn't need to know the printer IPs after all (the university's unix server already had the printers setup... just piped it through ssh to it).
- spindritf 13y ago"How to get dropped by your ISP in under an hour." There has been a very positive trend recently in the quality of documentation, a move away from dry, man-style listing of options to more operational descriptions, tutorial, examples, a bit of hand-holding. Here's a Docker tutorial[1], still on the front page. [1] https://www.docker.io/gettingstarted/ https://www.docker.io/gettingstarted/
- akl 13y agoA move away from having man pages isn't necessarily a thing to be celebrated - a well-written man page is an extremely useful thing. In the ideal case, you might have a quality man page that provides usage information and links to more detailed documentation (that would include tutorials, implementation info, etc.) on the web somewhere.
- jackinloadup 13y agoI've always thought tmux had a great man page. http://www.openbsd.org/cgi-bin/man.cgi?query=tmux&sektion=1 http://www.openbsd.org/cgi-bin/man.cgi?query=tmux&sektion=1
- nakkiel 13y agoI suppose the internet generation never really took the time to read man pages (nor to write one for that matter).
- jlgaddis 13y agoThat's too bad. When I was getting started, the best documentation available to me was the man pages (this was before I had an "always-on" broadband connection) and TLDP's "Linux HOWTO's". I printed many of them and took them with me to high school to study in class. It's great that we have blogs and such nowadays where anyone and everyone can contribute their own documentation, guides, tutorials, etc., but there was something awesome about having a single, centralized, authoritative HOWTO covering a particular topic.
- smutticus 13y ago
- __alexs 13y agoThe anonymously published whole Internet survey scanned 100 ports on every address as well as a few other things. It took something like 30,000 devices and months of work though so I guess this is pretty impressive. http://internetcensus2012.bitbucket.org/paper.html http://internetcensus2012.bitbucket.org/paper.html
- afreak 13y agoA friend of mine conveniently built a search around the data: http://exfiltrated.com/querystart.php http://exfiltrated.com/querystart.php
- ihsw 13y agoIt's interesting to see that this supports pushing to redis lists, which is clarified as being an 'output module.' https://zmap.io/documentation.html#extending https://zmap.io/documentation.html#extending Very good, very extensible.
- adamseabrook 13y agoRunning this from any web host will almost certainly get your server turned off along with a pile of abuse emails. To make it run even faster you should scrub all the Bogon routes on this list: https://www.team-cymru.org/Services/Bogons/http.html https://www.team-cymru.org/Services/Bogons/http.html I would also scrub all the sinkholes and captured botnet C&C ip addresses as hitting those will lower the reputation your netblock. List we use at meanpath is: http://mirror1.malwaredomains.com/files/domains.txt http://mirror1.malwaredomains.com/files/domains.txt https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist https://zeustracker.abuse.ch/blocklist.php?download=domainbl... https://zeustracker.abuse.ch/blocklist.php?download=ipblocklist https://zeustracker.abuse.ch/blocklist.php?download=ipblockl... http://malc0de.com/bl/IP_Blacklist.txt http://malc0de.com/bl/IP_Blacklist.txt http://hosts-file.net/download/hosts.txt http://hosts-file.net/download/hosts.txt http://www.joewein.net/dl/bl/dom-bl-base.txt http://www.joewein.net/dl/bl/dom-bl-base.txt http://www.dshield.org/feeds/suspiciousdomains_High.txt http://www.dshield.org/feeds/suspiciousdomains_High.txt http://www.malware.com.br/cgi/submit?action=list http://www.malware.com.br/cgi/submit?action=list https://spyeyetracker.abuse.ch/blocklist.php?download=domainblocklist https://spyeyetracker.abuse.ch/blocklist.php?download=domain... https://spyeyetracker.abuse.ch/blocklist.php?download=ipblocklist https://spyeyetracker.abuse.ch/blocklist.php?download=ipbloc...
- acd 13y agoAs a phun side note. Reverse mapping government censorship of DNS. Since I think governments are starting to go down the slippery slope road of censorship here is how map out the censorship. If you have the list of the whole internet servers which answers on http port 80. Then you can reverse map government censorship dns list. Ie you can find out what the government wants to censor by doing lookups in the censored dns and for example opendns on the servers ip that answers on port 80, then you diff the results from the dns servers and if you get different answers you find out the government black list.
- tptacek 13y agoI thought the approach they took to permuting the address space was particularly clever, but was quickly schooled on Twitter by how bog-standard the approach apparently is (I very much concede the point!). In the interest of honesty about how easily impressed I am, and because I still think it's a neat little trick: You want to generate a permutation of the entire IPv4 address space, but you don't simply want to shuffle every possible IP address because that would require you to keep an insane amount of state. So instead, work in the multiplicative group modulo p for prime p > 2^32, find an appropriate generator, and iterate by multiplying with the generator mod p. Remember the prime, the generator, the starting address, and your current address and you can detect a complete traversal of the space when the starting address recurs. There are a number of simpler ways to do this (after sheepishly conceding that this is pretty fundamental stuff, I played with using PRFs and card shuffling to do it; DrHoney suggested Gray codes), but I liked how immediately obvious the multiplicative group solution was, and that I could code it from a simple description.
- xyzzy123 13y agoI used skip32.c, but I like their approach too.
- buddydvd 13y ago+1 for skip32.c since it's O(1) space and O(1) time. With the prime approach, you may need to call it multiple times to reject numbers greater than 2^32-1. Also, a generalized Feistel algorithm may be used to generate permutations with fewer than 32 bits with the same constant space/time requirement and may be helpful in skipping specific IP ranges.
- tptacek 13y agoFWIW: http://www.cs.ucdavis.edu/~rogaway/papers/subset.pdf http://www.cs.ucdavis.edu/~rogaway/papers/subset.pdf (Marsh Ray tweeted this yesterday; it's an easy read)
- 13y ago
- anon90424671 13y agoWonder why they didn't mention unicornscan in their paper? User-land Distributed Portscanner released in 2005: http://unicornscan.com http://unicornscan.com Defcon Presentation Introducing Unicornscan from 2005: http://www.youtube.com/watch?v=ZdCEo6yoEWA http://www.youtube.com/watch?v=ZdCEo6yoEWA
- colonelxc 13y agoI'm somewhat surprised there is no reference to scanrand[1][2], a fast stateless syn scanner by Dan Kaminsky in 2002. It wasn't directly geared towards scanning the entire Internet, but instead scanning large subnets (like for a pen test of a /16 network... takes a long time to scan). It is a bit obscure, but it did do tricks like encoding encrypted data in extra mutable fields (just the sequence number for scanrand) for validation purposes. Actually, scanrand 2.0 can apparently measure latency (without state!) by encoding timing information in the source port field, which zmap doesn't currently do. I think this research is great, but I just hate to see interesting old projects get forgotten. [1] http://dankaminsky.com/2002/11/18/77/ http://dankaminsky.com/2002/11/18/77/ [2] http://www.sans.org/security-resources/idfaq/scanrand.php http://www.sans.org/security-resources/idfaq/scanrand.php [3] http://s3.amazonaws.com/dmk/SBO_Hiver.ppt http://s3.amazonaws.com/dmk/SBO_Hiver.ppt
- dsl 13y agoI currently scan the entire internet once a week using a re-implementation of scanrand I did myself. (and will be switching to Zmap shortly) There aren't as many people using it as you'd think because 1) finding a working download link is quite an exercise and 2) compiling paketto is near impossible except on Dan's machine. :)
- jnazario 13y agocheck out dscan from dugsong, which was built around 2003 or so to address that problem, that dan doesn't often write portable code. https://github.com/dugsong/dscan https://github.com/dugsong/dscan
- smutticus 13y ago1) "ZMap supports both blacklisting and whitelisting network prefixes. If ZMap is not provided with blacklist or whitelist parameters, ZMap will scan all IPv4 address (including local, reserved, and multicast addresses)." What an absolutely stupid default setting. Thanks for giving a bunch of noobs a simple IPMC DOS application. If this thing gets popular it will soon be the bane of network admins everywhere. 2) There are at least 2 obvious omissions from their default blacklist file. There might be more but these are the obvious ones that come to mind. class-E 240.0.0.0/4 CGN 100.64.0.0/10 3) Can someone explain to me why I wouldn't just want to use nmap to do this same thing? Why do we need a new tool for this?
- deleted 13y ago[deleted]
- muloka 13y agoIn case anyone is wondering if this worked on IPv6, and the rate was constant (45 minutes to do 2^32) it would take about 2 octillion days to scan all of the IPv6 addresses.