8 ms·
Shodan Search Engine: Search Engine for the Internet of Everything
- collsni 5y ago? Not a new product. Just a link to shodan?
- Izmaki 5y agoSomebody just realised the website exists I guess. Straight to the front page...
- guessmyname 5y agohttps://xkcd.com/1053/ https://xkcd.com/1053/
- belter 5y ago"The Lucky 10000 has been referenced over 10000 times on reddit" https://www.reddit.com/r/xkcd/comments/64yacz/the_lucky_10000_has_been_referenced_over_10000/ https://www.reddit.com/r/xkcd/comments/64yacz/the_lucky_1000...
- hbcondo714 5y agoMaybe, it was discussed here on HN eight years ago! https://news.ycombinator.com/item?id=5512477 https://news.ycombinator.com/item?id=5512477
- deleted 5y ago[deleted]
- proactivesvcs 5y agoI remember, years ago, sending abuse reports to Shodan's ISPs asking them to enforce their ToS against Shodan, and receiving several emails from Shodan telling me what their mission is and how it's not abuse. I continued reporting them and eventually their unsolicited scans ceased. Seemed like such a bizarre response to a perfectly cromulent request.
- vore 5y agoI feel like the incentive here is to make sure your devices are secured rather than asking them to stop scanning :-) Better someone who is openly providing you this information such that you can use it than a malicious attacker!
- proactivesvcs 5y agoPerhaps, but I knew how my systems were set-up, and their connections were odd enough that I noticed them. I would suggest this shows that my security precautions were working well.
- robbedpeter 5y agoThis seems like a perfectly Karen request. Information you expose to the public internet, behind an isp or not, is not private or secure. This is like complaining that people keep taking walks by your house whenever your wife sunbathes nude on your front porch. The people doing the looking aren't the problem. Where your wife is exposing anything sensitive is the problem. The solution is to move your wife to the privacy of the back yard, buy her a sunbooth,or accept that she's on display to the public. Much the same, make sure your firewall facing your public network connection only exposes what you want it to. Part of the implicit structure of the internet is that by connecting, you're agreeing to be connected to, to be visible to everyone else that's connected. Leave Shodan be, their presence is important and relevant to the ongoing discussions of educating the public and maintaining accountability for security decisions by giant corporations.
- deleted 5y ago[deleted]
- ozim 5y agoExample you picked has a flaw that nude wife will be fined even if she exposes herself on your property - but in a way that others can see her. Police officer will be quite quick to explain her why she should not do this :) Other point on "Karen request" is spot on because Shodan is making requests for research purposes and selling the results. They don't access anything that is behind password and they don't scan with intention to find something that they can break through. While I mostly don't agree with people "scanning is not a crime", because I don't believe randoms on the internet that they are "scanning just for fun" and I expect they scan to find something they can get into which is first step of crime "preparation". Then people don't understand intent of scanning is what makes it illegal/legal not the scan on itself. Just like knife is not good/bad but if you hold it to harm someone it makes it bad and illegal.
- GSGBen 5y agoThis is a great tool, but there's a caveat: Last time I checked Shodan didn't scan all ports, and from all locations. If you're using this to check the surface area of your infrastructure (including the notify-me-of-new-open-ports feature), always test with a full nmap scan yourself, preferably from within the same geographic region.
- napalmall 5y agothere are actually multiple alternatives available that are like Shodan but concentrate more on detailed port scanning for example https://www.idyllum.com https://www.idyllum.com or https://pentest-tools.com/network-vulnerability-scanning/tcp-port-scanner-online-nmap https://pentest-tools.com/network-vulnerability-scanning/tcp... There still really aren't much all encompassing tool sites out there and you need to mix and match unfortunately
- bikingbismuth 5y agoWhen I use Shodan as a data source for any info sec consulting, I try to get the point across that Shodan’s map of your network is not accurate (nmap is much better for that), but anything that an adversary sees on Shodan is likely to be an initial target (even if it is down at the time of the attacker looking). Shodan is a fantastic tool, but I never trust it as a single source of truth for a network perimeter.
- 1970-01-01 5y agoSlow news day? Next up: https://search.censys.io https://search.censys.io
- shp0ngle 5y agoI always wondered... how is shodan actually legal? In which jurisdiction is it? edit: apparently it's from Texas. https://www.linkedin.com/company/shodan/ https://www.linkedin.com/company/shodan/
- tephra 5y agoTo know is something is legal or not we have to know why it would be _illegal_. What laws are they breaking by port scanning? (I guess the default would be the CFAA but I doubt even that hammer could be used for simple portscanning?).
- rocqua 5y agoIf you are talking HTTP to an endpoint that you have no idea whether you are authorized to, and it happens that they have some ToS document stating "do not use unless", then it could be construed as 'unauthorized access'. This unauthorized access could be argued as being illegal under the CFAA. The argument is iffy, hinging on the difference between just talking a little bit to a random IP on the one hand; and talking HTTP to a specific site whose ToS you could have been aware off. That second case was recently relevant when some company (I think linkedIn?) managed to stop people crawling them because, even though the IP endpoint is public, the ToS of the site did not authorize crawlers. All this said, I could see the argument that it is illegal. As a bad analogy, suppose you just checked every door in the city for being locked, and if it wasn't locked, peeked in to record the door shape. That feels like trespassing.
- staysafeanon 5y agoI wonder how many leaks we've all heard about started with a simple Shodan search.
- ianmf 5y agoIf you are interested in shodan.io, every year the hold a one-day offer for a lifetime subscription for cheap.
- mkoryak 5y agoI got one of those once. Had it for about 2 years and then they banned me without any explanation. How do I know they banned me? Because all profile related actions (login, logout etc) suddenly became a 404 for me and no results would be returned. The 404 issue was resolved as soon as I deleted my cookies, but of course at that point I could no longer login either. I emailed them a few times and was ignored. It was still a pretty good deal though.
- mceoin 5y agoExcerpt worth pulling out for HN readers: Crawling incidentally I think is the biggest issue with making a new search engine these days. Websites flat out refuse to support any crawler than Google, and cloudflare and other protection services and CDN’s flat out deny access to incumbents. It is not a level playing field. I would actually like to see some sort of communal web crawl supported by all web crawlers that allows open access to everyone. The benefits to websites would be immense as well, as they could be hit by a single crawler, rather than multiple, and bugs could be ironed out.
- lifthrasiir 5y agoYour comment has been misplaced, it should be under https://news.ycombinator.com/item?id=28665395 https://news.ycombinator.com/item?id=28665395 (I was pretty confused until that realization).
- infogulch 5y agohttps://commoncrawl.org/ https://commoncrawl.org/ is a big crawler with all data made public. Probably a good resource for new search engines.
- boyter 5y agoYes and no. It works for the deep web, but for anything that needs more frequent crawling no. They don't do it often enough. I have mentioned this elsewhere, but id love there to be a single crawl that everyone was able use and build indexes off. It would be especially good for webmasters too as you would have one bot to worry about and optimise for, reducing the load on systems, and allowing for innovation in the search space that cloud flare and other cdn's are stifling because they understandably are blocking crawlers that aren't Google or Bing. For example id love to see something added to robots.txt that allows you to say "crawl at 2am, hit has hard as you want, don't come back for a week" or some such. Having a single crawl could allow for this.
- marginalia_nu 5y agoWhile it's an obstacle, I don't think this is as big of an issue as they make it out to be, and I say this as someone who runs a DIY crawler and a janky homebrew search engine. You can get listed as a good bot with cloudflare &c if you ask nicely enough. Even if you aren't, their IP ranges are public so you can crawl them really slowly if you absolutely must have their stuff. In the end, you are never going to get completeness, that's a bad mark to aim for, and not even Google does that. That goal requires you have a model of the Internet that's as old as Vannevar Bush. The Internet isn't a bunch of static files on a souped up FTP server anymore, most pages are generated request-time, increasingly client-side. There are websites that don't just have infinite documents, they have infinite wildcard subdomains as well. There are websites that give you different results depending on what time of day it is, or where you are from. The hard problem of search engines is to find a good subset of the internet for some definition of good; and the first step toward doing that (in a timely fashion) is to rein in the crawling. Search engines are always always going to present a selection of web sites, and there are always going to be pages you can't find because they didn't make the cut; either because they were excluded, or because they excluded themselves.
- weinzierl 5y agoSince it is pretty well hidden and I only found out about it in a previous HN thread: Besides a free plan and contrary to the mainstream, shodan.io has a pretty attractive one-time payment plan. It is probably not enough if you need it 9 to 5 but perfect for the occasional research. That being said, there is also censys.io. They also don't advertise it prominently, but with their free Community Plan you can search their database for free and you'll get similar results to the free shodan search.
- maddyboo 5y agoThe cost of the one-time payment is behind a registration wall. It’s $49 for those curious.
- jasonladuke0311 5y agoI'm not sure if they still offer this, but I got my lifetime sub for $5 on black friday a few years back.