6 ms·
Show HN: Scanning the Web for Security.txt Files
- BigBalli 6y agoMaybe I'm missing something but what's the point?
- kissgyorgy 6y agoI agree, some kind of analysis on the results would have been more interesting.
- _wldu 6y agoThanks. Someone had done a scan for security.txt files before but were unable to scan the entire Alexa Top 1 million websites. Only the top 1,000. I checked the top 1 million from a 1.5Mbit residential DSL line using Go routines. The blog post has more details and a short overview of the results. It's linked at the bottom of the repo. I was hoping to get feedback on the code. https://www.go350.com/posts/a-survey-of-security-dot-txt/ https://www.go350.com/posts/a-survey-of-security-dot-txt/
- reaperducer 6y agoAn aside, but it would be nice if the tech industry could move beyond three-character extensions. There's zero reason this couldn't be security.text. It's been at least 35 years since I first saw a non-three-character file name extension (Amiga 1000), there are probably older examples. Computers are supposed to work for people, not the other way around.
- ffpip 6y agoImagine a .JavaScriptObjectNotation instead of .json. Or .firefox-extension instead of .xpi There are only a few important extensions. just remember them. > Computers are supposed to work for people, not the other way around. Extensions are not for average computer users. They have icons and filenames. Windows also hides extensions by default.
- corty 6y agoThere actually was a short period where Windows 95 was new and had just introduced long filenames with long extensions to former DOS users. Then, for a short time, stuff like .SomeSoftwaresDocument was actually popular.
- chrismorgan 6y ago.JavaScriptObjectNotation would be terrible, because of its verbosity and because people call the language JSON: so .json is good. .xpi is a rather poor example: no one talks about XPInstall (and the majority of its surface area is now even obsolete), so .xpi is to most people completely meaningless unless they have encountered it before and know what it is. .firefox-extension would be a vast improvement over .xpi, because it says what it actually is.
- remexre 6y agoJSON is bad, but as someone who doesn't know what an XPI is offhand, .firefox-extension seems kinda nice?
- reaperducer 6y agoOr .firefox-extension instead of .xpi That's an excellent suggestion. WTF does ".xpi" even mean to someone who just wants to install a browser extension?
- glandium 6y agoWhat do ".zip" or ".exe" mean to someone who doesn't already know the answer?
- throwaway201103 6y ago> Extensions are not for average computer users. They have icons and filenames. Windows also hides extensions by default. As far as I recall, extensions became a thing with DOS. They actually had meaning to the OS, e.g. naming a file .exe would make it executable, as there was no other concept of file ownership or permissions. In Unix/Linux systems, filename extensions have always been for the user. Before GUIs and icons, they were a convention that let the user know something about the file contents. As far as the operating system is concerned, "." is just another character in the name and extensions are meaningless. Note that most binary executables don't have an extension at all, and other files have more than one (e.g. .tar.gz)
- ivanhoe 6y agoWhy? Is there a single person in Universe that would ever be looking at these files and not know that txt is short for text? Perhaps in some other cases it makes sense, and perhaps we don't even need extensions at all for many files, they can be misleading (not to mention that what 'text' means is just another convention). Microsoft even hides extensions from average users. But the common extensions that we all know by heart, why change, what's the gain?
- chrismorgan 6y ago.txt is an ancient and extremely well-established extension, and so will not cause any trouble anywhere—it’ll be mapped to text/plain in all standard servers, out of the box. .text is not a common extension. Some things know that it’s text/plain (my Arch Linux /etc/mime.types and /etc/nginx/mime.types both do), but I expect some common server software won’t handle it properly out of the box (haven’t checked beyond nginx’s mime.types), and common OSes won’t have a handler for .text files set up (Windows, for example, comes with .text set to PerceivedType text like .txt has, so that it’ll suggest the right sort of apps to open it, but it’s still not hooked up to any app by default, unlike .txt which is “Text Document”).
- reaperducer 6y agoI stand by my previous statement. Computers should work for us. We shouldn't work for computers. Dump .txt for .text. Dump .jsn for .json. Keep .html, .jpeg, etc because they are abbreviations for standards. Sucks for Windows that it can't handle ".text" like other operating systems have since the 1980's. Dumping three-letter extensions will also help avoid all the extension namespace collisions that happen all the time.
- johnr2 6y ago>Computers should work for us. We shouldn't work for computers. Dump .txt for .text. If my computer expects me to type an extra letter every time I name a file it isn't working for me. The short extensions make for more efficient typing.
- Minor49er 6y agoSpeaking from my own experience, I've seen plenty of ".json" in the wild, but have never seen ".jsn".
- kiallmacinnes 6y ago.json is roughly the same as .txt - an abbreviated version of "JavaScript Object Notation". We humans still have to do the translation to the final name. And, I've never seen ".jsn". I've seen ".json" hundreds of thousands of times, but never ".jsn".
- tomc1985 6y agoPlease, lets not make computing any more cavemanlike than it already it. Is a world of obnoxious push-button apps with zero options or customizability not enough for you? Learn to love the contours of what you got and it will serve you even better!
- temp0826 6y agoDefinitely felt like wishful thinking whenever I heard that security.txt was a thing. From the blog post- > Of the 666,771 most popular websites on the Alexa list, I found 2,884 security.txt files that were content-type “text/plain” and returned a HTTP 200 status code. Not all of these were valid security.txt files, but most were. Seems I was right?
- _wldu 6y agoYes, it's not as widely implemented as I expected. Adoption seems to drop quickly. Roughly 20% for the top 10 websites, 15% for the top 100 and about 10% for the top 1,000. It's downhill from there.
- achillean 6y agoBtw Shodan checks for the existence of the security.txt file and shows/ stores the information if it's available. Here's an example of how it looks: https://beta.shodan.io/host/172.217.31.43#securitytxt https://beta.shodan.io/host/172.217.31.43#securitytxt If you have a Shodan account you can also search the contents of the security.txt files using the "http.securitytxt" search filter. For example: https://beta.shodan.io/search?query=http.securitytxt%3Acontact https://beta.shodan.io/search?query=http.securitytxt%3Aconta...
- lwansbrough 6y agoPutting a security.txt file up on our site has generated a ton of noise from people sending us their interpretations of burp suite results. Nothing substantial, yet everyone wants a payout and sends follow up after follow up. Feels like the Hacktoberfest Github thing. If you have a real security issue that someone cares enough to disclose to you, they'll find a way to contact you.