18 ms·
Why HTTPS for Everything?
- ns8sl 10y agoAll web traffic that is not encrypted is vulnerable to having its contents altered enroute. This is a type of man in the middle vulnerability that allows for javascript, posts, etc. to be changed into something malicious.
- timthorn 10y agoI feel for later generations - learning tech gets harder and harder. Of course as the sum of knowledge increases this must be true, but as we go HTTPS-only the ability to type commands to a web server over telnet as a learning experience will be a loss.
- besselheim 10y agoYou could type them over openssl s_client instead.
- pimlottc 10y agoIndeed, the openssl tool is a veritable swiss army knife for PKI crypto. It's a netcat replacement: openssl s_client -connect www.google.com:443 while also providing information on the TLS handshake that's useful for debugging (like the server's certificate chain or its list of trusted CAs for client certificates). There's dozens of other subcommands to do useful things like decode certificates (x509), generate keys (genrsa/gendsa) and create certificate signing requests (req), just to name a few.
- syncsynchalt 10y agoIt's not a great netcat replacement, for one it's not binary transparent (eg. a SMTP "RCPT TO" causes a rekey due to the pattern "\nR.*\n"). The command line usage is atrocious too. That said there are many good alternatives (ncat, telnet-ssl, etc), and eventually one will gain the popularity and ubiquity that nc, curl, and similar tools did before them.
- majewsky 10y agoopenssl s_client is really valuable if you regularly work with minimal container images that don't have curl, wget or even ncat. openssl(1) is almost always there.
- ratchetratchet 10y agoI got you bro! $:openssl s_client -connect localhost:443 CONNECTED(00000003) ...snip.... lots of cert info ...snip... GET / HTTP/1.0
- userbinator 10y ago...as is the ability to inspect the traffic in your network to and from the devices you own. I think that is an even scarier situation, considering what others have discovered about "smart" devices precisely because their traffic was not encrypted. E.g. https://news.ycombinator.com/item?id=6759426 https://news.ycombinator.com/item?id=6759426 Personally, I'm for HTTPS connections to things like government websites (which is what this article seems to be mostly about), but against "HTTPS everything" in the way it's going to be implemented.
- ycmbntrthrwaway 10y agoIf you want to avoid telemetry, you need to use open source devices. Prohibiting HTTPS for everything non-government is not a way to go.
- test235 10y ago> but against "HTTPS everything" You being against HTTPS everything, is the same as being in support of MITM attacks somewhere. I am curious when is that the allowable case?
- userbinator 10y agoI own my computing devices and I should be able to control the traffic they create. Encryption must not prevent me from doing that.
- rhaps0dy 10y agoSince you own the computing device and the connection, it's theoretically possible to read the session encryption keys from its memory. This may be really hard in practice though.
- dschep 10y agoAnd you still can MITM the traffic, you just have to install your MITM's cert on your device you want to MITM.
- LeanderK 10y agoi think that's more a tooling problem. Making tech accessible to someone new really important.
- dTal 10y agoLearning a field should get easier the more we collectively know about it - the information should be structured better. This is particularly true of software, where we decide how to structure it. If tech is getting harder, this is our failing, not an inevitability.
- pjc50 10y agoOther way round: the more structure we build, the more there is to learn. The blank-slate nature of software is particularly problematic here; people end up proliferating solutions faster than they can be learned.
- spydum 10y agoAs others have pointed out, there are tools for creating a TLS socket, not much harder than nc or telnet. However, HTTP/2 will be a totally different game.
- marios 10y agoFWIW, OpenBSD's netcat supports creating a TLS socket. I'm not sure it will work on all Linux distributions since it depends on libtls/LibreSSL and most ship with OpenSSL.
- ycmbntrthrwaway 10y agoEven with HTTP you usually don't use netcat beyond playing with simple GET requests. Most web app developers and hackers use developer tools available in their browser, libraries for their favourite language (urllib for Python, LWP for Perl) and specialized command-line tools (curl).
- spydum 10y agofully agree, but I think the point was around the educational value of literally being able to handcraft HTTP requests. I learned a ton back in the 90's doing exactly that for various plaintext protocols (FTP, SMTP, POP3, IMAP, HTTP). Yes, there are tools that let you inspect them, but there's something about being able to walk around right at the protocol level to understand it's nuances (e.g.: CR/LF issues with HTTP). However, all of that being said, I'm sure the real old-school hackers think all this PHP/Python/Perl mumbo jumbo obscures the real C/C++ code which their interpreters actually drive. And those old-old-school hackers think those C/C++ guys are obscuring their assembly code.. okay, I kid, but you get the point. We all deal with abstractions at some point. Perhaps in time, HTTP/2 tooling will come to improve, and my concerns will vanish as well.
- omegaham 10y agoYou might have to do some custom work to make some of the "history" accessible. For example, Stanford's CS144 class uses a patched version of the Linux kernel to enable people to create their own TCP/IP clients[1]. I'm sure that if stuff like this becomes really inaccessible for newbies, similar modifications will be done for other applications to allow simpler concepts to be taught and explored. [1] http://web.stanford.edu/class/cs144/assignments/ctcp/assignment.html http://web.stanford.edu/class/cs144/assignments/ctcp/assignm...
- ycmbntrthrwaway 10y agoTelnet/netcat seems simple only because all the layers below are hidden inside the operating system. You don't type in all the MAC/LLC/TCP headers, and you can avoid typing in TLS headers just by hiding them inside the library (LibreSSL, GnuTLS etc.). With libtls API you can think about cryptography in terms of "padlock icon" until you want to learn more.
- timthorn 10y agoNo, but you can build your own stack from scratch fairly easily, and it's easier to debug when you can see the plaintext through the data structures.
- tetrep 10y agoAnd it's easier to debug a video codec if it's ASCII art, but there's huge benefits for using TLS everywhere instead of plaintext, and they more than outweigh "it's harder to debug."
- paulddraper 10y agoTrue, though reimplementing those is child's play compared to TLS.
- tptacek 10y agoHaving done semi-serious implementations of both, I disagree. Implementing IP and TCP and getting it hooked up to your OS so that your packets get spat out the network card and your responses get relayed back to your code is much harder than implementing a minimal TLS.
- ycmbntrthrwaway 10y agoTrue. Minimal TCP with 1 MSS window may be easy, but proper congestion control with fast recovery, F-RTO, tail loss probe, SACK etc. is much harder. Miss one of these aspects and you get a TCP that takes minutes to recover from a lost packet in some obscure case. It took years to debug Linux TCP stack. Even BSD stack is already way behind.
- Klathmon 10y agoThat's how progress works. 50 years ago you could work on a brand new car with just the tools in your garage, now you'd need specialized knowledge, equipment, and more to the point that it's not really possible to be a home mechanic on many areas. This isn't new for new's sake, these improvements bring real benefits. And as things get more complex, people will specialize more and more. In the future, your average "dev" might not know the ins and outs of how the transport layer works, but that's okay because there is someone else who mainly only does that.
- pythonlion 10y agoyou can build a brand new 1967 car with just the tools in your garage? Wow
- chucky_z 10y agoI mean... yeah, actually, you can -- if you had every (new) part from, say, a 1967 Ford F-series truck, anyone could absolutely, with just a Chiltons and a good set of tools, put together every nut, bolt, and screw from a giant crate of parts to build it.
- pythonlion 10y agoThat's interesting. I wish I could buy these "kits" today, put it together myself and drive a cheap cool car. Is that was a option then?
- stan_rogers 10y agoIt wouldn't be cheap. The safest, cheapest and most efficient "packing crate" for the parts for a single car is the assembled vehicle. (It's somewhat different when you can stack 50 left front fenders.) A kit car (most use fibreglass bodies) would be a much better option. Expect to spend about 1000 hours (much of that building jigs). The second one you build goes much smoother and faster than the first, even if it's a different kit based on a completely different pair of vehicles (the "source", from which most of the drive train parts will be drawn, and "target").
- tptacek 10y agoThis objection has always seemed pretty weird to me. Of course you can type HTTP commands character-by-character into a terminal. You just have to use a TLS-aware tool to do it. Meanwhile, you can't really just type HTTP commands to a server without tooling, because a whole bunch of TCP is happening behind the scenes. Why is "telnet" OK, but OpenSSL "s_client" isn't?
- tedunangst 10y agoYou can't type any lines beginning with Q or R with s_client.
- besselheim 10y agoYou can use the -quiet or -ign_eof options to disable this.
- icebraining 10y agoYou can also use socat (the multipurpose relay) to provide a bare TCP socket to which you can connect, even using telnet.
- mattstreet 10y agoThen it should also be noted that you really shouldn't use telnet to interact with HTTP or SMTP either. Telnet has issues if you send control characters, which wouldn't be used for either protocol, but might exist in embedded data you might want to send or receive. Just use netcat.
- tptacek 10y agoI guess, but it's hard to think of an HTTP request you can't reasonably make with telnet, while s_client apparently won't let you use the Referer header.
- discreditable 10y ago> the ability to type commands to a web server over telnet as a learning experience will be a loss. You can do this with openssl. openssl s_client -connect news.ycombinator.com:443 GET / HTTP/1.1 Host: news.ycombinator.com Press enter twice and you'll get HTML.
- cxseven 10y agoThere's also "socat", which is netcat with ssl support and it carries over its friendly command line.
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- wmil 10y agoThe bigger issue is hardware / software that manufacturers won't update. HTTPS best practices often require servers to drop support for old protocols. So if you pull out a 10 year old palm pilot and try to go to HN it won't work due to SSL.
- timthorn 10y agoIndeed, computer history museums are going to only be able to show older working exhibits - cloud backed gadgets will just be objects to look at and imagine what they used to be.
- meddlepal 10y agoJob security.
- kogepathic 10y agoIt's great to see a positive attitude toward security also gaining support in Government. I just wish all software projects felt the need to be more secure [0]: > Redis is designed to be accessed by trusted clients inside trusted environments. > While Redis does not try to implement Access Control, it provides a tiny layer of authentication that is optionally turned on editing the redis.conf file. > Redis does not support encryption. This is just broken by design software development. [1] It's irresponsible in 2016/2017 to assume you have impenetrable perimeter security. Redis' excuse is just bullshit. PostgreSQL supports authentication, SSL, and even client cert pinning. [0] https://redis.io/topics/security https://redis.io/topics/security [1] http://antirez.com/news/96 http://antirez.com/news/96 Edit: Loving the downvotes by butthurt developers who have never had a security audit...
- theGimp 10y agoRedis and Postgre are meant to play different roles. If you don't like what Redis does, you're welcome not to use it.
- kogepathic 10y ago> If you don't like what Redis does, you're welcome not to use it. I like what Redis does, and it's also heavily used in industry. What I am saying, not incorrectly, is that their approach to security is harmful to their users. Most of whom won't know, care, or implement additional security which they should. It's the same argument with HTTPS. Of course HTTPS is optional in a web server, but all major web servers support HTTPS, and there has been a concentrated push to have more people using SSL. e.g. LetsEncrypt We should be making it easier for people to deploy secure services. Redis' approach to security makes it extremely difficult for developers to deploy secure services. I'll say it again: It's irresponsible in 2016/2017 to assume you have impenetrable perimeter security.
- jrudolph 10y agoI don't get why it would be that way? It's built to be deployed in a DMZ on a private network and that's how 99% of users (typically professionals) use it.
- c3833174 10y agoSo, what's an embedded system supposed to do when it can't reach its NTP server and needs to validate a cert?
- rocqua 10y agoPresume the cert is wrong. If your system can't handle that, a few options: - put a clock in your embedded element - Pin certs - Use a frontend, embedded only connects to authenticated embedded system (say, with ssh Port forwarding). Frontend does connection correctly.
- lvh 10y agoAs usual: it depends. Why does it need to validate a cert? How acceptable is it if you get it wrong? Depending on the answer, perhaps "have a more reliable clock" is the right answer (plenty of embedded devices certainly have a decent idea of what time it is, and if it's already big enough to validate TLS). It seems reasonably probable that the NTP server stops being available for a reasonable amount of time before you have no idea what time it is anymore and can no longer validate certificates; so depending on the device, telemetry might be a good idea too. It doesn't sound like a reason to give up, though :)
- c3833174 10y agoWhat I meant is: - Device is rebooted - Can't reach NTP, no RTC or dead RTC battery, happy that time is January 1st, 1970 - HTTPS breaks
- yjftsjthsd-h 10y agoYou can reach a web host but not NTP? That seems like an edge case.
- Senji 10y agoAsk the user for the time.
- 10y ago
- dajohnson89 10y agoIsn't the CIO a presidential appointment? I wonder who the next one will be. Or, will there be one at all?
- konklone 10y agoYes, the federal CIO is a presidential appointment.
- therealmarv 10y agoMain part is "When properly configured". I recently tried out Brave browser on mobile which supports HTTPS everywhere. It slows the mobile web experience so much down that it is not funny. I really like to have HTTPS everywhere but unfortunately not everybody does a good job with that (Handshaking... SPDY, HTTP/2 etc.)
- nsgi 10y agoInteresting, do you get that when browsing HTTPS sites with mobile Chrome/Safari?
- dahart 10y ago> The IETF has said that pervasive monitoring is an attack, and the Internet Architecture Board (the IETF’s parent organization) recommends that new protocols use encryption by default. While HTTPS does prevent just anyone from monitoring, I've long been under the impression that the government, and possibly influential corporations, probably have access to any certificates issued by the large CAs. Is this a tinfoil hat theory? Is anything legally or technically preventing this from happening, and/or are there ways for me to know when my own browsing is truly private between myself and only the party at the other end, and not other curious or intrusive uninvited third parties?
- cesarb 10y ago> I've long been under the impression that the government, and possibly influential corporations, probably have access to any certificates issued by the large CAs. Everyone has that access. Click on the padlock icon on any HTTPS-using website, and after a few more clicks you can export a copy of the site's certificate (at least on Firefox). But that gains you nothing without the corresponding private key. The private key is generated on the website's server, and is never sent to the certificate authority (what is sent is a "certificate request", which has basically the same information found on the certificate). > are there ways for me to know when my own browsing is truly private between myself and only the party at the other end, and not other curious or intrusive uninvited third parties? Now that's a different question. While having access to the certificates is no problem at all, being able to create a new certificate for an arbitrary website allows one to pretend to be that website. The only defense against it is that, if a CA is caught issuing these certificates, it risks being removed from the browser's trust lists, which is a death penalty for a CA's business. Also, there is a new initiative (Certificate Transparency) to make it easier for these certificates to be caught.
- rhblake 10y ago> Now that's a different question. While having access to the certificates is no problem at all, being able to create a new certificate for an arbitrary website allows one to pretend to be that website. The only defense against it is that, if a CA is caught issuing these certificates, it risks being removed from the browser's trust lists, which is a death penalty for a CA's business. Also, there is a new initiative (Certificate Transparency) to make it easier for these certificates to be caught. There is a defense against rogue CAs: HTTP Public Key Pinning (HPKP) [0]. Chrome, Firefox et al use a HPKP preload list, but unlike with Strict Transport Security (HSTS) there currently appears to be no way to submit one's own site for inclusion in the preload lists. See e.g. Mozilla's policy [1]. [0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Public_Key_Pinning https://developer.mozilla.org/en-US/docs/Web/HTTP/Public_Key... [1] https://wiki.mozilla.org/SecurityEngineering/Public_Key_Pinning https://wiki.mozilla.org/SecurityEngineering/Public_Key_Pinn...
- eclipsetheworld 10y agoHaving a "https." subdomain feels so weird.
- paulddraper 10y agoYeah, should be https.www.cio.gov . It's too bad the www.www.extra-www.org people are gone; they could have teamed up for a new proposal.
- ggregoire 10y agoRelated: last week I deployed my web app for the first time, on AWS. In two clicks and for free I've been able to create and configure two certificates, one for the React app on CloudFront and one for the API on BeanStalk. Really surprised and amazed how simple and smooth was the whole process. Thanks Amazon for this.
- be21 10y agoNazi Enigma encryption was cracked, because they encrypted everything, even repetitive information like weather reports. Https for everything is promoted by NSA.
- Steeeve 10y agoIt's worth pointing out that this is an effort to secure government sites: "A Policy to Require Secure Connections across Federal Websites and Web Services" Which makes sense to a degree and on the surface. But it doesn't actually make sense. From an individual standpoint of "my personal government held information should be secure" it certainly does. From the standpoint that the government should provide easy and open access to data it does not. And let's face it - browser warnings about bad certificates are generally ignored, and the fact that they haven't presented one provides no real confidence that a connection is in fact secure. Should I really have to pay someone to find out if a car has been used as a rental car or in an accident? Is there a real benefit to that data being transmitted via an https session vs. a http session? If I'm looking at a congressman's voting history, is there a real benefit to that data being transmitted via an https session vs. http? What's the drawback? There's a layer of complexity in the way that isn't necessary or helpful. Anybody who has done web based automation has run into plenty of issues with expired / incorrect / insecure / misconfigured ssl certificates. And we've all seen trusted certificate authorities compromised. Heck, some of us compromise secure http ourselves pretty regularly as part of the development and testing cycle. What happens when we're accessing data from a custom device with minimal resources? Text processing and network connectivity is one thing. Adding in encryption support and maintaining SSL is a whole different problem to deal with. And what happens when a guy working a low level office wants to expose a web service for something like viewing an event calendar? There's additional work to be done to obtain certificates and configure his web server to use them. So now maybe he doesn't do it because the bureaucracy in place is too painful. --- I'm not a fan of making blanket decisions based on flimsy logic. I find it ironic that their first reason for doing this points to a TAG finding that not only points out several drawbacks, but lays out that one of the primary reasons for preferring secure communication would be to minimize pervasive government monitoring.
- cesarb 10y agoLike many, you are forgetting the other half of what TLS provides. It's not only confidentiality, it's also authenticity. It ensures not only that nobody can eavesdrop the connection within the browser and the server, but also that nobody can modify it, for instance to inject a piece of Javascript with a browser exploit.
- andrewfromx 10y agoI use neverssl.com everyday
- yjftsjthsd-h 10y agoWhy?
- andrewfromx 10y agocoffee shop wifi's will not connect unless u goto an http only site.
- majewsky 10y agoInteresting. I registered unencryptedwebsite.com for that purpose the other day (have not set it up yet, though).
- maxt 10y agoHere's a Mashable article about adopting HTTPS served via plain old HTTP: http://mashable.com/2011/05/31/https-web-security/ http://mashable.com/2011/05/31/https-web-security/ It worries me that major websites like this have still not made the switch to HTTPS/TLS yet. Quite irksome are the reasons (actually, excuses) site owners sometimes give like overhead, claiming switching over to HTTP/TLS will be costly and annoying, or even worse - that their threat model doesn't include HTTPS, and the burden is on the visitor to encrypt their connection to the site. The onus is on both parties to encrypt, instead of shunting the encryption to the visitor. As for threat models, the news can be a sensitive topic for some, and HTTPS can be of great service to visitors who enjoy their privacy. I enjoy initiatives like Secure The News[1] which is a small public awareness campaign urging news outlets to adopt HTTPS/TLS. Initiatives like Google's HTTPS Transparency Report[2] are great too and give us great insight into the adoption rate of HTTPS/TLS: [1] https://securethe.news/ https://securethe.news/ [2] https://www.google.com/transparencyreport/https/grid/ https://www.google.com/transparencyreport/https/grid/
- waqas- 10y agoim a lead dev for a large publisher. when we switched over to https we faced the following non-trivial issues: 1. a lot of third party advertisers/ad servers still run on http, these ads need to be embedded via DFP usually, which you can understand does not work out well. We made the switch months ago, to this date i am still making advertisers switch to https. 2. google says they give better ranking to https sites, thats simply not true so far as i have seen. in fact, in the short run your site takes a hit. not only that, in google webmaster console and google news, you cant shift from http to https, you have to make new accounts for ur https sites. to this day i do not know which ones is google crawling. For google news, my new https account has yet to be approved after months, it looks like google just magically shifts to https in google news. but if feels icky and hacky: explicit is always better than implicit. 3. microservices. remember those microservises that were all the rage? well, its a bunch of different servers and subdomains, you have to shift all to https when you shift the mothership to https. while above points are valid, i still pushed in my org to shift to https. we now use shiny stuff like http2 and web push, which is awesome. i'd recommend all publishers to do so. but its understandable that management finds all this scary, esp cuz its sounds like a major overhaul of your web assets (which is everything when ure a web publisher) - even though it isnt really actually an overhaul or anything.
- greggman 10y agoIs there any work on a solution for IoT or other end user programs/devices that would benefit from being able to serve HTTPS instead of HTTP?
- wereHamster 10y agoFunny that this comes from *.gov. It's in their (the governments) interest to keep traffic unencrypted so that they can intercept and store everything (makes life of CIA/FBI/NSA easier). Why bother enforcing HTTPS? /me confused The people at NSA are probably like "Oh god why? No, stahp it".
- vtlynch 10y agoIt's almost as if the government is made up of many departments and people who have different goals and values...
- konklone 10y agoAnd this is a White House policy, with their official blog post and rationale here: https://www.whitehouse.gov/blog/2015/06/08/https-everywhere-government https://www.whitehouse.gov/blog/2015/06/08/https-everywhere-... "It is critical that federal websites maintain the highest privacy standards for the users of its online services. With this new action, we are driving faster internet-wide adoption of HTTPS and promoting better privacy standards for the entire browsing public." (Disclaimer: I work on https://https.cio.gov https://https.cio.gov, at GSA.)
- noobiemcfoob 10y agoIt's only in the NSA's interest so long as having open access to criminal communications is more valuable than allowing all of their flock to walk around unprotected. Eventually the balance will flip, and our protectors will come to the rescue! Just in time, I'm sure.
- tommorris 10y agoThe government also have an interest in ensuring that the nation's businesses don't fuck up due to easily preventable infosec failure. Because infosec fails equals lost consumer confidence, lost economic development, lost taxes and much else besides. Governments have wider interests beyond simply sniffing all your traffic. This is why the UK's National Cyber Security Centre - which is the friendly business-and-civilian-facing side of GCHQ - publish lots of guidance telling people to use TLS. https://www.google.co.uk/search?num=20&q=allintext%3Ahttps+site%3Ancsc.gov.uk https://www.google.co.uk/search?num=20&q=allintext%3Ahttps+s...
- calvins 10y agohttps.cio.gov SSL Labs test result: https://www.ssllabs.com/ssltest/analyze.html?d=https.cio.gov https://www.ssllabs.com/ssltest/analyze.html?d=https.cio.gov Of note is that they're using a Let's Encrypt cert and running in AWS.
- davidgerard 10y agoThe thing that convinced my employer this was actually important was the announcement that Chrome 56 would mark non-SSL login pages as unsafe. Cheers to Google!
- edblarney 10y agoA better question is 'why HTTP for anything'?
- ArkyBeagle 10y ago"Today, there is no such thing as non-sensitive web traffic..." I just don't agree. Is it sensitive for me to google a man page or look up some algorithm or another? Because that's mainly what I use the Web for. The Web - not intranet ( which is usually quite sensitive ). Blog reading is sensitive? Hacker News? No, no they are not.
- Veratyr 10y ago> Is it sensitive for me to google a man page or look up some algorithm or another? It doesn't make sense to cut out such specific uses. The real question should be "are my Google searches sensitive?" and the answer to that, for most people, is "yes". > Blog reading is sensitive? Hacker News? No, no they are not. They're not particularly sensitive but they show anyone watching your internet connection that you're interested in these subjects. Why let that happen when you can not?
- ArkyBeagle 10y agoThe claim was that ALL Internet traffic is sensitive. I provided counterexamples. Position refuted. I don't care who reads my Google searches. Hope they have plenty of coffee, because it's pretty boring stuff. I still get on Usenet. So I know at a very deep level that it's all public. All to the better.
- newman314 10y agoOne thing I haven't necessarily seen raised is that including subdomains on HSTS headers would also affect internal websites on the same domain. I'd argue that it's a good reason/way to get all internal sites upgraded to HTTPS but it might not be feasible for a large organization. So something to consider.
- ge96 10y agoA+ on Qualys yeaaaa I see people mention Let's Encrypt, maybe I'm a sucker paying the $9.00 for a year's worth versus free but every 90 days.
- grzm 10y agoLet's Encrypt is set up to strongly encourage a completely automated set up, including renewal. From my experience "but every 90 days" is effectively for as long as desired.
- ge96 10y agoYeah maybe I'm just using it as an excuse for not learning to do Let's Encrypt. If it's the same as a standard $9.00 domain validation SSL then I could be saving that $9.00 by not being an idiot.