10 ms·
HTTP headers for the responsible developer
- Kiro 8y ago> Did you ever wonder why you can’t use local environments like my-site.dev via HTTP with your browser anymore? This internal record is the reason – .dev domains are automatically included in this list since it became a real top-level domain in February 2019. .dev hasn't been working locally for me for more than a year.
- chrisweekly 8y agoI use "foo.local" as a drop-in replacement, haven't had problems...
- voltagex_ 8y agoAny devices using mDNS might disagree.
- chrisweekly 8y agoSure; tho I only said _I_ haven't had problems using ".local". In any case, I think your peer commenter who replied about use of '.test' "wins", though, given the RFC.
- amichal 8y agoApparently “.test” is the RFC official one to use. Jts recently got around to fixing all my local projects https://en.wikipedia.org/wiki/.test https://en.wikipedia.org/wiki/.test
- LeonM 8y agoI have been trying to explain the importance of HTTP headers to some younger/junior devs I am working with. I have noticed that headers are often considered to be 'too technical' or even 'old tech'. I'm going to recommend them to read this, but I do think I need to explain a couple of things that the article is not clear about: - Be sure that you understand the concept of HSTS! Simply copy/pasting the example from this article will completely break subdomains that are not HTTPS enabled and preloading will break it permanently. I wish the authors made that more clear. Don't use includesubdomains and preload unless you know what you are doing. Scott Helme also did a great article about this [0]. - CSP can be really hard to set up. For instance: if you include google analytics, you need to set a script-src and a img-src. The article does a good job of explaining you should use CSP monitoring (I recommend sentry), but it doesn't explain how deceptive it can be. You'll get tons of reports of CSP exceptions caused by browsers plugins that attempt to inject CSS or JS. You must learn to distinguish which errors you can fix, and which are out of your control. - Modern popular frontend frameworks will be broken by CSP as they rely heavily on injecting CSS (a concept known as JSS or 'styled components'). As these techniques are often adopted by less experienced devs, you'll see many 'solutions' on StackOverflow and Github that you should set unsafe-inline in your CSP. This is bad advise as it will bascially disable CSP! I have attempted to raise awareness in the past but I always got the 'you're holding it wrong' reply (even on HN). The real solution is that your build system should separate the CSS from JS during build time. Not many popular build systems (such as create-react-app) support this. - Cache control can be really hard too. If you don't have time to fiddle with these settings, I recommend using a host like Netlify, they seem to do a proper job at caching in my experience. [0] https://scotthelme.co.uk/tag/hsts-preload/ https://scotthelme.co.uk/tag/hsts-preload/ edit: typos
- quickthrower2 8y agoIt could be that headers seem scary, and I think that is because you can get a lot done without them, so they might be unfamiliar to junior devs. To make them less scary maybe it is sufficient to say they are just key/value pairs, like an additional JSON object in the response. It's there for you to see in Chrome dev tools, and it's fun seeing how different sites might send different headers. As a motivator I always say look at the network tab and think about how you can make stuff faster. It was this mindset where I first saw these additional requests and thought 'WTF!' I better find out why it is happening, turned out to be preflight requests.
- geezerjay 8y ago> I think that is because you can get a lot done without them Reinventing the wheel is also a good way to fool yourself into assuming we're being clever and productive. I mean, it's as if everyone who preceded me in a field was incapable of looking into a problem I've managed to find a hacky solution in 5 minutes. Meanwhile let's not pretend that there are currently about half dozen standards and specifications for HATEOAS, all involving ugly hacks around response documents and funny media types, when all it takes to achieve the same goal is passing link relations through Link headers.
- barbecue_sauce 8y agoHow can you even get a job as a junior dev without knowing about headers?
- ljm 8y agoEasy, you just think you have to add a content type.
- quickthrower2 8y agoNever been asked about headers in a job interview, junior or otherwise. Usually interviews are a 'can you code' test. I've never been interviewed by FAANG etc. though.
- 8y ago
- kijin 8y agoThese are great features, but I wish there were better ways to communicate the security policies for my website than having to send lengthy headers with every page. CSP in particular tends to get rather long-winded. As the article says, it can contain up to 24 policies, many of which contain their own lists! It's bound to get even more complicated as web apps integrate with an ever greater number of external services. Feature-Policy also looks like it could easily balloon to 1KB or more if you wanted to control all the features. No matter how much compression you add, at some point this is going to affect the load time. Additional TCP round trips aren't cheap, especially for HTML resources that usually aren't cached at the edge. Wouldn't it be convenient if I could store a structured representation (JSON, YAML, whatever) at a predefined location under /.well-known/ and use ordinary Cache-Control headers to make browsers cache the rules?
- LeonM 8y ago> Feature-Policy also looks like it could easily balloon to 1KB or more Twitter sends over 6kb of CSP headers on every single request. This is what happens if you run loads of different advertisement and tracking vendors.
- perlgeek 8y ago> Twitter sends over 6kb of CSP headers on every single request. Now I understand why HTTP/2 uses compression for HTTP headers.
- r1ch 8y agoIf you're implementing CSP, you should only include the header on text/html or other rendered responses, so the overhead is more per-navigation than per-request. I've seen a lot of guides where CSP is added globally at the webserver level which can waste a lot of bandwidth with images etc.
- jeltz 8y agoYeah, I really do not like how CSP is implemented. It is hard to configure correctly and bloats all HTTP responses.
- 8y ago
- spiderfarmer 8y agoAren't advertising networks blocking the adoption of CSP headers? Seems like it's quite a job to maintain the exceptions needed for Doubleclick for example.
- LeonM 8y agoIt is always quite a job to maintain CSP. And it's really easy to break something with CSP. This is why loads of devs usually throw in the towel and disable CSP or use a unsafe-line. Basically like trying to solve a hard CSS problem and at some point give up and add !important statements everywhere. It's also really, really hard to explain to customers that it takes time to set up, and every time they install a new tracking/ad/video/whatever plugin on their CMS, you'll have to spend time on adjusting the CSP accordingly. That said, I do encourage developers to use CSP. It's a really powerful tool to secure your site and protect your visitors from fraud/phishing.
- anyzen 8y agoIt is also at least some level of defense against malicious npm packages (doesn't eliminate threat completely, but at least less sophisticated attacks will be thwarted). CSP headers are a very useful tool and I encourage everyone to use them. They are a PITA to set up though. Fortunately at least Firefox clearly communicates in console log when a CSP rule is hit, and how to relax it (if it was by mistake). Note that CSP can be set as META tags too. There's a gotcha though: if they are set in both places (HTTP headers and HTML META tags), an intersection of the rules is used.
- cuillevel3 8y agoWhen dealing with user uploads, we still need content disposition headers to force browsers to treat certain formats as attachments, rather than showing them inline, right?
- venins 8y agothanks :)
- deleted 8y ago[deleted]
- throwaway77384 8y agoI kind of understand why CSP isn't more widespread. I tried adopting CSPs on all my sites to full Mozilla Observatory[0] standard. One is a Go based Heroku instance, where I used unrolled/secure[1], though there are a few different packages achieving this. The others are static Netlify deploys using Netlify CMS. For those, you have to include a headers file (in my case I am instructing Hugo to build the site with a .headers media file included, which Netlify parses). Some observations: - It's a huge pain in the ass / trial and error process - The formatting for CSP rules was evidently made to be as insufferable as imaginable. All on the same line, with commas and semicolons being the only separators, no line-breaks, tabs or anything allowed. Seriously, wtf - When you think you've got it working, some other thing breaks in a weird, silent way - Debugging CSPs in Firefox is nearly impossible (as for certain in-line scripts, you will need to get SHA values to tell the CSP to let them through. Chrome provides the SHA in the console. Firefox bizarrely doesn't.) - Trying to integrate google recaptcha with CSP is hilariously complicated - You should try to host all fonts yourself, lest you need to enable google or fontawesome exceptions for font, CSS, script and svg, because apparently that's what you need just to get an FB icon on your page to work - Forget about React, or anything using inline-script or styles. Netlify CMS and the Netlify identity widget all require inline styles and scripts. Even generating SHA values for all of those, I could not get this stuff to work. In the end I gave up and disabled the CSP again And this is for static sites using really simple tooling. I have yet to find a viable way to make this work. edit (some additional notes): - Tools like this one[2] did not generate SHA values that were accepted by the CSP. I have tried a few different tools, checked all white spaces over and over. I just couldn't get it to work. Only Chrome returned the proper SHA value. - I tried fixing a hover state loading in improperly (it flickered on first hover). This wasn't related to the CSP, but because I had to try lots of different things, like load in an SVG sprite, or png sprite, try pre-loading, use some JS, etc. etc. I had to keep changing the CSP to work with this, too. So applying a CSP should only be done at the end of a project. At the same time, if anything breaks from one day to the next, your debugging will now include the CSP as well most likely. --- [0]https://observatory.mozilla.org/ https://observatory.mozilla.org/ [1]https://github.com/unrolled/secure https://github.com/unrolled/secure [2]https://passwordsgenerator.net/sha256-hash-generator/ https://passwordsgenerator.net/sha256-hash-generator/
- 8y ago
- loick 8y agoroblox mad city in hack
- loick 8y agomad city roblox hack
- hwj 8y agoThis is how you can get the headers: curl -I -X GET \ https://www.twilio.com/blog/a-http-headers-for-the-responsible-developer
- smt88 8y agoYou can see them in your browser's dev tools as well
- kbirkeland 8y agoI was curious why you added the `-X GET` to that, but it seems twilio returns 405 Method Not Allowed for HEAD requests. Is there any legitimate reason they would block these?
- pstuart 8y agoHEAD worked for me.
- pstuart 8y ago"-X GET" is superfluous, as that is the default method.
- bullen 8y agoI'm curious why/how XSS is a problem. Can someone describe a practical example of how this has been successfully abused? To me XSS allows a page to be distributed over many servers and that's more of a feature than a threat!
- justusthane 8y agoA very simple example could be a website that takes unsanitized forum posts. Include a link to a malicious script in your post, and anyone who visits that page loads the script. For another example, you could imagine a page on a website that includes the contents of a URL query in the response. For example, you visit www.goodsite.com/search.php?q=PurpleZebra, and the page displays "Your search for PurpleZebra did not return any results." Now in the URL replace "PurpleZebra" with "<script src='evilsite.com/script.js'>" and trick someone into clicking on that link - now the error page delivered by goodsite.com includes the script from evilsite.com.
- dmitryminkovsky 8y agoGoogle made a game that explains it well https://xss-game.appspot.com https://xss-game.appspot.com
- Gipetto 8y agoMaybe you are confusing this with RSS?
- whiskeykilo 8y agoXSS can cause many different kinds of problems: https://hackerone.com/hacktivity?order_direction=DESC&order_field=popular&filter=type%3Apublic&querystring=xss https://hackerone.com/hacktivity?order_direction=DESC&order_...
- cosmodisk 8y agoTry this one: https://trailhead.salesforce.com/en/content/learn/modules/secdev_injection_vulnerabilities/secdev_inject_cross_site_scripting https://trailhead.salesforce.com/en/content/learn/modules/se...
- gmiller123456 8y ago
- asaph 8y agoI read the whole article just to find out what the X-Shenanigans header shown in the picture at the top of the article is. There was no further mention of it. Looks like it's an inside joke from Twilio[0]. [0] https://github.com/kwhinnery/todomvc-plusplus/issues/7 https://github.com/kwhinnery/todomvc-plusplus/issues/7
- nurettin 8y agoDevelopers connect people. Developers help people. Developers enable people. I don't remember agreeing to these conditions. Is this some sort of psychological manoeuvre to get people to use SSL encryption on the web ?
- thrower123 8y agoI signed up for the building cool shit part, which seems to be omitted. One of the perks was that it was something you could do mostly by yourself, without dealing with the messy and irritating mass of humanity.
- twic 8y agoWhilst, if you ever upset one badly enough, you will find that sysadmins disable people.
- gambler 8y agoGotta love all the ritual incantations one has to perform to "keep your website safe" these days. Worst of all, people are clearly bragging about possessing this arcane knowledge, instead of constantly complaining about how stupid the whole thing is to begin with. "Responsible developer"? Hah. The web needs a real security model relevant to what browsers are doing today, not these piecemeal hacks ducktaped to a hypertext delivery protocol.
- admyral 8y agoAs someone who builds websites for money, I couldn't agree more. I rarely get to bill for making incremental changes, I get to bill for implementing features. Spending money to implement and log a properly restrictive Content-Security-Policy doesn't seem like wise use of my clients limited budget.
- dwheeler 8y agoIt may be a wise use if a security break-in would be a problem for your client. I am a big fan of restrictive CSP, but it's often hard to get there from an existing site. It's often better to do it in stages, e.g., when you work on page Q, you make that page have a restrictive CSP. Later, when you work on page R, that can grow one (or at least have fewer CSP issues). If having someone break into your site would be a serious problem, then you should speed up what it takes to get there.
- qubyte 8y agoCSP interacts in a surprising way (at least it was to me) with service workers. https://qubyte.codes/blog/content-security-policy-and-service-workers https://qubyte.codes/blog/content-security-policy-and-servic...
- davidcuddeback 8y agoYou can check your site's usage of most of these headers with https://securityheaders.com https://securityheaders.com. HSTS and more is checked by https://www.ssllabs.com/ssltest/ https://www.ssllabs.com/ssltest/. Definitely make sure you understand what the headers do before changing them. Don't just copy/paste what you see here.
- Tsubasachan 8y agoDevelopers connect people. Developers help people. Developers enable people. Look I understand that people in general feel the need to pretend that their work is very important and good but come on. You are not working for Warchild in a Lebanese refugee camp.
- llamataboot 8y agoTo me this just seems like more duct tape over the fact that everything on the web is doing things it wasn't really designed to do. We never could have imagined what we would be able to do with HTML/CSS/JS in a browser environment. We also never could have imagined how the pressure of business demands would essentially drive more and more duct tape solutions until the whole web was built on rickety scaffolding all sort of lashed together and swaying. There are backend bandaids and frontend bandaids but with the sheer amount of stack knowledge and framework knowledge required to do anything as a webdev these days, there's no way to stay on top of it all and we are just kinda winging some combination of best practices and getting shit done. I don't know if things like PWAs and WASM is going to allow us to move towards a change yet, and would love input form someone with an opinion.
- jimmaswell 8y ago> the sheer amount of stack knowledge and framework knowledge required to do anything as a webdev these days I think this is greatly exaggerated. You can get by just fine making your own sites knowing some basic html/css/maybe js, maybe some php too if you want backend stuff. Optionally some frameworks if you want, which should usually be easy enough to just follow some examples and get the functionality you want pretty fast. If you're put on an existing web project, you probably only have to learn the bits immediately surrounding the things you do, picking it up as you go along. I still don't know Angular, React, Vue, or much else in the way of JS frameworks other than jQuery after being in web dev professionally for years, as it simply hasn't been needed.
- llamataboot 8y agoYeah, sorry if I wasn't clear, but that's kind-of my point. You can make a site by knowing how to use some subset of stuff, but the rest you use as a black box - whether it is importing a bunch of third party libraries or being able to build a web app without understanding how cross-site attacks work, etc etc so at some point you either have to learn these little gotchyas all over the place or they go un-fixed. Like how many sites still don't have mandatory HTTPS even though it is free and easy?
- mro_name 8y agothx, my future headers cheat sheet.
- apple4ever 8y agoGreat article. Somebody else mentioned Scott Helme, but didn't link to three of his amazing sites: https://securityheaders.com https://securityheaders.com which checks important headers https://report-uri.com/ https://report-uri.com/ which allows sending CSP reports to to catch errors. It also has a CSP builder (among a bunch of tools) which is hugely helpful: https://report-uri.com/home/generate https://report-uri.com/home/generate https://scotthelme.co.uk/ https://scotthelme.co.uk/ is his blog with a ton of info. It also has a cheat sheet for CSP: https://scotthelme.co.uk/csp-cheat-sheet/ https://scotthelme.co.uk/csp-cheat-sheet/ (I might be a fan of the guy ha)
- awcode 8y ago"Browser support for CSP is good these days, but unfortunately, not many sites are using it....I think we can do better to make the web a safer place" Interestingly enough the blog this was posted on falls into the 94% not taking the effort to use CSP!
- deleted 8y ago[deleted]