16 ms·

The State of HTTP in 2022

271 points by harporoeder 4y ago

userbinator 4y ago
The state of proliferating sites that won't let you access them because you're not using an "approved" browser with JS and cookies enabled, and hiding behind the "security" excuse to do so. Ironic that I can't even read an article about HTTP in 2022 because of that racket. Thanks, Cloudflare.
- zdragnar 4y ago
  It isn't your security they're concerned about, it is the server's- it is preventing DDoS attacks. If you want to blame someone, blame the people who poisoned the well for everyone else.
  - maxbond 4y ago
    We can walk & chew gum here, spammers and DDoSers are poisoning the well for sure, but Cloudflare is responsible for externalities of their service.
    - deleted 4y ago
      [deleted]
    - zdragnar 4y ago
      Server hosts want protection from DDoS. They hire a bouncer (I e. Cloudflare) to keep the hordes at bay. The bouncer asks for some way to distinguish your suspicious looking request from that of the start of a DDoS attack. How is this an "externality" of cloudflare's service? If anything, it is an "externality" of the server hosts.
      - maxbond 4y ago
        Does it even matter? Blame is irrelevant, I find responsibility to be a more interesting thing to discuss in virtually all circumstances. The DDoSers and spammers should cut it out (but unfortunately we can't really count on them for anything), Cloudflare should make more of an effort to preserve accessibility, and their customers should care about it too & make it clear to Cloudflare that it's a priority to them. But if we take a step back it's clear that Cloudflare is the entity here that can have the biggest impact. Spammers and customers are diffuse, Cloudflare isn't. How much of the blame they deserve, I don't really care, but theres a problem, they're in the best position to act, so they have a responsibility to. In my opinion, that's the best mindset for solving problems.
        zdragnar 4y ago
        > Blame is irrelevant, I find responsibility to be a more interesting thing to discuss This is, to me, a bizarre notion. You're imposing some sort of moral edict upon cloudflare for providing an opt-in service to web hosts. > they're in the best position to act They have, and they decided that certain traffic requires additional consideration to distinguish from bad actors. Your refusal to cooperate is entirely on you. Just because you don't like their solution doesn't mean that there is a better one. If you have better ideas, I'm sure they're all ears, as less intrusive is generally cheaper to implement.
  - userbinator 4y ago
    it is preventing DDoS attacks. Is it? Those who want to DDoS will always find a way, and meanwhile users with slightly odd hardware/software are being locked out. Admittedly the latter is a minority, but one of the key tenets of the Internet and what made it so successful was interoperability. This is, in some ways, even worse than (but somewhat of a following effect of) the effective browser monopoly. Calling it "security" when it's really about "availability" is another deceptive misdirection, because the former is something that can more easily persuade the sheeple. I don't really like to go to extremes, but fingerprinting clients and deciding access based on such should really be regarded as a moral equivalent to racial profiling.
    - zdragnar 4y ago
      Profiling happens all the time for entirely legitimate reasons in the real world. Racial profiling is immoral not because it is profiling; it is immoral because it is based on immutable characteristics which have no intrinsic bearing on the purpose of the profile. To tie this back to the topic at hand, you're complaining that a service has decided your traffic resembles known patterns from bad actors, and is asking you to go through an extra step to access the content. Are there better options? Maybe, but it's utterly asinine to compare what cloudflare is doing to racial profiling.
    - jayd16 4y ago
      > Those who want to DDoS will always find a way How do you figure? You don't think far more DDoS events would occur if it was easier and more effective?
    - cortesoft 4y ago
      > Those who want to DDoS will always find a way Not really. We are actually pretty good these days at stopping DDOS attacks.
      - mysterydip 4y ago
        Not sure how accurate this is, but currently shows 150+ DDoS attacks going on globally: https://www.netscout.com/ddos-attack-map https://www.netscout.com/ddos-attack-map
  - deleted 4y ago
    [deleted]
  - worldofmatthew 4y ago
    DDOS attacks that exist because Cloudflare knowly protects booter websites which feeds the DDDOS economy with money to make DDOS botnets profitable?
- mnot 4y ago
  The IETF is working on a solution to this, and Cloudlare is actively contributing and implementing; see https://blog.cloudflare.com/eliminating-captchas-on-iphones-and-macs-using-new-standard/ https://blog.cloudflare.com/eliminating-captchas-on-iphones-...
  - dceddia 4y ago
    This plan looks even worse for privacy, and seems to rely on authenticating the unique device itself. > Know your user is coming from an authentic device and signed application, verified by the device vendor directly. If they manage to get wide adoption of something like this it seems like a very bad day for privacy, and a joyous day for advertisers, anyone who wants to prevent web scraping, Google and/or anyone who might want to make it hard to crawl the entire web…
    - mnot 4y ago
      Read a bit more; that’s not how the cryptography works. They’re blinded tokens.
      - dceddia 4y ago
        I read further, and it sounds like they've designed it with privacy in mind. That's nice. I'm still having a hard time seeing how this doesn't eventually lead to a completely locked-down internet, where users can only use approved browsers and devices. They have a list of steps for how a request would be made, where steps 2 and 3 are: > 2. Safari supports PATs, so it will make an API call to Apple’s Attester, asking them to attest. > 3. The Apple attester will check various device components, confirm they are valid, and then make an API call to the Cloudflare Issuer (since Cloudflare acting as an Origin chooses to use the Cloudflare Issuer). In a theoretical future world where 99% of site operators have set this up for protection, and 99% of users are using approved browsers, how would one do something like... Create a competitor to Google? You'd need to crawl the web for that. Would you imagine Apple or Cloudflare would gladly let your device request millions of tokens per hour? Or would that be throttled or disallowed entirely? Use curl (or telnet, or [any other HTTP client]) to grab a page? Use yt-dlp to download a YouTube video? Scrape a bunch of data for an AI project? See this article from the front page where someone scraped a bunch of car listings from KBB and trained a model to estimate car prices and found some interesting results. https://blog.aqnichol.com/2022/12/31/large-scale-vehicle-classification/ https://blog.aqnichol.com/2022/12/31/large-scale-vehicle-cla... - would something like that be permitted under a system like this? Or might you need to own/rent an army of authorized devices with authorized browsers to do that experiment?
- ddon 4y ago
  May be this link will work for you? https://archive.ph/1gcnQ https://archive.ph/1gcnQ
  - userbinator 4y ago
    Thanks for trying, but unfortunately archive.ph is also behind Cloudflare.
- anenefan 4y ago
  Funny enough I can actually view their article. Despite that I can visit websites where admins that set permissions not overly tough, I still run into enough blocks due to cloudflare that I'm considering investing time (ok for the last few years I've been really really lazy, I'm happy enough to copy and paste so I can manually write a nasty comment ) so that for each "bad load" the web site can be added to my disallow list. It might only save the smallest amount of wasted bandwidth but I guess it all adds up.
- deleted 4y ago
  [deleted]
- 1vuio0pswjnm7 4y ago
  The "new" HTTP is clearly targeted at the "approved browser". For example, this alleged "head-of-line blocking problem" that HTTP/2 purportedly "solves" was never a problem of HTTP outside of a specific program, the graphical web browser, the type of client that tries to pull resources from different domains for a single website. Not all programs that use HTTP need to do that. For instance I have been using HTTP/1.1 pipelining outside the browser for fast, reliable information retrieval for close to 20 years. It has always been supported by HTTP servers and it works great with the simple clients I use. I still rely on HTTP/1.1 pipelining today, on a daily basis. Never had a problem. There are uses for pipelining besides the ones envisioned by "tech" companies, web developers and their advertiser customers.
  - mnot 4y ago
    If early hints breaks your proxy, it’s likely your proxy doesn’t handle 1xx status codes correctly. Could you tell me which proxy it is (privately if you think it necessary)? I’d like to chase the bug with them.
  - raggi 4y ago
    The big problem with pipelining in http 1.x is that a response can break the pipeline part way through and there is no way to know what the server processed. A response night for example, mid pipeline be connection:close and that’s that, did any subsequent request get processed? Who knows.
  - 1vuio0pswjnm7 4y ago
    HTTP/2 seems to be designed for (graphical, interactive) webpages. The maintainer of a popular webserver has suggested HTTP/2 is slower than HTTP/1.1 for file download. https://stackoverflow.com/questions/44019565/http-2-file-download https://stackoverflow.com/questions/44019565/http-2-file-dow... As I stated, I use HTTP/1.1 pipelining every day. I use it for a variety of information retrieval tasks, even retrieving bulk DNS data. To give an arbitrary example, sometimes I will download a website's sitemaps. This usually involves downloading a cascade of XML files. For example, there might be a main XML file called "index.xml". This file then lists hundreds more sitemap XML files, e.g., archive-2002-1.xml, archive-2002-2.xml, containing every content URL on the website beginning with some prior year all the way up to the present day. Using a real world example, index.xml contains 246 URLs. Using HTTP/1.1 pipelining I can retrieve all of them into a single file using a single TCP connection. Then I retrieve batches of the URLs contained in that file, again over a single TCP connection. Many websites allow thousands of HTTP requests HTTP/1.1-pipelined over a TCP single connection, but I usually keep the batch size at around 500-1000 max. Of course I want the responses in the same order as the requests. The process looks something like this ftp -4o 1 https://[domainname]/sitemaps/index.xml yy030 < 1|(ka;nc0) > 2 yy030 < 2|wc -l 1337855 1337855 is the number of URLs for [domainname]. Content URLs, not Javascript, CSS or other garbage. yy030 is a C program that filters URLs from standard input ka is a shell alias that sets an environment variable that is read by the yy025 program to indicate an HTTP header, in this case the "Connection:" header set to "keep-alive" not "close" (ka- sets it back to close) nc0 is a one line shell script yy025|nc -vv h1b 80|yy045 yy025 is a C program that accepts URLs, e.g., dozens to hundreds to thousands of URLs, on stdin and outputs customised HTTP h1b is a HOSTS file entry containg the address of a localhost-bound forward TLS proxy yy045 is a C program that removes chunked transfer encoding from standard input To verify the download, I can look at the HTTP headers in file "2". I can also look at the log from the TLS proxy. I have it set configured to log all HTTP requests and responses. Is this a job for HTTP/2. It does not seem like it. This type of pipelining using only a single TCP connection is not possible using curl or libcurl. Nor is it possible using nghttp. Look around the web and one will see people opening up dozens, maybe hundreds of TCP connections and running jobs in parallel, trying to improve speed, and often getting banned. As with the comment from the Jetty maintainer, I suspect using HTTP/2 would actually be slower for this type of transfer. It is overkill. IMHO, HTTP, i.e., in the general sense, is not just for requesting webpages and resources for webpages. I find HTTP/1.1 to be very useful. It is certainly not just for requesting webpages full of JS, CSS, images and the like. That is only one way I might use it. Perhaps HTTP/2 is the better choice for webpages. TBH, if using a "modern" graphical browser, I would be inclined to let it use HTTP/2. Most of the time I am not using a graphical browser.
- svnpenn 4y ago
  Yep. Just try to cURL the page, and you get 403 Forbidden. That is completely unacceptable.
secondcoming 4y ago
HTTP is an unmitigated mess. It's in the same state as C++; tries to be everything but has lots of historical baggage that can't/won't be removed.
- travisgriggs 4y ago
  Similar thoughts. As I read through the “why HTTP took off” explanations, I was surprised that the more obvious “it was the first thing that a plurality of players networked on and after that it was all just bandwagoning, because it’s much easier to meet people at other people’s parties than try to host your own” wasn’t enumerated.
  - Simran-B 4y ago
    "Ease of use" is mentioned, and I suppose this refers to HTTP being text-based, pretty readable and intuitively understandable. HTTP/2 and later aren't like this anymore, for the sake of efficiency. It's a trade-off, but at this point I don't see much value in using it for APIs anymore. If you value efficiency and clean design, there are clearly better alternatives that aren't more complicated than what HTTP has become.
    - shukantpal 4y ago
      I'm curious, what other protocols do you have in mind?
      - nly 4y ago
        Grpc + protobufs?
        RedShift1 4y ago
        Lol, GRPC uses HTTP as its transport :'). Try again.
      - Simran-B 4y ago
benbjohnson 4y ago
One implementation detail about QUIC that I was surprised by was that it requires TLS. That’s great for improving the security on the public Internet but it seems like it adds complexity and CPU overhead if you’re running on something like an internal Wireguard network. Overall, though, it’s a minor complaint. I did like how they split apart the QUIC and HTTP/3 protocol from one another.
- mnot 4y ago
  The precedent for that was set by http/2; while the spec didn’t require it (Snowden happened after that decision was made), all of the inplementatuons did.
  - devmunchies 4y ago
    I believe GP is talking about the lower level QUIC protocol. That’s like TCP requiring TLS. A tradeoff would have been QUIC not requiring it but the higher level http/3 protocol requiring (if that’s possible idk)
    - tialaramex 4y ago
      TLS delivers three things, Confidentiality, Integrity and Authentication. But intuitively these three things are what people expect their transport protocol to do anyway, it's actually surprising as a user that TCP doesn't really bother providing say, Integrity. "Oh, yeah, your data might be arbitrarily changed by the time it is delivered". Er, what? BCP 78 "Pervasive Monitoring is an Attack" says the Internet should design new systems to resist such monitoring and so offering these intuitive properties for a new transport protocol made sense. And that's what QUIC is. In a BCP 78 universe it doesn't make sense to ship new protocols that will be rendered useless by the surveillance apparatus.
      - asdfghjhgfderty 4y ago
        it's insane to expect the opposite: that traffic in my own network will need to keep reaching to a certificate authorities outside to validate packages from one host to another. if you don't understand why these 3 things are on top of tcp, well, nevermind, I was going to say you shouldn't be designing networks buy you migth be already on the quic steering committee. most committee now are a joke so that some googler middle mamaget makes it to jr director. sigh.
deleted 4y ago
[deleted]
no_wizard 4y ago
I thought the most interesting bit was the privacy OHTTP which they’re building a service around[0]. How this will differ from a VPN will be interesting. The gist of it is that the http connections are naive and don’t really record an “accurate” up address or trace, if I understand correctly [0]: https://blog.cloudflare.com/building-privacy-into-internet-standards-and-how-to-make-your-app-more-private-today/ https://blog.cloudflare.com/building-privacy-into-internet-s...
- intelVISA 4y ago
  Maybe I misunderstand but this just looks like a buzzword mix of proxy + QUIC (i.e. TLS wrapped HTTP). Seems kinda pointless given that most of the internet is already behind a Cloudflare MITM anyway.
  - mnot 4y ago
    The idea is to compose intermediaries run by different parties, so that no single entity has access to both the unencrypted payload and client ip address / connection context. It’s a controlled form of proxying where the proxy doesn’t have access to the plaintext. Not currently designed for everyday browsing; the use cases are things like collecting telemetry.
    - nly 4y ago
      So....Tor
      - OverclockX64 4y ago
        Well, Tor is really slow to be honest, I think that’s where OHTTP will shine.
      - csande17 4y ago
        CloudFlare's solution is significantly less secure than Tor; it's more like visiting an HTTPS website using a VPN service. You have to trust that the VPN and the website operator aren't colluding to de-anonymize you, and that they aren't both being monitored by the same third party (who can de-anonymize you using timing information).
comfypotato 4y ago
The link to the new query method [1] intrigued me. Could this, if widely adopted, make GraphQL obsolete? (Or am I admittedly ignorant as to exactly what they each do?) [1] https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-method-w-body.html https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-m...
- kindaAnIdiot 4y ago
  IIRC query is just a method that you'd use for your GraphQL route. It doesn't actually have any querying system defined in the spec (See final sentence here https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-method-w-body.html#name-examples https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-m...)
- mnot 4y ago
  If everything works out, grapnel could use it and queries would be cacheable by generic http caches. A lot would need to happen to make that work, though. The immediate motivation is to get an alternative to people who want to put a request body on GET.
  - mnot 4y ago
    Graphql. I love autocorrect.
    - vram22 4y ago
      A sailor must have invented it.
- Simran-B 4y ago
  The main focus of HTTP 2 and 3 was apparently performance, and hardly any of the other fundamental issues of the protocol. It's nice to see that some work is put into something like QUERY as well, which improves HTTP as an API protocol (REST) - although the motivation appears to be performance again and I always have mixed feelings when I see the term "idempotent" in a context that likely involves database systems. Too often, such requests do have side effects of some kind and preventing request retries throughout the stack isn't easy.
cpsns 4y ago
Edit: Not worth commenting on this when I'm already getting accused of bad things. Sorry folks, but apparently wanting companies like CF regulated is too controversial.
- aqeelat 4y ago
  Can you elaborate?
  - bitwize 4y ago
    Kiwi Farms. This is about Kiwi Farms. Someone's a bit assmad that the internet increasingly filters hate.
    - jimmygrapes 4y ago
      You, too, will be assmad when the cultural zeitgeist decides that what you think is tolerance/jest becomes hate at some point.
      - bitwize 4y ago
        There is no confusing the targeted harassment of trans people undertaken by Kiwi Farms with tolerance or jest. At least one brilliant, talented person has died because of Kiwi Farms.
        resdef 4y ago
        Not true. What Kiwi Farms has done though is help expose grooming of teenagers. Some of these people like to believe they're untouchable and their misdeeds can be hidden. They're not.
    - mardifoufs 4y ago
      I think you are projecting, they never mentioned KF. Cloudflare filters much more than just KF, try accessing any CF backed website from Tor. I guess you don't seem to care as long as it also filters the stuff you disagree with, though, so net neutrality is already out of the question for you lol.
shaggie76 4y ago
As someone who's had to deal with many DDoS attacks I'm rather horrified at the thought of QUIC: dropping UDP at the network border eliminates a lot of headaches.
- userbinator 4y ago
  It's ironic to see that coming from a company which profits from DDoS prevention services.
  - shaggie76 4y ago
    Sorry to mislead you; I don't work for a protection service but my company has had contracts with a handful of them and I've tested a handful more.
    - userbinator 4y ago
      I'm referring to Cloudflare.
- deleted 4y ago
  [deleted]
- 10000truths 4y ago
  I'm not sure I follow. Such attacks happen because of poorly designed protocols with very asymmetric resource requirements between endpoints. My understanding is that the QUIC handshake has measures in place to prevent it from being a vector for amplification/reflection based denial-of-service attacks.
  - masklinn 4y ago
    > I'm not sure I follow. They're saying that blocking UDP entirely is a very effective (and cheap) mitigation of DDOS. Because QUIC is built on UDP, however, you can't just block it, you have to ingest it and dedicate resources to filtering between QUIC and other UDP traffic.
_eht 4y ago
My resolutions for 2023 include helping people not use Cloudflare.
- CuteDepravity 4y ago
  What's wrong with CloudFlare? I recently discovered their free zero trust http proxy and really love it
  - Gigachad 4y ago
    They are a danger to the free and open internet.
    - int_19h 4y ago
      They protect a lot of "objectionable" websites from DDoS, though.
  - _eht 4y ago
    [flagged]
    - i2shar 4y ago
      That's not helpful. You said you wanted to help :)
      - _eht 4y ago
        Instilling doubt or question in something is often more practical in effecting change than immediately announcing every facet of your argument. Sometimes it’s more practical to coerce change by presenting an opportunity to question an assumed assumption. Maybe someone will see my comment and start down their own rabbit hole to find a conclusion. That is more ideal than immediately assuming the details of my personal assumptions and conclusions. :)
        4y ago
superkuh 4y ago
HTTP/1.1 support is the last bastion between the web and complete corporate control. Once the megacorp browsers and man in the middle companies like Cloudflare drop HTTP/1.1 we will no longer be able to host a website without the continued approval of a third party corporation. HTTP/2 and HTTP/3 implemenations require the use of CA based TLS. Just to preempt misunderstanding: HTTPS is great. But HTTPS only, with no option for HTTP is very much worse than HTTP+HTTPS for human people. Despite being great for for profit companies and institutions.
- MuffinFlavored 4y ago
  > HTTP/2 and HTTP/3 implemenations require the use of CA based TLS. You can't do LetsEncrypt?
  - acdha 4y ago
    You can. You can also continue to use private CAs, which presumably could be used to add more options if something happens to Let’s Encrypt.
  - PenguinCoder 4y ago
    Not for LAN only personal websites.
    - sli 4y ago
      Why not just self-sign for that? Outside of using it to test configurations or deployments, SSL seems a lot less necessary if it's just inside a LAN and all the clients are known.
      - PenguinCoder 4y ago
        Some services don't work without HTTPS or transport security. Sometimes, I want transport security on traffic for various reasons. Current mainstream browsers are also refusing to connect to 'self signed certificate' HTTPS sites because they're 'insecure' and continually are disallowing you the user to bypass these 'protections'.
est31 4y ago
I'm still not over the fact that they made headers all-lowercase in HTTP/2. I know the reasons, but it's so weird to have all-lowercase headers. TBH I don't see much of an uptake in the community either: since HTTP/2 came out, I've barely seen lowercase headers be proliferated in documentation of headers, e.g. MDN lists them HTTP/1 style: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
- treve 4y ago
  Anything working with headers should always have been case insensitive, so there's no reason to update the any documentation.
  - opportune 4y ago
    As someone who works in infrastructure I disagree with this. While it’s true that applications should always have done this, many many do not and need to be dealt with. If they are third party paying customers, your only option is to figure out how to make things work, because asking customers to upgrade will piss many of them off. The simple fact so many applications do not treat headers as case insensitive is one of the major things holding back http 2. With HTTP/1 it was suggested but not enforced. Upgrading to http/2 with forced-lowecasing breaks these applications
    - treve 4y ago
      What part are you disagreeing with?
- avx56 4y ago
  Someone should write a DevTools extension that uppercases the headers, if such a thing is possible. :)
  - codetrotter 4y ago
    If you want to break protocol, I suggest that we also change all of the HTTP verbs for no reason whatsoever :^) GET -> FETCH HEAD -> EXAMINE POST -> SUBMIT PUT -> REPLACE DELETE -> REMOVE CONNECT -> PROXY OPTIONS -> ALTERNATIVES TRACE -> ECHO PATCH -> PARTIALUPDATE
NickLamp 4y ago
The new QUERY method strikes me as a really promising addition. Not being able to send a body with a GET-type request is a gnawing issue I have with HTTP
- 0xb0565e487 4y ago
  Just send a POST request lol
  - HideousKojima 4y ago
    But you're only "supposed" to use POST with requests that add/modify data, or something silly like that. In practice, QUERY is most useful for where you want a bunch of different verbs for the same endpoint and need a body.
  - spiffytech 4y ago
    QUERY is cacheable.
    - est 4y ago
      ... in theory only. Not that many http cache programs support QUERY. And many HTTP middle-boxes bans non GET/POST verbs.
      - counttheforks 4y ago
        For now. Change has to come from somewhere.
    - cogman10 4y ago
      And retry-able.
  - duxup 4y ago
    In practice I think that’s exactly what many people do.
  - Existenceblinks 4y ago
Existenceblinks 4y ago
I would love a queryable file format akin to sqlite but with simple spec tailor made for the QUERY method.
high_byte 4y ago
HTTP is such a mess. only a new protocol can save us, but at this point this is near impossible.
pdimitar 4y ago
What's this "HTTP core"? Please tell me it's just the sane parts of HTTP and without any dark corners and ambiguous specs? Please tell me it's something you can write a parser for in an hour? We're no less 10 years overdue for something like this. And if it's not that... shame.
jimjag 4y ago
IMO, the reason for the initial success of HTTP (1.x) was due to its extreme simplicity. Things like text-based protocol, straightforward stateless design, capability to implement both server and client in easy, basic code. All this meant that the protocol itself was stable, usable, and a reliable standard to use. The current path is to drastically increase complexity due to the demands of the content provider overlord(s); Basically, in order to better accommodate the needs of Google (and a handful of others), we must redefine things for everyone. It's becoming a complex, over-designed protocol that is being crammed down people's throats, instead of a protocol that is embraced because it makes sense.
- afiori 4y ago
  HTTP 2 allowed for in-stream multiplexing
- rubatuga 4y ago
  It's interesting that cloudflare states on their own blog that HTTP/3 has no benefits in real-world usage: https://blog.cloudflare.com/http-3-vs-http-2/ https://blog.cloudflare.com/http-3-vs-http-2/
Alifatisk 4y ago
When can I expect, let’s say Rack & Rails to implement HTTP/3?

271 points by harporoeder 4y ago

The state of proliferating sites that won't let you access them because you're not using an "approved" browser with JS and cookies enabled, and hiding behind the "security" excuse to do so. Ironic that I can't even read an article about HTTP in 2022 because of that racket. Thanks, Cloudflare.

zdragnar 4y ago
It isn't your security they're concerned about, it is the server's- it is preventing DDoS attacks. If you want to blame someone, blame the people who poisoned the well for everyone else.
- maxbond 4y ago
  We can walk & chew gum here, spammers and DDoSers are poisoning the well for sure, but Cloudflare is responsible for externalities of their service.
  - deleted 4y ago
    [deleted]
  - zdragnar 4y ago
    Server hosts want protection from DDoS. They hire a bouncer (I e. Cloudflare) to keep the hordes at bay. The bouncer asks for some way to distinguish your suspicious looking request from that of the start of a DDoS attack. How is this an "externality" of cloudflare's service? If anything, it is an "externality" of the server hosts.
    - maxbond 4y ago
      Does it even matter? Blame is irrelevant, I find responsibility to be a more interesting thing to discuss in virtually all circumstances. The DDoSers and spammers should cut it out (but unfortunately we can't really count on them for anything), Cloudflare should make more of an effort to preserve accessibility, and their customers should care about it too & make it clear to Cloudflare that it's a priority to them. But if we take a step back it's clear that Cloudflare is the entity here that can have the biggest impact. Spammers and customers are diffuse, Cloudflare isn't. How much of the blame they deserve, I don't really care, but theres a problem, they're in the best position to act, so they have a responsibility to. In my opinion, that's the best mindset for solving problems.
      - zdragnar 4y ago
        > Blame is irrelevant, I find responsibility to be a more interesting thing to discuss This is, to me, a bizarre notion. You're imposing some sort of moral edict upon cloudflare for providing an opt-in service to web hosts. > they're in the best position to act They have, and they decided that certain traffic requires additional consideration to distinguish from bad actors. Your refusal to cooperate is entirely on you. Just because you don't like their solution doesn't mean that there is a better one. If you have better ideas, I'm sure they're all ears, as less intrusive is generally cheaper to implement.
- userbinator 4y ago
  it is preventing DDoS attacks. Is it? Those who want to DDoS will always find a way, and meanwhile users with slightly odd hardware/software are being locked out. Admittedly the latter is a minority, but one of the key tenets of the Internet and what made it so successful was interoperability. This is, in some ways, even worse than (but somewhat of a following effect of) the effective browser monopoly. Calling it "security" when it's really about "availability" is another deceptive misdirection, because the former is something that can more easily persuade the sheeple. I don't really like to go to extremes, but fingerprinting clients and deciding access based on such should really be regarded as a moral equivalent to racial profiling.
  - zdragnar 4y ago
    Profiling happens all the time for entirely legitimate reasons in the real world. Racial profiling is immoral not because it is profiling; it is immoral because it is based on immutable characteristics which have no intrinsic bearing on the purpose of the profile. To tie this back to the topic at hand, you're complaining that a service has decided your traffic resembles known patterns from bad actors, and is asking you to go through an extra step to access the content. Are there better options? Maybe, but it's utterly asinine to compare what cloudflare is doing to racial profiling.
  - jayd16 4y ago
    > Those who want to DDoS will always find a way How do you figure? You don't think far more DDoS events would occur if it was easier and more effective?
  - cortesoft 4y ago
    > Those who want to DDoS will always find a way Not really. We are actually pretty good these days at stopping DDOS attacks.
    - mysterydip 4y ago
      Not sure how accurate this is, but currently shows 150+ DDoS attacks going on globally: https://www.netscout.com/ddos-attack-map https://www.netscout.com/ddos-attack-map
- deleted 4y ago
  [deleted]
- worldofmatthew 4y ago
  DDOS attacks that exist because Cloudflare knowly protects booter websites which feeds the DDDOS economy with money to make DDOS botnets profitable?
mnot 4y ago
The IETF is working on a solution to this, and Cloudlare is actively contributing and implementing; see https://blog.cloudflare.com/eliminating-captchas-on-iphones-and-macs-using-new-standard/ https://blog.cloudflare.com/eliminating-captchas-on-iphones-...
- dceddia 4y ago
  This plan looks even worse for privacy, and seems to rely on authenticating the unique device itself. > Know your user is coming from an authentic device and signed application, verified by the device vendor directly. If they manage to get wide adoption of something like this it seems like a very bad day for privacy, and a joyous day for advertisers, anyone who wants to prevent web scraping, Google and/or anyone who might want to make it hard to crawl the entire web…
  - mnot 4y ago
    Read a bit more; that’s not how the cryptography works. They’re blinded tokens.
    - dceddia 4y ago
      I read further, and it sounds like they've designed it with privacy in mind. That's nice. I'm still having a hard time seeing how this doesn't eventually lead to a completely locked-down internet, where users can only use approved browsers and devices. They have a list of steps for how a request would be made, where steps 2 and 3 are: > 2. Safari supports PATs, so it will make an API call to Apple’s Attester, asking them to attest. > 3. The Apple attester will check various device components, confirm they are valid, and then make an API call to the Cloudflare Issuer (since Cloudflare acting as an Origin chooses to use the Cloudflare Issuer). In a theoretical future world where 99% of site operators have set this up for protection, and 99% of users are using approved browsers, how would one do something like... Create a competitor to Google? You'd need to crawl the web for that. Would you imagine Apple or Cloudflare would gladly let your device request millions of tokens per hour? Or would that be throttled or disallowed entirely? Use curl (or telnet, or [any other HTTP client]) to grab a page? Use yt-dlp to download a YouTube video? Scrape a bunch of data for an AI project? See this article from the front page where someone scraped a bunch of car listings from KBB and trained a model to estimate car prices and found some interesting results. https://blog.aqnichol.com/2022/12/31/large-scale-vehicle-classification/ https://blog.aqnichol.com/2022/12/31/large-scale-vehicle-cla... - would something like that be permitted under a system like this? Or might you need to own/rent an army of authorized devices with authorized browsers to do that experiment?
ddon 4y ago
May be this link will work for you? https://archive.ph/1gcnQ https://archive.ph/1gcnQ
- userbinator 4y ago
  Thanks for trying, but unfortunately archive.ph is also behind Cloudflare.
anenefan 4y ago
Funny enough I can actually view their article. Despite that I can visit websites where admins that set permissions not overly tough, I still run into enough blocks due to cloudflare that I'm considering investing time (ok for the last few years I've been really really lazy, I'm happy enough to copy and paste so I can manually write a nasty comment ) so that for each "bad load" the web site can be added to my disallow list. It might only save the smallest amount of wasted bandwidth but I guess it all adds up.
deleted 4y ago
[deleted]
1vuio0pswjnm7 4y ago
The "new" HTTP is clearly targeted at the "approved browser". For example, this alleged "head-of-line blocking problem" that HTTP/2 purportedly "solves" was never a problem of HTTP outside of a specific program, the graphical web browser, the type of client that tries to pull resources from different domains for a single website. Not all programs that use HTTP need to do that. For instance I have been using HTTP/1.1 pipelining outside the browser for fast, reliable information retrieval for close to 20 years. It has always been supported by HTTP servers and it works great with the simple clients I use. I still rely on HTTP/1.1 pipelining today, on a daily basis. Never had a problem. There are uses for pipelining besides the ones envisioned by "tech" companies, web developers and their advertiser customers.
- mnot 4y ago
  If early hints breaks your proxy, it’s likely your proxy doesn’t handle 1xx status codes correctly. Could you tell me which proxy it is (privately if you think it necessary)? I’d like to chase the bug with them.
- raggi 4y ago
  The big problem with pipelining in http 1.x is that a response can break the pipeline part way through and there is no way to know what the server processed. A response night for example, mid pipeline be connection:close and that’s that, did any subsequent request get processed? Who knows.
- 1vuio0pswjnm7 4y ago
  HTTP/2 seems to be designed for (graphical, interactive) webpages. The maintainer of a popular webserver has suggested HTTP/2 is slower than HTTP/1.1 for file download. https://stackoverflow.com/questions/44019565/http-2-file-download https://stackoverflow.com/questions/44019565/http-2-file-dow... As I stated, I use HTTP/1.1 pipelining every day. I use it for a variety of information retrieval tasks, even retrieving bulk DNS data. To give an arbitrary example, sometimes I will download a website's sitemaps. This usually involves downloading a cascade of XML files. For example, there might be a main XML file called "index.xml". This file then lists hundreds more sitemap XML files, e.g., archive-2002-1.xml, archive-2002-2.xml, containing every content URL on the website beginning with some prior year all the way up to the present day. Using a real world example, index.xml contains 246 URLs. Using HTTP/1.1 pipelining I can retrieve all of them into a single file using a single TCP connection. Then I retrieve batches of the URLs contained in that file, again over a single TCP connection. Many websites allow thousands of HTTP requests HTTP/1.1-pipelined over a TCP single connection, but I usually keep the batch size at around 500-1000 max. Of course I want the responses in the same order as the requests. The process looks something like this ftp -4o 1 https://[domainname]/sitemaps/index.xml yy030 < 1|(ka;nc0) > 2 yy030 < 2|wc -l 1337855 1337855 is the number of URLs for [domainname]. Content URLs, not Javascript, CSS or other garbage. yy030 is a C program that filters URLs from standard input ka is a shell alias that sets an environment variable that is read by the yy025 program to indicate an HTTP header, in this case the "Connection:" header set to "keep-alive" not "close" (ka- sets it back to close) nc0 is a one line shell script yy025|nc -vv h1b 80|yy045 yy025 is a C program that accepts URLs, e.g., dozens to hundreds to thousands of URLs, on stdin and outputs customised HTTP h1b is a HOSTS file entry containg the address of a localhost-bound forward TLS proxy yy045 is a C program that removes chunked transfer encoding from standard input To verify the download, I can look at the HTTP headers in file "2". I can also look at the log from the TLS proxy. I have it set configured to log all HTTP requests and responses. Is this a job for HTTP/2. It does not seem like it. This type of pipelining using only a single TCP connection is not possible using curl or libcurl. Nor is it possible using nghttp. Look around the web and one will see people opening up dozens, maybe hundreds of TCP connections and running jobs in parallel, trying to improve speed, and often getting banned. As with the comment from the Jetty maintainer, I suspect using HTTP/2 would actually be slower for this type of transfer. It is overkill. IMHO, HTTP, i.e., in the general sense, is not just for requesting webpages and resources for webpages. I find HTTP/1.1 to be very useful. It is certainly not just for requesting webpages full of JS, CSS, images and the like. That is only one way I might use it. Perhaps HTTP/2 is the better choice for webpages. TBH, if using a "modern" graphical browser, I would be inclined to let it use HTTP/2. Most of the time I am not using a graphical browser.
svnpenn 4y ago
Yep. Just try to cURL the page, and you get 403 Forbidden. That is completely unacceptable.

secondcoming 4y ago

HTTP is an unmitigated mess. It's in the same state as C++; tries to be everything but has lots of historical baggage that can't/won't be removed.

travisgriggs 4y ago
Similar thoughts. As I read through the “why HTTP took off” explanations, I was surprised that the more obvious “it was the first thing that a plurality of players networked on and after that it was all just bandwagoning, because it’s much easier to meet people at other people’s parties than try to host your own” wasn’t enumerated.
- Simran-B 4y ago
  "Ease of use" is mentioned, and I suppose this refers to HTTP being text-based, pretty readable and intuitively understandable. HTTP/2 and later aren't like this anymore, for the sake of efficiency. It's a trade-off, but at this point I don't see much value in using it for APIs anymore. If you value efficiency and clean design, there are clearly better alternatives that aren't more complicated than what HTTP has become.
  - shukantpal 4y ago
    I'm curious, what other protocols do you have in mind?
    - nly 4y ago
      Grpc + protobufs?
      - RedShift1 4y ago
        Lol, GRPC uses HTTP as its transport :'). Try again.
    - Simran-B 4y ago

benbjohnson 4y ago

One implementation detail about QUIC that I was surprised by was that it requires TLS. That’s great for improving the security on the public Internet but it seems like it adds complexity and CPU overhead if you’re running on something like an internal Wireguard network. Overall, though, it’s a minor complaint. I did like how they split apart the QUIC and HTTP/3 protocol from one another.

mnot 4y ago
The precedent for that was set by http/2; while the spec didn’t require it (Snowden happened after that decision was made), all of the inplementatuons did.
- devmunchies 4y ago
  I believe GP is talking about the lower level QUIC protocol. That’s like TCP requiring TLS. A tradeoff would have been QUIC not requiring it but the higher level http/3 protocol requiring (if that’s possible idk)
  - tialaramex 4y ago
    TLS delivers three things, Confidentiality, Integrity and Authentication. But intuitively these three things are what people expect their transport protocol to do anyway, it's actually surprising as a user that TCP doesn't really bother providing say, Integrity. "Oh, yeah, your data might be arbitrarily changed by the time it is delivered". Er, what? BCP 78 "Pervasive Monitoring is an Attack" says the Internet should design new systems to resist such monitoring and so offering these intuitive properties for a new transport protocol made sense. And that's what QUIC is. In a BCP 78 universe it doesn't make sense to ship new protocols that will be rendered useless by the surveillance apparatus.
    - asdfghjhgfderty 4y ago
      it's insane to expect the opposite: that traffic in my own network will need to keep reaching to a certificate authorities outside to validate packages from one host to another. if you don't understand why these 3 things are on top of tcp, well, nevermind, I was going to say you shouldn't be designing networks buy you migth be already on the quic steering committee. most committee now are a joke so that some googler middle mamaget makes it to jr director. sigh.

deleted 4y ago

[deleted]

no_wizard 4y ago

I thought the most interesting bit was the privacy OHTTP which they’re building a service around[0]. How this will differ from a VPN will be interesting. The gist of it is that the http connections are naive and don’t really record an “accurate” up address or trace, if I understand correctly [0]: https://blog.cloudflare.com/building-privacy-into-internet-standards-and-how-to-make-your-app-more-private-today/ https://blog.cloudflare.com/building-privacy-into-internet-s...

intelVISA 4y ago
Maybe I misunderstand but this just looks like a buzzword mix of proxy + QUIC (i.e. TLS wrapped HTTP). Seems kinda pointless given that most of the internet is already behind a Cloudflare MITM anyway.
- mnot 4y ago
  The idea is to compose intermediaries run by different parties, so that no single entity has access to both the unencrypted payload and client ip address / connection context. It’s a controlled form of proxying where the proxy doesn’t have access to the plaintext. Not currently designed for everyday browsing; the use cases are things like collecting telemetry.
  - nly 4y ago
    So....Tor
    - OverclockX64 4y ago
      Well, Tor is really slow to be honest, I think that’s where OHTTP will shine.
    - csande17 4y ago
      CloudFlare's solution is significantly less secure than Tor; it's more like visiting an HTTPS website using a VPN service. You have to trust that the VPN and the website operator aren't colluding to de-anonymize you, and that they aren't both being monitored by the same third party (who can de-anonymize you using timing information).

comfypotato 4y ago

The link to the new query method [1] intrigued me. Could this, if widely adopted, make GraphQL obsolete? (Or am I admittedly ignorant as to exactly what they each do?) [1] https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-method-w-body.html https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-m...

kindaAnIdiot 4y ago
IIRC query is just a method that you'd use for your GraphQL route. It doesn't actually have any querying system defined in the spec (See final sentence here https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-method-w-body.html#name-examples https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-m...)
mnot 4y ago
If everything works out, grapnel could use it and queries would be cacheable by generic http caches. A lot would need to happen to make that work, though. The immediate motivation is to get an alternative to people who want to put a request body on GET.
- mnot 4y ago
  Graphql. I love autocorrect.
  - vram22 4y ago
    A sailor must have invented it.
Simran-B 4y ago
The main focus of HTTP 2 and 3 was apparently performance, and hardly any of the other fundamental issues of the protocol. It's nice to see that some work is put into something like QUERY as well, which improves HTTP as an API protocol (REST) - although the motivation appears to be performance again and I always have mixed feelings when I see the term "idempotent" in a context that likely involves database systems. Too often, such requests do have side effects of some kind and preventing request retries throughout the stack isn't easy.

cpsns 4y ago

Edit: Not worth commenting on this when I'm already getting accused of bad things. Sorry folks, but apparently wanting companies like CF regulated is too controversial.

aqeelat 4y ago
Can you elaborate?
- bitwize 4y ago
  Kiwi Farms. This is about Kiwi Farms. Someone's a bit assmad that the internet increasingly filters hate.
  - jimmygrapes 4y ago
    You, too, will be assmad when the cultural zeitgeist decides that what you think is tolerance/jest becomes hate at some point.
    - bitwize 4y ago
      There is no confusing the targeted harassment of trans people undertaken by Kiwi Farms with tolerance or jest. At least one brilliant, talented person has died because of Kiwi Farms.
      - resdef 4y ago
        Not true. What Kiwi Farms has done though is help expose grooming of teenagers. Some of these people like to believe they're untouchable and their misdeeds can be hidden. They're not.
  - mardifoufs 4y ago
    I think you are projecting, they never mentioned KF. Cloudflare filters much more than just KF, try accessing any CF backed website from Tor. I guess you don't seem to care as long as it also filters the stuff you disagree with, though, so net neutrality is already out of the question for you lol.

shaggie76 4y ago

As someone who's had to deal with many DDoS attacks I'm rather horrified at the thought of QUIC: dropping UDP at the network border eliminates a lot of headaches.

userbinator 4y ago
It's ironic to see that coming from a company which profits from DDoS prevention services.
- shaggie76 4y ago
  Sorry to mislead you; I don't work for a protection service but my company has had contracts with a handful of them and I've tested a handful more.
  - userbinator 4y ago
    I'm referring to Cloudflare.
deleted 4y ago
[deleted]
10000truths 4y ago
I'm not sure I follow. Such attacks happen because of poorly designed protocols with very asymmetric resource requirements between endpoints. My understanding is that the QUIC handshake has measures in place to prevent it from being a vector for amplification/reflection based denial-of-service attacks.
- masklinn 4y ago
  > I'm not sure I follow. They're saying that blocking UDP entirely is a very effective (and cheap) mitigation of DDOS. Because QUIC is built on UDP, however, you can't just block it, you have to ingest it and dedicate resources to filtering between QUIC and other UDP traffic.

_eht 4y ago

My resolutions for 2023 include helping people not use Cloudflare.

CuteDepravity 4y ago
What's wrong with CloudFlare? I recently discovered their free zero trust http proxy and really love it
- Gigachad 4y ago
  They are a danger to the free and open internet.
  - int_19h 4y ago
    They protect a lot of "objectionable" websites from DDoS, though.
- _eht 4y ago
  [flagged]
  - i2shar 4y ago
    That's not helpful. You said you wanted to help :)
    - _eht 4y ago
      Instilling doubt or question in something is often more practical in effecting change than immediately announcing every facet of your argument. Sometimes it’s more practical to coerce change by presenting an opportunity to question an assumed assumption. Maybe someone will see my comment and start down their own rabbit hole to find a conclusion. That is more ideal than immediately assuming the details of my personal assumptions and conclusions. :)
      - 4y ago

superkuh 4y ago

HTTP/1.1 support is the last bastion between the web and complete corporate control. Once the megacorp browsers and man in the middle companies like Cloudflare drop HTTP/1.1 we will no longer be able to host a website without the continued approval of a third party corporation. HTTP/2 and HTTP/3 implemenations require the use of CA based TLS. Just to preempt misunderstanding: HTTPS is great. But HTTPS only, with no option for HTTP is very much worse than HTTP+HTTPS for human people. Despite being great for for profit companies and institutions.

MuffinFlavored 4y ago
> HTTP/2 and HTTP/3 implemenations require the use of CA based TLS. You can't do LetsEncrypt?
- acdha 4y ago
  You can. You can also continue to use private CAs, which presumably could be used to add more options if something happens to Let’s Encrypt.
- PenguinCoder 4y ago
  Not for LAN only personal websites.
  - sli 4y ago
    Why not just self-sign for that? Outside of using it to test configurations or deployments, SSL seems a lot less necessary if it's just inside a LAN and all the clients are known.
    - PenguinCoder 4y ago
      Some services don't work without HTTPS or transport security. Sometimes, I want transport security on traffic for various reasons. Current mainstream browsers are also refusing to connect to 'self signed certificate' HTTPS sites because they're 'insecure' and continually are disallowing you the user to bypass these 'protections'.

est31 4y ago

I'm still not over the fact that they made headers all-lowercase in HTTP/2. I know the reasons, but it's so weird to have all-lowercase headers. TBH I don't see much of an uptake in the community either: since HTTP/2 came out, I've barely seen lowercase headers be proliferated in documentation of headers, e.g. MDN lists them HTTP/1 style: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers

treve 4y ago
Anything working with headers should always have been case insensitive, so there's no reason to update the any documentation.
- opportune 4y ago
  As someone who works in infrastructure I disagree with this. While it’s true that applications should always have done this, many many do not and need to be dealt with. If they are third party paying customers, your only option is to figure out how to make things work, because asking customers to upgrade will piss many of them off. The simple fact so many applications do not treat headers as case insensitive is one of the major things holding back http 2. With HTTP/1 it was suggested but not enforced. Upgrading to http/2 with forced-lowecasing breaks these applications
  - treve 4y ago
    What part are you disagreeing with?
avx56 4y ago
Someone should write a DevTools extension that uppercases the headers, if such a thing is possible. :)
- codetrotter 4y ago
  If you want to break protocol, I suggest that we also change all of the HTTP verbs for no reason whatsoever :^) GET -> FETCH HEAD -> EXAMINE POST -> SUBMIT PUT -> REPLACE DELETE -> REMOVE CONNECT -> PROXY OPTIONS -> ALTERNATIVES TRACE -> ECHO PATCH -> PARTIALUPDATE

NickLamp 4y ago

The new QUERY method strikes me as a really promising addition. Not being able to send a body with a GET-type request is a gnawing issue I have with HTTP

0xb0565e487 4y ago
Just send a POST request lol
- HideousKojima 4y ago
  But you're only "supposed" to use POST with requests that add/modify data, or something silly like that. In practice, QUERY is most useful for where you want a bunch of different verbs for the same endpoint and need a body.
- spiffytech 4y ago
  QUERY is cacheable.
  - est 4y ago
    ... in theory only. Not that many http cache programs support QUERY. And many HTTP middle-boxes bans non GET/POST verbs.
    - counttheforks 4y ago
      For now. Change has to come from somewhere.
  - cogman10 4y ago
    And retry-able.
- duxup 4y ago
  In practice I think that’s exactly what many people do.
- Existenceblinks 4y ago

Existenceblinks 4y ago

I would love a queryable file format akin to sqlite but with simple spec tailor made for the QUERY method.

high_byte 4y ago

HTTP is such a mess. only a new protocol can save us, but at this point this is near impossible.

pdimitar 4y ago

What's this "HTTP core"? Please tell me it's just the sane parts of HTTP and without any dark corners and ambiguous specs? Please tell me it's something you can write a parser for in an hour? We're no less 10 years overdue for something like this. And if it's not that... shame.

jimjag 4y ago

IMO, the reason for the initial success of HTTP (1.x) was due to its extreme simplicity. Things like text-based protocol, straightforward stateless design, capability to implement both server and client in easy, basic code. All this meant that the protocol itself was stable, usable, and a reliable standard to use. The current path is to drastically increase complexity due to the demands of the content provider overlord(s); Basically, in order to better accommodate the needs of Google (and a handful of others), we must redefine things for everyone. It's becoming a complex, over-designed protocol that is being crammed down people's throats, instead of a protocol that is embraced because it makes sense.

afiori 4y ago
HTTP 2 allowed for in-stream multiplexing
rubatuga 4y ago
It's interesting that cloudflare states on their own blog that HTTP/3 has no benefits in real-world usage: https://blog.cloudflare.com/http-3-vs-http-2/ https://blog.cloudflare.com/http-3-vs-http-2/

Alifatisk 4y ago

When can I expect, let’s say Rack & Rails to implement HTTP/3?