7 ms·
RFC 10008: The new HTTP Query Method
- lanycrost 4mo agoquery strings always had size limit, seems this new type will solve it which will be really good.
- brookst 4mo agoWouldn’t just putting an etag on POST requests accomplish the same thing? If I’m understanding it the server has to maintain state to ensure idempotency.
- CodesInChaos 4mo agoQUERY is GET with a request body. So it must be safe, not just idempotent. Where safe means it has no significant side-effects. Typically servers will not keep any state for QUERY requests. There is one interesting variant though, which uses state: The client sends a QUERY containing the full query, and the server returns a url usable with GET with which this query can be triggered in the future. Similar to prepared statements in SQL databases. Using QUERY for GraphQL queries (not mutations) would be a good match. These only read data, but are sometimes bigger than the url length limit.
- trollbridge 4mo agoIdeally, libraries like FastAPI, etc. could be configured to translate QUERYs to GETs, until you can rewrite your code to automatically support both.
- n_e 4mo agoInterestingly, despite the QUERY request being safe, the RFC says it's subject to preflight requests: > A QUERY request from user agents implementing Cross-Origin Resource Sharing (CORS) will require a "preflight" request, as QUERY does not belong to the set of CORS-safelisted methods (see [FETCH]).
- CodesInChaos 4mo agoThat paragraph merely describes how existing browsers behave, it doesn't specify how future browsers must behave. After all, a HTTP RFC isn't really the right place to specify browser specific behavior like CORS, that belongs in a W3C/WHATWG specification.
- brookst 4mo agoThanks for the explanation! I still don’t get how idempotency can typically be ensured without state. It very much depends on data model and application design. Even side effects like using a user’s lookup quota need to be handled at a higher layer than HTTP (I think?).
- Joker_vD 4mo ago> I still don’t get how idempotency can typically be ensured without state. Well, how is "GET /index.html HTTP/1.1" made idempotent in practice without (additional) state?
- brookst 4mo agoAh, QUERY is explicitly scoped to be read-only, unlike PUT, POST, DELETE, etc. I had missed that part.
- CodesInChaos 4mo agoMinor side-effects like quotas or request logging are generally ignored when considering the semantics of http methods. I don't see any complications for QUERY that don't already apply to GET. It just allows you to bypass the url length limit by putting the data in the body instead of the url itself.
- uberex 4mo agoExactly. "GET / is idempotent" ... "But I just launched a DDOS against / and now it returns 502...
- wongarsu 4mo agoImagine a forum where comment ids are client-generated UUIDs, and comments are inserted with "ON CONFLICT IGNORE". Submitting the same comment twice would simply be a noop But what the Query method really targets are things like a graphql query that can be multiple kb for a single query, but only reads data. Sure, it might count against rate limits, trigger logs, etc. But at a conceptual level resubmitting the same query should give the same result (if the data didn't change). And since you are only reading data, resubmitting is safe
- deleted 4mo ago[deleted]
- deleted 4mo ago[deleted]
- Joker_vD 4mo agoUnlike POST, however, the method is explicitly safe and idempotent, allowing functions like caching and automatic retries to operate. Essentially, it's for things that are inherently safe/idempotent already (e.g. search or indeed, anything that you don't mind being retried) but require a lot of data passed in the request.
- pwdisswordfishq 4mo agoWait, it's already past 10 thousand?
- rhplus 4mo agoSomeone has an ambiguous bet predicting when RFC 10000 will be published, but the numbers went straight from 9998 to 10008. No-one wins! https://manifold.markets/CollectedOverSpread/when-will-rfc-10000-be-published https://manifold.markets/CollectedOverSpread/when-will-rfc-1...
- Imustaskforhelp 4mo agoEverytime I think that prediction markets bets can't get worse, they do, all in weird ways. I never expected someone betting over when RFC 10,000 will be published but somehow its fits just about right for prediction markets. just wow, people seem to be having too much money it seems for them to bet over when RFC's are gonna get released. This isn't even one of the worst offenders on prediction market or even comparable to it but I am just amazed (in a negative manner, surprised? its just strange) by the depth on what people actually bet on these markets.
- networked 4mo agoPeople aren't betting real money on this. Manifold uses "mana" points similar to HN karma, which is why you get more for-fun silly bets. I don't see anything inherently wrong with it. Disclosure: my mana net worth is 75k; I haven't been active on Manifold.
- Imustaskforhelp 4mo agoAh okay. I didn't know that. Interesting thing actually. Seems similar to the trend in South Korea recently where you can online shop to get the thrill of shopping but you aren't actually paying with money. But I am unsure of the overlap between manifold and polymarket/kalshi. I imagine that some might win in manifold and try to bet on polymarket to win "real" money which ends up being a bit gambling-esque. But good for manifold for atleast not playing with real money but rather points like this. I would argue that Manifold might be better than polygon/kalshi in terms of net positive outcome of its existence for the world perhaps.
- andltsemi3 4mo agoIf this is actually going to replace GET requests w/ query strings in the wild, Im very much hoping for browser bookmarks to support keeping request parameters.
- toybeaver 4mo agoThis makes me happy tbh, I was never a fan of creating `POST /search` endpoints when working with robust APIs
- 100ms 4mo agoIncluding a strong motivating example might have helped sell this, using an example that could trivially be expressed as a GET is extremely distracting. Even imagining a QUERY with a large JSON filtering structure, or say an image input as request body, it feels extremely odd to include the request body as part of the cache key. It also implies an unbounded and user-controlled cache key, with the only really meaningful general caching strategy being bitwise compare of the request body (or a hash), which in a hostile scenario implies cache busting would be trivial. This invokes multiple semantic oddities in one go with obvious difficulties for a very niche use case. If I'm writing a service that needs complex filtering or complex input like an image, any form of caching (e.g. individual data columns of a join, or embeddings keyed by perceptual hashes of a decoded image input) is going to be far away from the HTTP layer and certainly unrelated to the exact bit representation of the request on the wire. Why even bother trying to capture this in a generic way? I would be far more inclined to try and capture this caching semantic as a new header for POST. Something like "Vary: request-body" or similar. Perfectly backwards compatible and perfectly ignorable for all but the 0.1% of CDN use cases where the behaviour might turn out useful
- epolanski 4mo ago> Why even bother trying to capture this in a generic way? I guess it's about resolving the odd semantics of using POST which is not idempotent and thus allowing easier control flow of caches and retrys. Your perspective is 100% correct if you think at the application-layer, but with a dedicated method, you can have that behaviour out-of-the-box out of your HTTP infrastructure (whether it's at your hyperscaler's router or your apache/nginx/browser whatever) and stop implementing yourself the post-as-a-query edge case.
- davidkwast 4mo agoI would use a hash of the body content (the query) as a URL parameter /?hash=123456789
- Joker_vD 4mo agoWhy? That's pushing more work to do both on yourself and the cache.
- haeseong 4mo agoQUERY has existed in spirit for nearly two decades as WebDAV's SEARCH method https://www.rfc-editor.org/rfc/rfc5323 https://www.rfc-editor.org/rfc/rfc5323 and the thing that always killed it in practice was intermediaries. Plenty of proxies, WAFs, and load balancers either strip the body from methods they do not recognize or reject the request outright, so the guarantee that sending a body is safe evaporates the moment traffic crosses a middlebox you do not control. Until gateway and CDN support is real rather than just on paper, POST with a header marking the body as part of the cache key stays the pragmatic choice.
- CodesInChaos 4mo agoI wonder if HTML forms will add support for QUERY: <form action="..." method="query"> This would avoid the annoying re-submission warnings you're getting if you refresh a page that was returned by a POST form submission, since QUERY is required to be idempotent.
- 100ms 4mo agoForms, HTTP implementations, public API surfaces, and all for what exactly. Introducing a new verb for this feels profoundly misplaced
- jagged-chisel 4mo agoIdempotency is an important attribute for correctness. Yep, you can document that POSTing to $ENDPOINT is idempotent, but you can't communicate that to caching layers throughout the network. QUERY, by definition, is idempotent and cacheable.
- jnewton_dev 4mo ago[flagged]
- inigyou 4mo agoLarger scales like what? I expect that everywhere you currently cache GETs you can cache QUERYs. But does caching GETs work at scale?
- resters 4mo agoGreat point. I wish more people realized that intuitively.
- alpinisme 4mo agoAt least support - or lack thereof - for a new verb is unambiguous (compared to changing the semantics of GET)
- 4mo ago
- nottorp 4mo agoIt's as bookmarkable as a query with its parameters in the POST data...
- piterrro 4mo ago> GET request with a body was heavily considered by the IETF working group, but it was ultimately rejected in favor of creating the new QUERY method. The decision to create a distinct method came down to historical interoperability issues and strict compliance with the core architectural definitions of HTTP. I've been sending request body along GET method for years now
- huskyr 4mo agoApparently some load balancers drop the body.
- inigyou 4mo agoI expect all sorts of intermediaries may drop the body, since having a body is forbidden by the standard. When it's your client talking to your server you can obviously do whatever you want - it doesn't cause problems until you want to involve third-party code, such as a reverse proxy (such as nginx) or a CDN. This includes proxies your customers may be using.
- jmaw 4mo agoWhere is it forbidden by the standard? I don't see anything in the GET definition in RFC 9110 [1] forbidding that. My understanding was that this is just undefined behavior. And not recommended due to your point about some third-party CDNs and RPs handling that UB in different ways. [1]: https://datatracker.ietf.org/doc/html/rfc9110#name-get https://datatracker.ietf.org/doc/html/rfc9110#name-get
- giobox 4mo agoIt's never been explicitly forbidden, just heavily discouraged in virtually every version of the spec. In the current 9110: > "A client SHOULD NOT generate content in a GET request unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported. An origin server SHOULD NOT rely on private agreements to receive content, since participants in HTTP communication are often unaware of intermediaries along the request chain." "Content" is what the RFC calls a request body. The simplest argument against IMO is it makes caching more complex, as now the request body would have to be part of any cache keying etc, you can't just blindly key off the request + URI params for every single GET. There are plenty of other reasons to not do it. The spec expects responses to GETs generally to be cacheable: > "The response to a GET request is cacheable; a cache MAY use it to satisfy subsequent GET and HEAD requests unless otherwise indicated by the Cache-Control header field (Section 5.2 of [CACHING])."
- drzaiusx11 4mo agoI don't hate it. Covers all the bases: 1.1, 2, 3/quic and solves real problems: get query limitations vs body content & post-without-mutation. Yes there are preexisting workarounds, but they're non-obvious.
- smashed 4mo agoUse the QUERY method in your http query to query search results. Do not add query parameters. I think the name is confusing because the term 'query' is already used to refer to http requests in general. Just the title of the RFC confused me.
- comfydragon 4mo ago> the term 'query' is already used to refer to http requests in general In what circles is this the case? I sometimes colloquially refer to a GET request as a query, but definitely not so on a POST, PUT or DELETE.
- jbmchuck 4mo agoI'm guessing the op is referring to https://en.wikipedia.org/wiki/Query_string https://en.wikipedia.org/wiki/Query_string
- rehevkor5 4mo agoYeah, and it doesn't even have to be a query, it could be an idempotent effect. I think they'd be better off calling it IPOST (for idempotent post). Edit: ah, they declare QUERY as "safe" meaning no side effects, for cacheability. My mistake.
- tonympls 4mo agoAs "just a guy that programs" (ok, now guides agents to program) and tries to follow the rules (with a big dose of pragmatism), this totally makes sense to me. This is also the first time I've seen or heard about this coming. I like that we now have a way to not being forced to define Resources when we want to query. It always felt like I was missing something that there could be an infinite, defined-on-the-fly number of Resources for a "part" of a given Resource. Do I really want to define "all cats that sleep more than 20 hours a day and like sunbeams and want to eat breakfast at 3 am" as a Resource? (ok, we all know that is actually the full set of cats). I'm ok that you want to define that as a Resource but in my system, it makes more sense that Cats is the Resource and I just need some accepted way to query. I like the implementation (again, as just a guy that programs). I don't see how it could have done it better or simpler which probably hides the complexity of getting there. I also especially appreciate how the spec is written. Opening a spec, I wonder how far I'll get before I don't know what the heck they're talking about (and, again, as just a guy that programs). I don't think it's easy to write a spec that is complete and approachable like this. Really appreciate that.
- inigyou 4mo agoThe standards have always been a bit more abstract than you use in practice. Common practice prior to this would be /catsearch?sleep=20&sunbeams=yes&breakfast=0300 where the "resource" is /catsearch and the rest are query parameters, but you could also use /catsearch/sleep/20/sunbeams/yes/breakfast/0300 which looks like a "resource", but nobody is actually enforcing that it is a "resource".
- mlhpdx 4mo agoWow, it still isn’t a standard? I’ve been building with the QUERY method for years now. I’ve enjoyed the combination with Range headers for paging, despite this tidbit: > It is expected that these built-in features will be used instead of HTTP Range Requests Using the QUERY request as the definition of a set, and Range to retrieve subsets seems very natural.
- ynac 4mo agoJust in case anyone wants to pretend it's still that other century: https://www.rfc-editor.org/rfc/rfc10008.txt https://www.rfc-editor.org/rfc/rfc10008.txt
- jjice 4mo agoI'll forever love a long, totally plain text document like this. So many good times with video game FAQs as a kid. It really is a superior form of information in a lot of ways (not all).
- riffic 4mo agoI have a thesis brewing that explores how rich text WYSIWYG editors create a "what you see is all there is" cognitive bias, while plain text overcomes it.
- riffic 4mo agobeautiful formatting. I should crib this style template for internal work memos, it's timeless.
- deleted 4mo ago[deleted]
- geth101 4mo agoJust allow body for get. Problem solved.
- CodeWriter23 4mo agoNah, you need the 405 Method Not Allowed to be returned by legacy systems and proxies along the way to your bleeding edge server rather than silently failing whilst dropping the params in the request body.
- cosmotic 4mo agoWhy not just define the semantics of a GET request body?
- chadgpt3 4mo agoProxies often delete it
- elAhmo 4mo agoThey could be updated to not delete it, like they would require for this new method anyway.
- moralestapia 4mo agoAgree. They should not delete the body in the first place.
- flufluflufluffy 4mo agoThe HTTP standard does not define a body for GET requests. Therefore, proxy implementations will typically only copy the data from the request header when sending it off to its next destination. They don’t technically “delete” anything. This saves compute (and possibly bandwidth, if the request happened to have a body, although the whole point is that one can safely assume a GET request does not have a body per the standard).
- jerf 4mo agoIf we're going to play the "should" game, whatever originated the body "shouldn't" have because the spec says that's illegal. Although we could also go with, a proxy shouldn't delete the body, it "should" reject the request outright as ill-formed. The meta-point I'm making here, which I'm sure will be missed if I don't spell it out, is that if we're going to talk about what "should" be done when it is explicily out of the scope of a standard, there's no way around the fact that there are multiple completely sensible ways to extend the standard and there's every reason to expect that in the real world people aren't going to agree. Sometimes they manage to, but even then often quite imperfectly. Our human intuitions that standards are something that are "built" is perhaps not wrong, but you can also productively look at standards as taking the raw material of all possible things two systems could send to each other and removing possibilities. If you reach into a space that has been explicitly removed, you can't expect everyone else to do so in exactly the same way you will.
- simon84 4mo agoThis whole thing is non sense. It basically mixes technical constraints (body or not body) with a functional requirement that arises from people that are tied to semantics of the protocol. HTTP is transfer protocol. It should not ever imply anything at the business level. Yes REST made it's worst mistake out if it by giving a meaning to the verb. Yes proxies rule how the body is re-interpreted in spite of the will of the sender (wtf). But the original RFC states clearly that any verb can be used. This is how WebDav normalised its own. But playing fancy by introducing a change that all HTTP implementation will have to honor is a very bad and irrational choice.
- pie_flavor 4mo agoYes this, yes that, yes the other, because proxies are in agreement with patterns are in agreement with the HTTP spec that methods exist and have semantic and functional requirements. Your 'should' seems to be discussing a hypothetical technology that is not HTTP, because HTTP has worked this way since 1997.
- simon84 4mo agoYou are correct, better concede than argue. What I meant was a tangent: the HTTP was designed with a strong assumption that the application implementing it (the web server) would be the one providing the logic. Hence all the RFC terminology "should", "must", targeted at the implementor. But very quickly, the logic was deferred to a layer on top (PHP,...) which would focus on the business aspect. The wiring was strong but the contract intent is loose (the requirements on the transport do not apply to the function). Different layers, different people involved. What survived are the more or less conventions about it, which for the ease and limitations imposed by the protocol layer, led to infinite discussions about GET having a body or not. The whole question arises because there is this clash between transport and logic that is wired on top, not built-in. So while indeed, introducing QUERY solves a protocol gap, the people designing the business method never cared (or even knew) about that gap in the first place. This was another people's job to try to reconcile the two. That's why I'm saying that digging into the initial assumption that the implementor of the HTTP is bound to business-level contraints is not reflecting the reality that has been going on since the early days of the dynamic web.
- barbazoo 4mo ago> GET: Content (body) "no defined semantics" I thought it wouldn't be a terrible idea to open up the GET method to contain a body but according to the original spec the GET body is to be ignored completely. There's also caching which would break because the important bit of the request would live in the stripped body.
- angrybards 4mo agoGET with only a URI has the semantics of retrieving the current representation of the resource. This is the most basic form of hyperlinking and quite important to how the web works. Adding a body parameter to GET would break that constraints of the method as you couldn't treat two requests using the same URI as referencing the same thing.
- stymaar 4mo agoWhy not standardize a body in the GET request (which isn't forbidden per spec and works in many places already but isn't supported everywhere because it's not mandated to support it)?
- mholt 4mo agoToo many servers ignore/drop/reject body in GET requests. RFC 9110 does allow it, but is only recommended if server documentation states that it is supported.
- stymaar 4mo agoThese servers will likely also reject the QUERY request though…
- greghines 4mo agoBut at least in that case you may get a much more meaningful 405 Method Not Allowed response rather than the server just silently dropping the GET body content.
- nfw2 4mo agoYeah those servers need to be updated anyway to support the new standard Also, anyone using these servers is not currently putting params in GET body because doing so wouldn't work. Those that oversee evolving standards seem to take extreme cautious to not rock the boat, and so more and more gotchas for the sake of backwards-compatibility keep getting added to the mountain of random details new devs have to learn. Which will soon include the difference between a GET and a QUERY I guess.
- AtNightWeCode 4mo agoKinda pointless since traffic is encrypted. If you can terminate TLS you can apply any rule based on the content as well. Like headers which is more reliable.
- CodeWriter23 4mo agoIt's about time.
- drob518 4mo agoAside: Wow, we’ve hit 5-digit RFC numbers now!
- etchalon 4mo agoWhy does this feel like GraphQL demanding everyone else solve their problems?
- inlined 4mo agoI know agents are out of scope of this RFC but I love that this could easily be extended to make the JS EventSource to work on streaming AI queries. Due to the need of bodies in requests, everyone uses POST and streaming results often use the text/event-stream protocol for responses. But this is technically a bad fit because no state is actually changing and because EventSource can only use GET for some obstinate reason. So many APIs reimplement the functionality with their own parser
- ygouzerh 4mo agoThe exemples section at the end, with csv and sql are quite powerful. It open the door to easy caching of raw data and probably other use cases, quite interesting!
- TheGRS 4mo agoI think I'm seeing the advantages of using this, but I can't help feel like momentum is going to be strongly against adoption. Already feeling like if I suggested this it would meet a ton of eye-rolls, there'd be all this new plumbing needed just to support something we can already do in similar ways.
- exabrial 4mo agoJust curious, was it Google/Facebook that sponsored this?
- userbinator 4mo agoTwo of the authors are from Akamai and Cloudflare.
- exabrial 4mo agoInteresting. The reason I asked is both companies were on a user hostile (per usual) campaign to hide query parameters from the regular consumer in an attempt to track them better.
- paulddraper 4mo agoWhat no one has been able to explain: Why is that better than a longer URL? Semantically it identifies the resource. And it must be included in the cache key. True, not everything supports long URLs. But not everything supports QUERY either, so it’s an absurd argument.
- deleted 4mo ago[deleted]
- deleted 4mo ago[deleted]
- userbinator 4mo agoSo this is basically just a "big GET"? Edit: refute or agree, please.
- impara 4mo ago[flagged]
- yoshi389111 4mo agoRFC numbers finally reached 10000. It's interesting to think that the RFC series started in 1969 and has continued for more than 50 years.
- robertlagrant 4mo ago> https://www.rfc-editor.org/info/rfc10008/#section-2.8 https://www.rfc-editor.org/info/rfc10008/#section-2.8 Having a standard way to do pagination would be extremely useful. If we could add an items (or similar) range header type that would be excellent! For example, in the HEAD response (and the QUERY response): accept-ranges: items items-length: 342 In the QUERY request: Range: items=0-9
- codedokode 4mo agoThe pagination is not always done with page numbers, sometimes there are "next" and "previous" tokens.
- robertlagrant 3mo agoTrue. Perhaps it could be done using a Link header, although that won't give you the total number of records.
- codedokode 4mo agoWhen adding new HTTP methods, they should have included protection against cross-domain requests into the method, i.e. the server should not response to QUERY requests from another domain by default and the browser should not include cookies and auth in cross-domain requests. This was a mistake not to disable cross-domain GET/POST requests and it should not be repeated.
- therealdrag0 4mo agoI’m so glad I work with RPC frameworks in the backend that don’t have to deal with these weird restrictions and bike shedding. I get that REST is well embedded in the public internet, but I don’t get why so many people use it in their private systems.