5 ms·
The HTTP Query Method
- MuffinFlavored 2y agoA simple query with a direct response: QUERY /contacts HTTP/1.1 Host: example.org Content-Type: example/query Accept: text/csv select surname, givenname, email limit 10 Not quite full SQL (no JOIN or WHERE in any examples I see) Hmm... as long as you handle authentication/authorization correctly, why is it bad? It's a way to pluck certain JSON fields that were otherwise going to be returned? Kind of like one of the benefits of GraphQL? Will this catch on?
- SahAssar 2y agoThe request body is just an example, this does not specify the query language, just the http method.
- cjpearson 2y agoI think it would be a good idea to remove these examples from the draft as it gives people the impression that this method is SQL-over-HTTP. A more typical body might be a JSON object with various filter criteria.
- pimlottc 2y agoThe WHERE is implied by the endpoint (“contacts”)
- MuffinFlavored 2y agoThat would be the FROM in my opinion. WHERE would be ? filters, etc.
- pimlottc 2y agoOops, what am I thinking, you’re right
- cryptonector 2y agoThe example uses a non-existent MIME type named `example/query` as the request's `Content-Type`. We don't know what the syntax and semantics of queries of `example/query` type are, and we don't need to know because this is just an example! But the intent is clear that one could have `application/sql`, `application/sparql`, and many other MIME types for representations of queries, and if one where to use one of those then the request body should be valid when interpreted in the context of the request's `Content-Type`. Thus you absolutely could have the full power of SQL, if the server were to support it.
- jessekv 2y agoThis paragraph is the key: > The QUERY method provides a solution that spans the gap between the use of GET and POST. As with POST, the input to the query operation is passed along within the content of the request rather than as part of the request URI. Unlike POST, however, the method is explicitly safe and idempotent, allowing functions like caching and automatic retries to operate.
- TYPE_FASTER 2y agoYes. I was skeptical at first. Why add another method? This explains why.
- solardev 2y agoGraphQL reads are like that. You POST a query with the content inside the request body. This would be a nicer verb for it.
- ako 2y agoOData provides GraphQL functionality but using GET + URL parameters, for large queries it's really hard to read or edit. Using the request body makes sense for readability, so OData also offers the option to query with POST + request body.
- nonethewiser 2y agoBut doesn't this beg the question: why not GET with query params? I'm not necessarily against it - I've had the urge to send a body on a GET request before (cant recall or justify the use-case, however). Reasons I can think of: - browsers limit url (and therefore) query param size - general dev ergonomics The practical limits (ie, not standard specified limits) on query param size seems fair but its worth mentioning there are practical limits to body size as well introduced by web servers, proxies, cloud api gateways, etc or ultimately hardware. From what I can see this is something like 2kb for query params in the lower end case (Chrome's 2083 char limit) and 10-100mb (but highly variable and potentially only hardware bound) for body size. In both cases it's worth stating that the spec is being informed by practical realities outside of the spec. In terms of the spec itself, there is no limit to either body size or query param length. How much should the spec be determined by particular implementation details of browsers, cloud services, etc.?
- cryptonym 2y agoCan be useful for some APIs. The use-case 4.2 with both Content-Location and Location feels weird to me. Not sure you would want multiple urls with different meanings. Isn't it harder to keep it idempotent if we are generating URLs for request and result? Not sure Location is generally meaningful in 200... that may impact other RFC. Could be interesting to see a sample where query is created but result will be available later. That's probably just the 303 See Other with a Retry-After?
- cryptonector 2y agoContent-Location refers to a specific response to the given query. Location refers to a URI that denotes the given query. It's a bit strange, since I don't think it's likely that a server will want to persist any particular response to a query for very long. But it's not unreasonable to support that.
- pgris2 2y agoI don't like using the body. GET's are easily shareable, bookmarkeable, etc, (even editable by humans). etc. thans to query strings. I rather have a GET with better (shorter) serialization for parameters and a standarized max-length of 16k or something like that.
- imbnwa 2y agoThey discuss this desire in the draft: >A server MAY create or locate a resource that identifies the query operation for future use. If the server does so, the URI of the resource can be included in the Location header field of the response (see Section 10.2.2 of [HTTP]). This represents a claim that a client can send a GET request to the indicated URI to repeat the query operation just performed without resending the query parameters.
- lokar 2y agoHTTP is used in many contexts where no human will see the request
- deleted 2y ago[deleted]
- westurner 2y agoThere is currently no way to determine whether there's, for example, an application/json representation of a URL that returns text/html by default. OPTIONS and PROPFIND don't get it. There should be an HTTP method (or a .well-known url path prefix) to query and list every Content-Type available for a given URL. From https://x.com/westurner/status/1111000098174050304 https://x.com/westurner/status/1111000098174050304 : > So, given the current web standards, it's still not possible to determine what's lurking behind a URL given the correct headers? Seems like that could've been the first task for structured data on the internet
- tarasglek 2y agoYeah no way to ask for openapi schema definition.
- westurner 2y agohttps://schema.org/WebAPI https://schema.org/WebAPI https://github.com/schemaorg/schemaorg/issues/1423#issuecomment-667586962 https://github.com/schemaorg/schemaorg/issues/1423#issuecomm... : > Examples of how to represent GraphQL, SPARQL, LDP, and SOLID APIs [as properties of one or more rdfs:subClassOf schema:WebAPI (as RDFa in HTML or JSON-LD)]
- treve 2y agoServers can return something like: Link: </foo>; rel="alternate" type="application/json" You could return multiple of these as a response to a HEAD request for example. You could also use the Accept header in response to a HTTP OPTIONS request, or as part of a 415 error response. Accept: application/json, text/html https://httpwg.org/specs/rfc9110.html#field.accept https://httpwg.org/specs/rfc9110.html#field.accept (yes Accept can be used in responses) .well-known is not a good place for this kind of thing. That discovery mechanism should only be used in situations where only a domainname is known, and a feature or endpoint needs to be discovered for a full domain. In most cases you want a link. The building blocks for most of this stuff is certainly there. There's a lot of wheels being reinvented all the time.
- mg 2y agoThe first example shows a GET request and states that it is "common and interoperable query". But that "if the query parameters extend to several kilobytes or more of data it may not be, because many implementations place limits on their size". So how about simply removing those limits on GET requests?
- IncreasePosts 2y agoThere are thousands of pieces of software which have those limitations built in intentionally or unintentionally though. Using a new verb avoids that issue
- pimlottc 2y agoThe installed base of HTTP processing devices is ridiculously huge. There is no practical way to change the existing behavior of GET.
- packetlost 2y agoI wonder how caching will work when the body has the potential to be huge? You could hash it to save on RAM, but I imagine that would get "expensive" at scale.
- cryptonym 2y agoA robust cache implementation would probably put a limit on request content-length, either denying the request or forwarding it as-is and bypassing cache.
- packetlost 2y agoI mean, yeah? The standard does say *MAY* cache content. It also says the cache server should try to do semantic interpretation of the body if possible, which I think is a misstep. IMO it would be better to have canonical representations of whatever format you're using and then let the cache layer rely on hashing or similar mechanisms. More complicated edge servers/proxies is not what we want because it leads to vulnerabilities.
- nine_k 2y agoHuge as in 50 kB? Fine. Huge as in 50 MB? Well, pass it through maybe if your NVMe device is running out of space.
- paulddraper 2y agohttps://en.wikipedia.org/wiki/Hash_function https://en.wikipedia.org/wiki/Hash_function
- thrance 2y agoI often find myself creating endpoints like `POST /<resource>/query`, which should absolutely be a GET if they could have bodies. Why add yet more to the web specs when we could get rid of the "no body" exception on GET?
- nonethewiser 2y ago> I often find myself creating endpoints like `POST /<resource>/query` Why?
- k__ 2y agoSo, QUERY is a POST but with GET semantics? Sounds like GraphQL won.
- marcosdumay 2y agoIt's way more than GraphQL. Searching and analyzing large things is a real need. It won't go away by changing your protocol.
- paulddraper 2y agoGraphQL queries are a specific usecase for HTTP QUERY. (Today they rely on HTTP POST.)
- dehrmann 2y agoYears ago, I had an interview question that was essentially "model a restful query API." I'm not sure if it was the gotcha or that it was a bad question, but rest isn't a good fit. GET lacking a request body doesn't help, and except for long-running queries, modeling things as resources doesn't work. You can either design an API that isn't very restful or do confusing things to squeeze it into that box.
- the_arun 2y agoFor taking advantage of `Cache-Control` don't we need the URI to be unique? For eg. /Contacts?id=abcd with graphql query in the body? the id can be a hash/signature of query string?
- treve 2y agoNo, the key is not just the URI. It's potentially the URI + method + Some request headers + some response headers. The Vary header lets you control which request headers should be considered. This method specifically includes the request body in that key.
- giobox 2y agoI was just working with OpenSearch's API recently, who sort of abuse the semantics of the HTTP spec by using GETs with a body to perform search queries, largely to solve a similar problem. A "QUERY" message type would probably easily replace the GET with a body used by OpenSearch etc. I'd go as far as to argue thats largely what this new QUERY type is - official recognition for a "GET" request with a body. > https://opensearch.org/docs/latest/api-reference/search/ https://opensearch.org/docs/latest/api-reference/search/
- BugsJustFindMe 2y agoIt's not abuse of the spec. The spec says that GET is allowed to have a body. Whether an endpoint must honor it is left undefined, and this has lead people to believe wrongly that that means endpoints are not allowed to expect it. But certainly the endpoint itself defines what it does with your requests, and that's what actually matters. QUERY is just GET once more with feeling, because people are worried about existing software that made the wrong decision about denying or stripping GET bodies and would need to be updated to allow them through, and detecting an unimplemented verb is I guess maybe simpler than detecting some middle layer being a dick and dropping your data.
- treve 2y agoThis is not a correct interpretation of the spec, as far as I can tell. I did some more research on this + primary sources on my blog if you're interested in why and why people are confused about this: https://evertpot.com/get-request-bodies/ https://evertpot.com/get-request-bodies/
- BugsJustFindMe 2y agoYour post puts a tremendous amount of weight on how you personally feel about using it across load balancers that you don't control and whatnot, but you also still acknowledge that it doesn't violate the spec. > It’s true that undefined behavior means that you as a server developer can define it Good. Agreed. Done. Someone else complying with the spec in an incompatible way doesn't make your compliance wrong. It just reduces the circumstances where it will work. But those circumstances are...well... circumstantial. And I think it's important to establish that there's a meaningful difference between "won't work as expected in certain common but not universal circumstances" and "not allowed to at all".
- divbzero 2y agoRelated: Why an HTTP Get Request Shouldn’t Have a Body - https://www.baeldung.com/cs/http-get-with-body https://www.baeldung.com/cs/http-get-with-body - July 2024
- jkrems 2y ago> When doing so, caches SHOULD first normalize request content to remove semantically insignificant differences, thereby improving cache efficiency, by: [...] That part sounds like it's asking for trouble. I'm curious if this will make it to the final draft. If the client mis-identifies which parts of the request body are semantically insignificant, the result would be immediate cache poisoning and fun hard-to-debug bugs. If it's meant as a "MAY", then that seems kind of meaningless: If the client for some reason knows that one particular aspect of the request body is insignificant, it could just generate request bodies that are normalized in the first place..?
- orf 2y agoYeah, that’s nuts and is obviously flawed behaviour that can interact poorly with any number of things - not least of all any kind of checksums within the response. I’m surprised to see that in a RFC. Edit: it’s only for the cache key: > Note that any such normalization is performed solely for the purpose of generating a cache key; it does not change the request itself. Still super dangerous. Edit edit: I just typed out a long message on the GitHub issue tracker for this, but submitting it errored and I’ve lost all the content. Urgh
- cryptonector 2y agoRequest body normalization is not really feasible, not without having the normalization function be specified by the request body's MIME type, and MIME types specify no such thing. Besides, normalization of things like JSON is fiendishly tricky, and may not be feasible in the case of many MIME types. IMO this should be removed from the spec. Instead the server should normalize if it can and wants to, and the resulting URI should be used by the cache. The 3xx approach might work well for this, whereas having the server immediately assign a Content-Location: URI as I propose elsewhere in this thread would not allow for server-side normalization in time for a cache.
- apitman 2y agoLooks interesting. Sadly it will likely be hamstring by CORS rules. When designing an API, I frequently send all non-GET requests as POSTs with content type text/plain in order for them to classify as simple requests and avoid CORS preflights, which add an entire round trip per request. Obviously only safe if you're doing proper authorization with a token or something. Another fun bit is that you have to put the token in the query string, going against best practices for things like OAuth2, because the Authorization header isn't CORS approved. CORS enforcement is an abomination.
- cryptonector 2y agoBecause of the semantics of QUERY CORS rules should apply as to GETs -- one would think anyways. The Internet-Draft ought to say something about CORS though, that's for sure.
- mtrovo 2y agoI can't stop thinking how HTTP REST is just an abuse on the original design of HTTP and the whole thing of using HTTP verbs to add meaning to requests is not a very good abstraction for RPC calls. The web evolved from being a tool to access documents on directories to this whole apps in the cloud thing and we kept using the same tree abstraction to sync state to the server, which doesn't make a lot of sense in lots of places. Maybe we need a better abstraction to begin with, something like discoverable native RPCs using a protocol designed for it like thrift or grpc.
- TZubiri 2y agoEn.wikipedia.org/wiki/constitutionalism
- jarjoura 2y agoHTTP as a pure transport protocol keeps coming back to the default, because it works. Its superpower is that it pushes stateless (and secure) design from end to end. You have to fight a losing battle to get around that, so if you play along, you can end up with better power efficiency, resiliency and scalability. REST is just very simple to understand and easy to prototype with. There's better abstractions on top of HTTP, like GraphQL and gRPC (as you mentioned), but you can layer those on after you have a working solution and are looking for more performance. HTTP3 is on the way this decade and I'm excited for its promises. Given how long it took for HTTP2 to standardize, I'm not optimistic it will be soon, but it does mean we have a path forward.
- Two4 2y agoHTTP2 has already been mostly replaced by HTTP3, iirc. We now mostly have a split between HTTP1.1 And 3, if I'm remembering an article I read correctly.
- cryptonector 2y ago> The web evolved from being a tool to access documents on directories to this whole apps in the cloud thing and we kept using the same tree abstraction to sync state to the server, which doesn't make a lot of sense in lots of places. The first part is correct, but hard disagree on the rest. HTTP makes a lot of sense for RPC-ish things because a) it can do those things better than RPC, b) HTTP can do things that RPC typically can't (like: content type negotiations and conversions, caching, online / indefinite length content transmission, etc). HTTP is basically a filesystem access protocol with extra semantics made possible by a) headers, b) MIME types, and if you think of some "files" as active/dynamic resources rather than static resources, then presto, you have a pretty good understanding of HTTP. ("Dynamic" means code processes a request and produces response content, possibly altering state, that a plain filesystem can't possibly do. An RDBMS is an example of "dynamic", while a filesystem is an example of "static".) REST is quite fine. It's very nice actually, and much nicer than RPC. And everything that's nice about REST and not nice about RPC is to do with those extensible headers and MIME types, and especially semantics and cache controls. But devs always want to generate APIs from IDLs, so RPC is always so tempting. As for RPC, there's nothing terribly wrong with it as long as one can generate async-capable APIs from IDLs. The thing everyone hates about RPC is that typically it gets synchronous interfaces for distributed computations, which is a mistake. But RPC protocols do not imply synchronous interfaces -- that's just a mistake in the codegen tools designs. Ultimately the non-RESTful thing about RPCs that sucks is that - no URIs (see below response to your point about discovery) - nothing is exposed about RPC idempotence, which makes "routing" fraught - lack of 3xx redirects, which makes "routing" hard - lack of cache controls - lack of online streaming ("chunked" encoding w/ indefinite content-length) Conversely, the things that make HTTP/REST good are: - URIs!! - idempotence is explicitly part of the interface because it is part of the protocol - generic status codes including redirects - content type negotiation - conditional requests - byte range requests - request/response body streaming > Maybe we need a better abstraction to begin with, something like discoverable native RPCs using a protocol designed for it like thrift or grpc. That's been tried. ONC RPC and DCE RPC for example had a service discovery system. It's not enough, and it can't be enough. You really need URIs. And you really need URIs to be embeddable in contents like HTML, XML, JSON, etc. -- by convention if need be (e.g., in JSON it can only be by convention / schema). You also need to be able to send extra metadata in request/response headers, including URIs. (Re: URIs and URIs headers, HATEOAS really depends on very smart user-agents, which basically haven't materialized because it turns out that HTML+JS is enough to make UIs good, and so URIs in headers are not useful for UIs, but they are useful for APIs.) It took me a long time to understand all of this, that REST is right and typical RPCs are lame. Many of the points are very subtle, and you might have to build a RESTful application that uses many of these features of HTTP/REST in order to come around -- that's a lot to ask for! The industry seems to be constantly vacillating between REST and RPC. SOAP came and went; no one misses it. gRPC is the RPC of the day, but I think in the end the only nice thing about it is binary encodings and schema, and that it won't survive in the long run.
- cryptonector 2y agoSo basically a GET with a request body, with the query as the request body rather than as URI local-part query-parameters.
- TZubiri 2y agoSo a POST with different semantics?
- cryptonector 2y agoYes. A POST that is idempotent, safe, cacheable (POST normally isn't any of those). Alternatively this is a standardization of "GET-with-request-body" (which is NOT allowed by the spec but which some implement anyways).
- PaulDavisThe1st 2y agoIETF 2029: The HTTP ASK Method IETF 2034: The HTTP TELLME Method IETF 2038: The HTTP WHATIF Method
- cryptonector 2y agoThose sound like good April 1st RFC fodder.
- arcuri82 2y agoit looks great. Covering a few pain points in modelling APIs when you need to retrieve data idempotently but the URL is not identifying a specific resource, ie, "The content returned in response to a QUERY cannot be assumed to be a representation of the resource identified by the target URI". This is a "work in progress". Is there any estimation of when it will be finalized by? Something like during 2025, and frameworks/libraries starting to support it by something like 2026? Just to have a reference, anyone remember how long it took for PATCH?
- cryptonector 2y agoThis appears to be a Working Group item of the HTTPbis WG [0] which has a Zulip stream [1], a mailing list [2][3], a site [4], and a GitHub organization [5], thus lots of ways to send feedback. The Internet-Draft itself has a GitHub repository [6]. For this I think a GitHub issue/issues [7] would be best; some already exist now for issues in this thread. [0] https://datatracker.ietf.org/wg/httpbis/about/ [1] https://zulip.ietf.org/#narrow/stream/225-httpbis [2] mailto:httpbisa@ietf.org (I think) [3] https://mailarchive.ietf.org/arch/browse/httpbisa/ [4] https://httpwg.org/ [5] https://github.com/httpwg [6] https://github.com/httpwg/http-extensions https://github.com/httpwg/http-extensions/blob/main/draft-ietf-httpbis-safe-method-w-body.xml [7] https://github.com/httpwg/http-extensions/issues
- treve 2y agoMany things never need explicit support, because good HTTP citizens allow unknown HTTP methods (and treat them basically as POST). True for for example PHP and fetch(). Node.js needs explicit support, this got added a few months ago. There's a good chance that as long as your server supports it, you can start using it today.