9 ms·
The HTTP QUERY Method specification
- andyk 3y agoSaw this mentioned by @dragonwriter in the discussion at https://news.ycombinator.com/item?id=36095032 https://news.ycombinator.com/item?id=36095032 but it seemed buried/easy to miss there. Also the article that thread is discussing is from 2021, whereas this was just published yesterday.
- franky47 3y agoWhile you *can* do the same with a GET (include data in the body), it's not spec-compliant for servers to parse and interpret this data. https://stackoverflow.com/questions/978061/http-get-with-request-body#983458 https://stackoverflow.com/questions/978061/http-get-with-req... When designing an API, and if spec compliance is not key, I wonder if client-compliance would become the issue (clients refusing to emit a GET body).
- mgaunard 3y agoParsing the body of GET can be added as an extension, same as adding this new QUERY method.
- DaiPlusPlus 3y agoMany “WAF” (Web-Application Firewalls) and reverse-proxies are configured to block “unusual” traffic though, including GET-with-body - but I feel that this approach is like how (around 2000-2006) everyone switched from high-performance or legacy binary protocols to XML/SOAP-over-HTTPS to avoid corporate firewall headaches.
- thwarted 3y agoThere will most likely be problems with such web-application firewalls anyway, since those same firewalls will probably reject HTTP methods that they don't know about. But adding a new new method is probably overall better and matches people's understanding of the implementation and interpretation of GET, even if (with extensions) GET can have a body people don't think of it like that. So a new method with defined semantics and interpretation avoids a whole bunch of sideshow debate about if GET with a body is possible or appropriate.
- mgaunard 3y agoIt's not up to debate, GET does allow a body.
- thwarted 3y agoYou can make all noise you want about how it's not up for debate because of what the standard says, and said noise does not avoid sideshow bs debates about the how it is used and restricted/limited in practice. A new method avoids that because a new method comes with exactly zero historical baggage.
- mgaunard 3y agoA new method does not give any advantage over extending an existing method, it's going to have to use the same code anyway.
- preseinger 3y agoGET allows, but does not require, server implementations to read/parse a body sending a body with a GET request does not suggest the recipient server will receive that body, in general
- preseinger 3y agomiddleboxes are free to drop the body of any GET request, and generally do so
- adev_ 3y ago> middleboxes are free to drop the body of any GET request, and generally do so Most middlebox will equally drop any HTTP verb which is not whitelisted. Even the extension of WebDAV, which are 15 years old are still commonly blocked.
- preseinger 3y agoblocking a request is fine, the request is stopped, the client gets an error my point is that a middlebox that receives a GET request with a body may drop the body from the request and still forward the request onwards, without the body these are categorically different outcomes
- BugsJustFindMe 3y ago> it's not spec-compliant for servers to parse and interpret this data. That's wrong. A lot of people just don't understand the difference between SHOULD and MUST when reading standards. The standard just says that you shouldn't rely on servers accepting it unless they tell you.
- paulddraper 3y agoI see the need, and good write up, but just use this for the definition of GET body. Nothing in the existing spec prevents GET from having a body, though there isn't currently a semantic meaning to it. This would fit perfectly, be more compatible and result in a simpler spec and protocol.
- rektide 3y agoWe'd have to change the behavior of browsers & server implementations to do this. Itvs much less risky, much more managable change, to do this with a new more deliberate difference. It'll make it clear that the new behavior is intended.
- paulddraper 3y ago> We'd have to change the behavior But you wouldn't for QUERY? This is backwards compatible and in many cases will just work since GET with body is already syntactically valid.
- janderland 3y agoI think they mean we’d have to modify existing features. There are probably assumptions made in code and tests around how GET will be used. Instead of breaking those assumptions and potentially implementing new bugs in an old feature, you can use a completely new HTTP method with completely new code paths. Older feature remains unchanged.
- rektide 3y agoQUERY is an explicit negotiating forwards. It would need support, but nothing with existing systems would change. No web page which accidentally tried to send a http bodied GET would start having new behavior.
- preseinger 3y agomiddleboxes are free to drop the body of a GET request, and many do
- 3y ago
- paddw 3y agoThis seems like it would introduce a tremendous amount of work to solve a problem that basically does not exist. You can just handle your POST request idempotently. We should just live with the semantics we have.
- xrisk 3y agodid you even read the article? it describes why this is a bad idea — it’s not just your server that’s handling the request, it’s all the other middleboxes in between.
- crooked-v 3y agoWith POST, you can't know at a system level if it's safe to cache the response or not.
- DaiPlusPlus 3y agoCache-Control response headers?
- preseinger 3y agoinsufficient
- mnot 3y agohttps://httpwg.org/specs/rfc9110.html#POST https://httpwg.org/specs/rfc9110.html#POST
- kortex 3y ago> A cached POST response can be reused to satisfy a later GET or HEAD request. In contrast, a POST request cannot be satisfied by a cached POST response because POST is potentially unsafe; see Section 4 of [CACHING]. So if you are using POST to query, you can't cache the response. You have to resort to POST/GET. With QUERY, you have idempotent requests with cacheable responses you can directly return.
- codetrotter 3y agoI was wondering which versions of HTTP will this be added to Thinking, will it be possible to send HTTP/1.1 QUERY requests? HTTP/2 QUERY requests? Or will it be for HTTP/3 or something even higher? Well the examples given in the document seems to indicate that it will be possible to use with HTTP/1.1 even QUERY /contacts HTTP/1.1 Host: example.org Content-Type: example/query Accept: text/csv select surname, givenname, email limit 10
- treve 3y agoAny good HTTP implementation will support any HTTP method. If a HTTP method is not recognized, it will basically be treated as if it's a POST method.
- jsd1982 3y agoWhere do you get this assertion from that methods are post by default? Most http server implementations will reply with a 405 error for an unexpected method.
- treve 3y agoServers that just serve static files: yes you are correct. I'm talking about application servers, proxies, etc. You need to decide to do something with a method, but all libraries, servers and clients are ready to handle them and treat unknown methods as unsafe and idempotent (like POST). If you read my comment as 'QUERY will automatically become POST', that's not what I meant.
- denton-scratch 3y ago> Any good HTTP implementation will support any HTTP method. By "support", do you mean "will not crash on encountering an unexpected verb"? WEBDAV and CALDAV rely on a slew of new HTTP verbs; but IME only specialized servers actually support these verbs, in the sense of knowing what to do with them. There's an addon module for Apache that supports WEBDAV, but the logic to support CALDAV is insanely complicated, and nearly all implementations rely on a real database server.
- bornfreddy 3y agoRequiring every proxy and web server to implement their own cache hashing algorithm, especially one that should ignore encoding-specific "non-consequential" parts, sounds like a monumentally bad idea.
- thwarted 3y agoThe cache is local to the proxy or web server, it doesn't matter what the hashing algorithm is as long as the cache accurately returns cached results given the same inputs. The semantic meaning of "input" is different for if it's a proxy or if it's the origin. The origin web server could very well cache based on the result of post processing and validation of the input, while a proxy should cache based on a much more strict (exact series of bytes) interpretation of the input. This is no different than how any other caching proxy is expected to operate given a set of inputs. It's never been up to the proxy to interpret if the queries "name=joe%20user" and "first=joe&last=user" are the same, it just passes the input along to its upstream and then locally stores the result, assuming that the same input will occur again and save a trip to upstream.
- preseinger 3y agoyou're papering over the important details namely, urls are finite, bodies are infinite
- kortex 3y agoSo? Just deal with it however one would deal with unruly POST requests, slow-walked multi-part, and other protocol abuse. No matter what, you need protection against bad actors trying to get the servers to do bad things.
- preseinger 3y agoi'm not sure how malicious requests are relevant to this conversation specifically, a URL can basically always be fully read and cached in memory in a server, a request body cannot
- betimsl 3y agoLove it :)
- deepzn 3y agopreviously it was titled SEARCH. I like query better personally, as it kind of aligns with SQL like requests. Here's a great post on it- https://httptoolkit.com/blog/http-search-method/ https://httptoolkit.com/blog/http-search-method/ and earlier HN thread- https://news.ycombinator.com/item?id=36095032 https://news.ycombinator.com/item?id=36095032
- mholt 3y agoIt is a little ironic that a new HTTP method called "QUERY" is being created largely to be able to remove the query from the URL.
- kortex 3y agoI can't wait until this becomes a spec. I wrote a small middleware to cache sql and graphql queries and I implemented QUERY and Cache-Control. Worked great and saved a ton of bandwidth for developing and running reports, as I didn't have to worry about caching progress. I just reran the whole job. It was like <50 lines of python to pull it off.
- linuxdude314 3y agoI'd love to see JSON-RPC overhauled with this method. It's quite a pain to cache idempotent methods exposed over JSON-RPC as-is currently.
- ta-run 3y agoWhat I fail to understand is how the server interprets the query content/body; how does the server "apply" the "sql" query in the request to the resource? is there something similar to a gql resolver that you need to write?
- kortex 3y agoIt's agnostic, the QUERY verb has nothing to do with the actual implementation, or the content encoding. You can use any content encoding for your query body, and you can resolve it any way you see fit. In mine, I was just caching raw sql queries, so it was literally just text/sql encoding, the query in the body, some metadata in headers, and a sqlalchemy engine to execute the query. It's basically a way to get around the non-idempotency of POST and URI limitations of GET.
- ta-run 3y agoSQL-like queries in the request body seem like a bad idea, what are the security implications and how do we protect against it? Or will the QUERY method end up with the same fate as GraphQL - wherein it's more effective and "secure" in a server-to-server setup and the client only deals with REST.
- sarthak-ag 3y agoIt's just an example, not a production use case.
- crooked-v 3y agoQUERY has nothing to do with the actual contents of the request body beyond "it is hashable text". That could be JSON, GraphQL, weird SQL cousins, or anything else.
- ta-run 3y agoYeah, figured that out in thanks to another thread on here. I got confused as all the examples were primarily sql ones.
- 9dev 3y agoHow is that less secure than a REST API Frontend to an SQL database, like PHPMyAdmin? I don’t think anyone suggests we all open our databases to the web; but if you choose to do so, or if you happen to work on a modern database, like Elasticsearch or CouchDB, which accept queries via HTTP, now there’s a better way to implement queries in regard to caching. That being said: I’ve been wondering for a long time what a backend API could look like that used SQL instead of JSON as the query format - not to pass it to the database verbatim, but with an application layer that speaks SQL, applies business logic, queries the database, and responds in SQL. That would save a lot of reinvented wheels in terms of filtering, ordering, joining, and so on, and give developers a query language they already know. And suddenly, having a QUERY method available sounds useful, too :)
- usrbinbash 3y ago> Unlike POST, however, the method is explicitly safe and idempotent, allowing functions like caching and automatic retries to operate. And just like with POST, whether or not this is actually the case in a given API, depends entirely on the server-side implementation. Look, I get it. We want to make rules. Rules are good. Rules define things. But in a world where so many "REST APIs" are actually "RESTful" or REST-ish, or "actually about as REST as a Pelican, but we really liked the sound of the acronym", I wonder if adding one more rule to the pile is really going to substantially change anything. A majority of APIs don't even use all of the existing HTTP verbs, or HTTP response codes for that matter. And every API is free to make up their own rules. I had the dubious pleasure of consuming APIs that required GET with both a body and urlparams, and which returned 200 on error, but with an `{error: {...}}` object in the body. The crown jewel so far, was an authentication system that had me send credentials as multipart/form-data, with a PUT request (because you inPUT the credentials, see? Not a joke, that was the rationale given to me by the dev who made it.)
- silasdavis 3y agoIf you control both ends of the pipe you are free to use the methods and conventions and maybe benefit when something gets cached. That you cannot rely on consistency is not a reason to have none at all. Destructive actions can and are also hidden behind a GET. When such an endpoint gets called indiscriminately by the rest of the internet it usually becomes the faulty implementation's problem. Presumably the same would be true for QUERY.
- usrbinbash 3y agoIf I control both ends. And if the server side is not a legacy application that no resources get expended on to fix something that isn't broken. Those are 2 very big if's. > That you cannot rely on consistency is not a reason to have none at all. I want consistency. I stated as much in my post: "Rules are good." I'm just not convinced adding more rules to a collection of rules that already isn't consistently applied in the wild, will give me more consistency.
- plugin-baby 3y ago
- ivan_gammel 3y agoI seriously doubt the usefulness of this method. If you cannot fit the query into URL, because it has a complex structure, you are likely requesting a significant amount of work from server. In data-rich environments which constantly update that work will likely produce unique results every time the request is submitted, so caching etc may not make sense. To me, this is pretty much the same as making the request via POST. The very special use case where you execute complex queries against static data does not deserve such a change in protocol.
- erinaceousjones 3y ago> If you cannot fit the query into URL, because it has a complex structure, you are likely requesting a significant amount of work from server. But - are you asking the server to create/update a resource (POST and PATCH semantics), or are you asking the server to provide you a _view_ of a resource? I'm thinking of query languages like GraphQL and jq here, where a client is making a potentially complex request for a view of the data returned in a structure which is most efficient for them. The client isn't intending to _modify_ that data in any way, but it is asking for a potentially computationally expensive view of that data. The way we've done it so far is to either: 1. Cram that complex filtering and view-building into the GET request URI (brittle, not guaranteed URI will function correctly past ~2,000 characters, not clearly defined maximum limit to request URI length in specs) 2. Send the query in a POST request - which works, and is almost what we want, but semantically feels a bit icky, because a query is more cacheable than your average POST, see below: > caching etc may not make sense. Perhaps you have a large and complex but static dataset _which will never change_ which nevertheless has many ways of filtering it -- all of which would benefit greatly from e.g. CDN caching. I'm in two minds here. I half agree with you that POST works fine as the wording of the RFC spec is general enough that what we want to do with QUERY works with it fine as-is. But then I also see how _de-facto_ REST API design conventions have worked against this and treat POST as "create a new resource everytime", and trying to stuff endpoints that go "filter and restructure the responses to my specification" alongside feels.... wrong. (There is of course the argument that you shouldn't be mixing and matching 'strictly' 'RESTful' patterns with other patterns like the GraphQL one which does away with some of the REST HTTP method semantics in favour of defining the operation type in the request data itself, and of course both of these things are patterns which happen to use HTTP and don't need to influence HTTP spec itself...... But still.... I really want that QUERY lol)
- Alifatisk 3y agoThis would be fun to play around with