12 ms·
IETF: The HTTP Query Method ( Draft)
- anticristi 5y agoMy OCD is really happy that the symmetry is restored. I always felt that GET with the optional-but-actually-forbidden request body stands out. Once we transition from GET to QUERY, all HTTP transactions will be header+body in one direction, followed by header+body in the other.
- dragonwriter 5y ago> Once we transition from GET to QUERY, all HTTP transactions will be header+body in one direction, followed by header+body in the other. Why would you put a body in a HEAD, OPTIONS, TRACE, or DELETE request (or GET, which will continue in use alongside QUERY)? And, no, a lot of response also don't have bodies, and that will continue.
- unilynx 5y agoThere will be no transition from GET to QUERY, they will exist next to each other. Odd proxy bugs/client quirks with zero/non-zero Content-Length headers in GET requests will remain with us forever
- politelemon 5y ago> Implementations are free to use any format they wish on both the request and response. The samples should include some non-SQL, completely made up ones, as I think a lot of people are going to fixate on the SQL-like syntax and its associated problems.
- dschep 5y agoit does https://www.ietf.org/archive/id/draft-ietf-httpbis-safe-method-w-body-02.html#section-1-7 https://www.ietf.org/archive/id/draft-ietf-httpbis-safe-meth...
- sdesalas 5y agoNext up, we'll be getting an ENQUIRE method. Then we'll be ready for INQUIRE too. Which is similar but involves more formalised header definitions. Following this. We'll get an RFC for OBTAIN, which will be similar to GET but will formally renounce the use of a payload.
- tester34 5y agoWhat is the difference between this and allowing HTTP GET with Body?
- dragonwriter 5y ago> What is the difference between this and allowing HTTP GET with Body? What's the difference between PUT and allowing DELETE with a body?
- soheilpro 5y agoFrom RFC 7231: A payload within a GET request message has no defined semantics; sending a payload body on a GET request might cause some existing implementations to reject the request.
- NovemberWhiskey 5y agoRight. The HTTP RFCs have been backing off gently from the initial position that implementations should not sent bodies with GETs and that the semantics of the GET request were defined purely in the request URI. But presumably no-one is brave/foolhardy enough actually to redefine GET as having a semantic body because a bazillion different implementations (clients, servers and middle boxes) probably become non-compliant.
- saurik 5y agoAre they more or less likely to be compliant, though, than with a new verb?
- detaro 5y agoHandling a method or not is a much more obvious thing to discover/observe.
- tester34 5y ago>redefine GET as having a semantic body because a bazillion different implementations (clients, servers and middle boxes) probably become non-compliant. So what actually? apps that didnt use GET Body, will not care anyway apps that will use HTTP GET Body will be checked anyway So, unless somebody downgrades HTTP Server then what could be the problem?
- ferdowsi 5y ago> The QUERY method provides a solution that spans the gap between the use of GET and POST. As with POST, the input to the query operation is passed along within the payload of the request rather than as part of the request URI. Unlike POST, however, the method is explicitly safe and idempotent, allowing functions like caching and automatic retries to operate. Is this really worth a change to every HTTP client library out there to support this? The limited applications that really need this can easily use POST and document their own semantics around this. If anything the trend with GraphQL is to ignore HTTP verbs outright because they are limited and inexpressive beyond simple CRUD tasks.
- jonwinstanley 5y agoAgreed. Seems like a slight improvement over something that was decided and settled many years ago.
- cryptonym 5y agoPOST request caching is always tricky and often not allowed. It can also improve reliability: you can safely retry such request.
- brightstep 5y agoDefinitely. The HTTP spec has a gap that's being filled with a hack, albeit a widely accepted and implemented one. QUERY removes ambiguity, aids self-documentation of APIs, and improves caching.
- jayd16 5y agoIt also seems trivial to fallback to POST for backwards compatibility, no? I'm not sure it needs every lib to be updated before devs can gain value from this.
- _8j50 5y agoThis is cool and all but why not just expand the scope of GET requests in newer HTTP standards? Maybe have a X-GET-QUERY header to indicate the type if GET request? the problem I see with a new method is that it isn't just webservers that need to support it, it is also webapps. Ideally this would be transparent to the webapp (which would just see really big arrays of GET params). The user-agents (browsers) would ideally support this transparently where as with a new method the JS/HTML would need to explicitly support it.
- throwoutway 5y agoOnce browsers offer support, the web app+ server would need to support it. Both of those are in control of the devs. The real problem lies in getting infrastructure teams to update the expensive F5 load balancers, and PA/CP/TP firewalls to process the requests. Those aren’t in control of the devs (unless they’re operating together well as a team)
- brightstep 5y agoOne of the issues that QUERY solves is that POST is overloaded and is being used for purposes beyond its intended responsibility. Shifting that overloading to GET feels to me like just another hacky approach. I prefer the well-defined, single responsibility that QUERY brings and restores to POST.
- wojcikstefan 5y ago> Maybe have a X-GET-QUERY header to indicate the type if GET request? Note that using the "X-" prefix has been deprecated since 2012: https://datatracker.ietf.org/doc/html/rfc6648 https://datatracker.ietf.org/doc/html/rfc6648
- _8j50 5y agoWow, news to me. Thanks
- wizzwizz4 5y agoHere's the relevant section of the guidelines: https://datatracker.ietf.org/doc/html/rfc6648#section-3 https://datatracker.ietf.org/doc/html/rfc6648#section-3 Creators of new parameters to be used in the context of application protocols: 1. SHOULD assume that all parameters they create might become standardized, public, commonly deployed, or usable across multiple implementations. 2. SHOULD employ meaningful parameter names that they have reason to believe are currently unused. 3. SHOULD NOT prefix their parameter names with "X-" or similar constructs. Note: If the relevant parameter name space has conventions about associating parameter names with those who create them, a parameter name could incorporate the organization's name or primary domain name (see Appendix B for examples).
- rvr_ 5y agoThis must be some form of late April 1st joke. The samples are even SQL-injection! We need to stop modifying basic protocols like HTTP. We should've stop at 1.1.
- Angharad 5y agoThis is only a proposal for a new HTTP verb. How you use it (and how you interpret is) is completely up to you.
- nostoc 5y agoInjection means you can modify an existing query. The example are not SQL injections, they are full queries.
- otabdeveloper4 5y agoHTTP is not just for the web. In fact, the vast majority of HTTP trafic doesn't involve the browser at all. The examples are realistic and useful. E.g., Clickhouse uses POST methods for queries, and a ridiculous `&readonly=2` parameter to differentiate modifying queries from readonly SELECT queries.
- intrasight 5y agoI don't see any sql injection in samples
- mjb 5y ago> QUERY requests are both safe and idempotent with regards to the resource identified by the request URI. Is that really what you want from a query operation? I read 'idempotent' as implying that result sets don't change over time, which would be surprising behavior for queries for most database-like things. It's probably also worth mentioning that SQL's SELECT isn't idempotent in the way HTTP means it, because of the existence of session state, pessimistic locking, and the requirements of higher isolation levels. It would be useful for an RFC to define 'idempotent' in a way that clearly addressed these issues (and, for that matter, the larger topic of sessions/transactions) more clearly. > When doing so, caches SHOULD first normalize request content to remove semantically insignificant differences, thereby improving cache efficiency Unfortunately, again when you look at SQL by comparison, queries are not purely expressions of what to return. Practically, they also encode how to compute the query (either explicitly through hints, or implicitly through things like join order). These behaviors are weird, tricky, and change version-to-version. > The QUERY method is subject to the same general security considerations as all HTTP methods as described in As another commenter said, this is quite incomplete. Query parameter injection, DoS by locking, DoS by exploiting work the database needs to do to ensure isolation, DoS by extremely expensive query, etc. > 4.2. Simple QUERY with indirect response (303 See Other) At least the examples here are naive - most applications don't want query result sets to be easily accessible to others. The semantics of authn and authz need to be really crisp here to make sure that attackers can't access the location of other queries result sets purely by guessing. At least a "SHOULD use auth" or "SHOULD have large, unguessable, names" would be valuable here.
- detaro 5y ago> I read 'idempotent' as implying that result sets don't change over time, which would be surprising behavior for queries for most database-like things. That's not what idempotent means in HTTP. > As another commenter said, this is quite incomplete. Query parameter injection, DoS by locking, DoS by exploiting work the database needs to do to ensure isolation, DoS by extremely expensive query, etc. Is application-dependent and applies to all other HTTP methods too.
- mjb 5y ago
- est 5y agoHTTP middle boxes are going to love this.
- dagss 5y agoWith an official readonly header to POST instead all middlewares and proxies would have automatic support and this can be adopted in months instead of years or decades...
- tsimionescu 5y agoWhy would that be easier to implement than a new method? It's literally just a string. Many non-standard request methods are already supported.
- glenjamin 5y agoBecause unknown headers are already passed through safely in existing implementations, whereas unknown methods are handled in a variety of different ways
- intrasight 5y agoI for one welcome the addition. And am sort of surprised that it wasn't made a verb earlier.
- hodgesrm 5y agoIt seems as if the idea behind this proposal is to help out database folks. If so, that is misguided. POST is a better implementation than QUERY (or GET) at least for SQL databases. Here's why. In SQL this is a query: SELECT a, b, c FROM foo LIMIT 1 But this is also a "query" in many if not most connectivity APIs. INSERT INTO foo VALUES (1, 2, 3) Most client libraries don't know and don't care about the content of the query. It's the database's job to parse it and and do the right thing. The different between the above queries is that the first one returns a result set and the second returns an update count. Here's a simple example using Python and the clickhouse-driver library. # An UPDATE to the database client.execute('INSERT INTO iris SELECT * FROM another_iris_table') # A harmless "query" result = client.execute('SELECT COUNT(*) FROM iris') print(result) For this to work you need to use something underneath that is generic and works regardless of output. POST does this already. The clickhouse-driver does not use HTTP protocol though other ClickHouse drivers do. I'm just using it as example of why you need a protocol than can handle any type of SQL "query" the same way on the wire. Otherwise the client will have to have a SQL parser to figure out which one to use. (Some clients actually do that but they are a very small minority.)
- deleted 5y ago[deleted]
- brightstep 5y agoThe examples in section 4 are just that, examples. They are not intended to be the only format a query may take. The issue with POST that QUERY solves is lack of an idempotency constraint.
- wojcikstefan 5y agoIMO this helps a lot more people than just database folks. Any web application which implements a fairly granular search/filtering mechanism for its resources may run into the URL character limit with GETs. QUERY sounds much more appropriate here than what most applications do today (i.e. abuse POSTs).
- hodgesrm 5y ago
- danidiaz 5y ago> QUERY requests are both safe and idempotent with regards to the resource identified by the request URI. That is, QUERY requests do not alter the state of the targeted resource. However, while processing a QUERY request, a server can be expected to allocate computing and memory resources or even create additional HTTP resources through which the response can be retrieved. The possible creation of extra HTTP resources (response resorces?) seems to me contrary to idempotency. That seems more like the territory of POST. If two identical QUERY requests might produce different response resources, how to square that with the the fact that QUERY will be cacheable?
- anticristi 5y agoI was wondering the same, but this seems to clarify that: > The cache key for a query (see Section 2 of [HTTP-CACHING]) MUST incorporate the request content.
- brainwipe 5y agoIn HTTP idempotent is where the state of the server remains unchanged. A resource in HTTP is what you get back from an URL, so https://example.com/people/show/10 https://example.com/people/show/10 (where 10 is the page number) is a different resource to https://example.com/people/show/100 https://example.com/people/show/100. If I interpret it correctly - adding more resources isn't changing the server state but adding more ways of getting to the state.
- danidiaz 5y agoSo that means that, without a cache, repeating a QUERY might create two response resources but, with a cache, only one will be created. I find that odd. My understanding of HTTP idempotency is that it's more of a "whole-server" concept (excepting perhaps things like creation of log entries and metrics). Always creating a new resource for each request seems contrary to that. A way to square creation of response resources with idempotency could be: the second identical QUERY that arrives should always reuse the result resource created by the first QUERY.
- 5y ago
- indymike 5y agoThe HTTP Query method is problematic: every request to a web server is by definition a query, so it is at a minimum poorly named. Second, most queries are not idempotent and the return value can and will change. In other words, YAGNI.
- layer8 5y agoI don’t agree that "every request to a web server is by definition a query", similar to how not every SQL statement is a query. In terms of command-query separation, commands may return information about the execution of the command; the fact that they return something doesn’t make them a query. For example, an SQL UPDATE statement may return how many rows were updated, or that some error occurred; that doesn’t make it a query.
- indymike 5y ago> I don’t agree that "every request to a web server is by definition a query", Query is a synonym for ask. Request is a synonym for ask. We are now making request requests? This is redundant and is not needed.
- wojcikstefan 5y agoI'm very happy about this proposal. The only sad thing is that it has come so late, after so many tools and protocols (e.g. GraphQL) already abuse POSTs for this use case.
- hermitdev 5y agoI agree. It takes me back arguments I had with my PM when I worked for a small SaaS close to 10 years ago. I had to use POST for a query API because of the limitations around GET & URL encoding of the parameters for the exact reasons outlined in TFA. She insisted it be a GET until I showed real, existing client queries that couldn't be handled. Only then, did she relent. Same PM also insisted I send results of queries as a list of objects in JSON, instead of a more compact tabular format, because tables aren't REST-y. I lost that battle, and the serialized results of queries were an order of magnitude larger than they needed to be...
- Spivak 5y ago> She insisted it be a GET until I showed real, existing client queries that couldn't be handled... Same PM also insisted I send results of queries as a list of objects in JSON, instead of a more compact tabular format, because tables aren't REST-y. I think I'm on the side of the PM with this one on both counts. You sound like someone who really cares about efficiency, performance, and edge cases -- a proper engineer. But PMs are supposed to bring us down to earth and say that simplicity and maintainability are more important than saving bytes and to not waste time fixing things that aren't broken. In a past life I spent so much effort optimizing our stack to lower our AWS bill until a PM sat me down with the company's finances and showed me the teeeeny little bar that was our cloud expenses and then legitimately 20x taller bar that was salaries and basically said that spending money to buy back my or my team's time was more important.
- wizzwizz4 5y ago> lost that battle, and the serialized results of queries were an order of magnitude larger than they needed to be... Before or after Content-Encoding: gzip?
- davidhariri 5y agoI'm in favour of a QUERY verb. Always felt wrong to use GET or POST for it.
- jasonhansel 5y ago> caches SHOULD first normalize request content to remove semantically insignificant differences, thereby improving cache efficiency This feels like a bad idea, since (a) different caches will support different content types and normalize them in different ways, leading to unexpected changes in behavior and (b) some servers may behave differently depending on something that a cache considers to be a "semantically insignificant" distinction. I'm not sure, in other words, if I trust caches to get this right. It seems like it might be better to require clients to submit requests in a "pre-canonicalized" form, or to have caches allow this behavior but disable it by default.
- mjevans 5y agoThis is what I would like to see as a logical conclusion that is future proof: The query and response MUST be transmitted and stored 'as is' (a sequence of octets). The query and response SHOULD be encoded in either UTF-8 or WTF-8 https://simonsapin.github.io/wtf-8/ https://simonsapin.github.io/wtf-8/ Future standards or non-standard systems MAY use different encodings; conformant implementations MUST NOT alter the sequence of bytes. They MAY perform a validation check and add additional headers.
- progval 5y agoFrom your link: > WTF-8 must not be used to represent text in a file format or for transmission over the Internet.
- mjevans 5y agoIt's eventually going to be anyway. This is a problem that isn't limited to Rust and single hosts. Also, as per the rest of my ideal solution, the true encoding is: Sequence of Octets / Bytes
- dragonwriter 5y agoSequence of octets isn't an encoding, and content-aware intermediaries need to know the actual encoding.
- the_arun 5y agoI am unable to understand - what is the difference between GET and QUERY? Just that in QUERY you can send parameters in the request body? Do we need a new method for that?
- tsimionescu 5y agoYes, because there are various assumptions about GET that won't fit if GET can suddenly contain a request body. For example, existing caching servers may continue to cache content based only on the URL and headers, ignoring the request body entirely, producing bad results. Additionally, there may be some more subtle problems related to the Content-Length header, which is supposed to NEVER be sent for GET, but would be required for QUERY (since all requests that can contain a body MUST have a Content-Length header, depending on encoding; while requests that can't contain a body MUST NOT have a Content-Length header).
- deleted 5y ago[deleted]
- zestyping 5y agoThere's no way to do client-side caching with this, which seems like a fatal omission — in any given situation where you would consider using QUERY, it'll almost always be more efficient to put the query in the parameters of a GET requset.
- remram 5y agoWhy do you think you can't do client-side caching?
- efitz 5y agoRemind me again why we are using a document transfer protocol for API use cases and trying to bolt on functionality like Frankenstein’s monster?
- mpolichette 5y agoI'll entertain this... Because, like it or not, you're often still sending/receiving documents? :) Also the semantics of the actions map really well to the use cases still. You're, of course, more than welcome to avoid the whole thing and use WebSockets or WebRTC data channels with a custom protocol.
- the_arun 5y agoWould this help GraphQL to use HTTP in proper way? like use HTTP QUERY for non-mutational requests?
- dschep 5y agoDoubtful given that GraphQL doesn't prescribe a transport layer. [0] [0] https://graphql.org/faq/#does-graphql-use-http https://graphql.org/faq/#does-graphql-use-http
- chrismorgan 5y agoDoesn’t prescribe one, no, but when it is over HTTP, it’d be perfectly reasonable to have it accept QUERY for non-mutating requests, like it can currently use GET or POST.
- amdelamar 5y agoA year ago I had a coworker build a CRUD app with an API that required GET with a body. Something simple like: GET /api/query Host: example.org Content-Type: application/json { "a": "valueWith$pecialChars", "b": "valueWith$pecialChars", "limit": 100 } It worked fine for him because he used curl, which allows GET with a body. But I was using Paw (similar to Postman) which refused to send it. I mentioned the issue to him to which the reply was along the lines of "its a non issue, just use curl". I kid you not, 1 week after this coworker left for another job I fixed the service to accept POST requests. If QUERY was around I'm sure I could've made a stronger case to fix it sooner.
- spyspy 5y agoI’ve never accepted GET with a body but I’ve been burned several times using curl to test APIs. curl and browsers are very different beasts.
- torgard 5y ago> required GET with a body Elasticsearch also encourages GET with body. But a request payload is undefined, according to the RFC: A payload within a GET request message has no defined semantics; sending a payload body on a GET request might cause some existing implementations to reject the request.
- JanLikar 5y agoIf we need GET requests with bodies, why not simply add a body to GET?
- dragonwriter 5y agoBecause then you don't know if something that supports GET supports the new broader definition or the old definition, whereas if does or does not support QUERY is more clear. Also because a different method means OPTIONS tells you information about what is supported, while overloading GET would not. And because “same guarantees” doesn't mean “means the same thing”; PUT and DELETE have the same guarantees (idempotent but not safe), but we don't use PUT with no body for DELETE.
- mrcarruthers 5y agoI'm fairly certain that (technically) nothing's stopping you from doing so. However, there are so many libraries/clients/etc... that do not allow it that it would be almost impossible to patch them all. Adding a new method and having libraries add it and support it properly would be better.
- throw_m239339 5y agocause HTTP agents are allowed to ignore the body of a get request, per spec.
- axiosgunnar 5y agoWhy not just allow bodies (that can be arbitrarily large) with GET requests?
- distalx 5y agoWould it only have affect on the way we query SQL databases or it would have an overall affect on the way we query things in general?