7 ms·

There should be an option for deleting many objects. For that, you have to do a POST request on AWS S3: https://docs.aws.amazon.com/AmazonS3/latest/API/API_Del

There should be an option for deleting many objects. For that, you have to do a POST request on AWS S3: https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteOb... Instead of polluting existing method names to mean new things, HTTP could offer more method names. Having POST alternatively mean "send a command" makes it meaningless. The command could do anything.

treve 7y ago
What advantage is there to having a bulk operation vs many operations? With HTTP2 you can send a ton of requests in parallel.
- JMTQp8lwXL 7y ago
  A lot of sites might have an HTTP/2 upstream server, but everything downstream is still communicating via HTTP/1.1. I would imagine you can only the benefits of HTTP/2 multiplexing if you fully utilize HTTP/2 end-to-end. That means your CDN, your load balancers, and finally your actual applications-- any any other intermediaries -- all communicating via HTTP/2. Not to say that HTTP/2 isn't the solution, but it's going to be slower than looking up "number of sites using HTTP/2", since you can't easily inspect the behavior of the intermediary servers.
  - treve 7y ago
    Getting onto H2 is worth investing in then though, and drastically simplifies things that would otherwise need bulk operations everywhere. I'm not entirely sure what you mean with downstream, but in my experience H2 is really well supported almost everywhere these days. Ymmv ofc
    - JMTQp8lwXL 7y ago
      What I mean is, typically, there are many layers between an end user's network request and reaching the server where it will be fulfilled. For example, you might have your site fronted by Cloudflare. They provide DDOS protection, etc. That Cloudflare server might accept HTTP/2 requests, and then open a new request to your load balancer server. Then, your load balancer opens another network request to the server fulfilling the request. The connections between these servers may only support HTTP/1.1. So you lose the benefits of HTTP/2 since you can't multiplex the network request end-to-end.
      - treve 7y ago
        In this particular example (S3), I don't think this is really a concern. Anyway, I'm curious... what components do you run into that don't support this yet? At least in your examples I can't think of any major players that don't do HTTP/2 out of the box, except, well, old versions. Is this more of a hypothetical? In my experience HTTP/2 is pretty much ubiquitous for anything current.
        bebop 7y ago
        There are environments that do not allow http2 or websocket traffic on the network. Most government/big enterprise have these restrictions.
  - felixfbecker 7y ago
    The load balancer could append a header though with a session ID that is the same for the same HTTP/2 session, and then the server can batch based on the session ID and timeframe.
- coldtea 7y ago
  >What advantage is there to having a bulk operation vs many operations? For one, having it be transactional. >With HTTP2 you can send a ton of requests in parallel. That's orthogonal. And they still have a cost (in time and size of transfer).
  - sshumaker 7y ago
    Http2 has efficiency gains at the transport later but not typically at the application tier. Having a single http request lets you things like handling authentication, fetching common state (user object), etc a single time for a batch request. Http2 doesn’t help you here.
- yxhuvud 7y ago
  Have you ever heard the concept of the common antipattern of n+1 queries that the back-end developers try to avoid? What you are suggesting would create that pattern on the front-end too.
  - felixfbecker 7y ago
    The backend can batch deletes made in the same timeframe in the same HTTP/2 session, like how you can batch queries with Facebook Dataloader in GraphQL APIs to avoid the n+1 query problem.
  - treve 7y ago
    Yes, I understand the concept. I totally get that certain situations call for batching operations, including things that need to work transactional. I think though that most operations don't need this. It's not an optimization I would blindly add to any operation, unless there's a specific case that warrants it. I don't think that this is an anti-pattern. Most APIs do '1 thing at a time'.
kevin_thibedeau 7y ago
We don't need a profusion of operations in the core protocol. Do you need idempotence or not? Pick from one of these two things and elaborate on top of that.
- JMTQp8lwXL 7y ago
  Having "create many", "update many", and "delete many" is three additional operations. Profusion may not be the word I'd use. Also, I will say they are very commonly requested operations, and they've lead to POST becoming overloaded in meaning.
  - ar_lan 7y ago
    Alternatively, in a GraphQL server, everything becomes POST and it all becomes meaningless.
jayd16 7y ago
Seems like you could just use Delete with query params on the collection path. Is that not RESTful?
- JMTQp8lwXL 7y ago
  It doesn't really scale. There's a de-facto limit of 2,000 characters [0] in a URL. HTTP/1.1 is not opinionated about URL length, but practical implementations are. If your ids are UUIDs, each has a length of 36. If your URL consisted of nothing but UUIDs, you could safely delete 55.5(repeating) of them. [0]: https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers https://stackoverflow.com/questions/417142/what-is-the-maxim...
- jimbobenfrance 7y ago
  I usually implement this as Http delete with a json array of ids in the body. Probably in violation of some standard, but seems sensible enough.
richardwhiuk 7y ago
PATCH on the collection.
paulddraper 7y ago
> There should be an option for deleting many objects. What about fetching many objects or creating many objects or updating many objects?
- littlestymaar 7y ago
  That's right, that's the same issue. But in both cases, you can just send your list of items to fetch (respectively delete) in the body of the GET or DELETE request. (Because AFAIK, it's HTTP compliant to put a body in a GET request, even if it's rarely used). Edit: I was wrong. It's allowed to put a body in a GET request, but it isn't OK to use it according to the HTTP spec (which admittedly kind of weird). Source: https://stackoverflow.com/a/983458 https://stackoverflow.com/a/983458 Edit2: according to an edit int the SO response, it looks like I wasn't actually wrong: or at least not since 2014 when these RFCs[1][2] became the standard for HTTP. [1]: https://tools.ietf.org/html/rfc7231#section-4.3.1 https://tools.ietf.org/html/rfc7231#section-4.3.1 [2]: https://tools.ietf.org/html/rfc7230#section-3.3 https://tools.ietf.org/html/rfc7230#section-3.3
  - paulddraper 7y ago
    A body is and has always been syntactically permitted on any HTTP request. However, it is semantic nonsense to put it on a DELETE or GET. DELETE is defined to remove the resource identified by the URL; GET is defined to fetch the resource identified by the URL. There no purpose for an entity.
    - littlestymaar 7y ago
      This is the point made in the SO answer I quoted, and this was the official position of the rfc2616[1], but as I said in my second edit, this rule was relaxed in the new version of the specification: rfc7231 published in 2014. [1]: https://tools.ietf.org/html/rfc2616#section-9.3 https://tools.ietf.org/html/rfc2616#section-9.3 [2]: https://tools.ietf.org/html/rfc7231#section-4.3.1 https://tools.ietf.org/html/rfc7231#section-4.3.1
      - 7y ago
NicoJuicy 7y ago
There is json batch https://docs.microsoft.com/en-us/graph/json-batching https://docs.microsoft.com/en-us/graph/json-batching
felixfbecker 7y ago
If you don’t need it to be atomic, you can just make parallel requests. Should not be an issue with HTTP/2
- charrondev 7y ago
  Except now your server has to handle a lot more connections. Was recently dealing with a list of 100+ simple text fields each with unique identifying meta info. Most usage of the form would only involve updating a few at a time, but on initial entry or a large update all of them could be updated or deleted. A RESTful approach would have me submit 100+ individual PUT requests. However for a smaller instance of our server it would likely be affected by 100s of requests at a time and it would significantly complicate the client code to make all of these requests at a time and do the error handling for it. It’s actually a small payload, even with 100 items, so it’s ridiculous to send them individual in order to be “correct” if it - leads to a worse user experience. - increases load on the server. - complicates the codebase.
deleted 7y ago
[deleted]

treve 7y ago

What advantage is there to having a bulk operation vs many operations? With HTTP2 you can send a ton of requests in parallel.

JMTQp8lwXL 7y ago
A lot of sites might have an HTTP/2 upstream server, but everything downstream is still communicating via HTTP/1.1. I would imagine you can only the benefits of HTTP/2 multiplexing if you fully utilize HTTP/2 end-to-end. That means your CDN, your load balancers, and finally your actual applications-- any any other intermediaries -- all communicating via HTTP/2. Not to say that HTTP/2 isn't the solution, but it's going to be slower than looking up "number of sites using HTTP/2", since you can't easily inspect the behavior of the intermediary servers.
- treve 7y ago
  Getting onto H2 is worth investing in then though, and drastically simplifies things that would otherwise need bulk operations everywhere. I'm not entirely sure what you mean with downstream, but in my experience H2 is really well supported almost everywhere these days. Ymmv ofc
  - JMTQp8lwXL 7y ago
    What I mean is, typically, there are many layers between an end user's network request and reaching the server where it will be fulfilled. For example, you might have your site fronted by Cloudflare. They provide DDOS protection, etc. That Cloudflare server might accept HTTP/2 requests, and then open a new request to your load balancer server. Then, your load balancer opens another network request to the server fulfilling the request. The connections between these servers may only support HTTP/1.1. So you lose the benefits of HTTP/2 since you can't multiplex the network request end-to-end.
    - treve 7y ago
      In this particular example (S3), I don't think this is really a concern. Anyway, I'm curious... what components do you run into that don't support this yet? At least in your examples I can't think of any major players that don't do HTTP/2 out of the box, except, well, old versions. Is this more of a hypothetical? In my experience HTTP/2 is pretty much ubiquitous for anything current.
      - bebop 7y ago
        There are environments that do not allow http2 or websocket traffic on the network. Most government/big enterprise have these restrictions.
- felixfbecker 7y ago
  The load balancer could append a header though with a session ID that is the same for the same HTTP/2 session, and then the server can batch based on the session ID and timeframe.
coldtea 7y ago
>What advantage is there to having a bulk operation vs many operations? For one, having it be transactional. >With HTTP2 you can send a ton of requests in parallel. That's orthogonal. And they still have a cost (in time and size of transfer).
- sshumaker 7y ago
  Http2 has efficiency gains at the transport later but not typically at the application tier. Having a single http request lets you things like handling authentication, fetching common state (user object), etc a single time for a batch request. Http2 doesn’t help you here.
yxhuvud 7y ago
Have you ever heard the concept of the common antipattern of n+1 queries that the back-end developers try to avoid? What you are suggesting would create that pattern on the front-end too.
- felixfbecker 7y ago
  The backend can batch deletes made in the same timeframe in the same HTTP/2 session, like how you can batch queries with Facebook Dataloader in GraphQL APIs to avoid the n+1 query problem.
- treve 7y ago
  Yes, I understand the concept. I totally get that certain situations call for batching operations, including things that need to work transactional. I think though that most operations don't need this. It's not an optimization I would blindly add to any operation, unless there's a specific case that warrants it. I don't think that this is an anti-pattern. Most APIs do '1 thing at a time'.

kevin_thibedeau 7y ago

We don't need a profusion of operations in the core protocol. Do you need idempotence or not? Pick from one of these two things and elaborate on top of that.

JMTQp8lwXL 7y ago
Having "create many", "update many", and "delete many" is three additional operations. Profusion may not be the word I'd use. Also, I will say they are very commonly requested operations, and they've lead to POST becoming overloaded in meaning.
- ar_lan 7y ago
  Alternatively, in a GraphQL server, everything becomes POST and it all becomes meaningless.

jayd16 7y ago

Seems like you could just use Delete with query params on the collection path. Is that not RESTful?

JMTQp8lwXL 7y ago
It doesn't really scale. There's a de-facto limit of 2,000 characters [0] in a URL. HTTP/1.1 is not opinionated about URL length, but practical implementations are. If your ids are UUIDs, each has a length of 36. If your URL consisted of nothing but UUIDs, you could safely delete 55.5(repeating) of them. [0]: https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers https://stackoverflow.com/questions/417142/what-is-the-maxim...
jimbobenfrance 7y ago
I usually implement this as Http delete with a json array of ids in the body. Probably in violation of some standard, but seems sensible enough.

richardwhiuk 7y ago

PATCH on the collection.

paulddraper 7y ago

> There should be an option for deleting many objects. What about fetching many objects or creating many objects or updating many objects?

littlestymaar 7y ago
That's right, that's the same issue. But in both cases, you can just send your list of items to fetch (respectively delete) in the body of the GET or DELETE request. (Because AFAIK, it's HTTP compliant to put a body in a GET request, even if it's rarely used). Edit: I was wrong. It's allowed to put a body in a GET request, but it isn't OK to use it according to the HTTP spec (which admittedly kind of weird). Source: https://stackoverflow.com/a/983458 https://stackoverflow.com/a/983458 Edit2: according to an edit int the SO response, it looks like I wasn't actually wrong: or at least not since 2014 when these RFCs[1][2] became the standard for HTTP. [1]: https://tools.ietf.org/html/rfc7231#section-4.3.1 https://tools.ietf.org/html/rfc7231#section-4.3.1 [2]: https://tools.ietf.org/html/rfc7230#section-3.3 https://tools.ietf.org/html/rfc7230#section-3.3
- paulddraper 7y ago
  A body is and has always been syntactically permitted on any HTTP request. However, it is semantic nonsense to put it on a DELETE or GET. DELETE is defined to remove the resource identified by the URL; GET is defined to fetch the resource identified by the URL. There no purpose for an entity.
  - littlestymaar 7y ago
    This is the point made in the SO answer I quoted, and this was the official position of the rfc2616[1], but as I said in my second edit, this rule was relaxed in the new version of the specification: rfc7231 published in 2014. [1]: https://tools.ietf.org/html/rfc2616#section-9.3 https://tools.ietf.org/html/rfc2616#section-9.3 [2]: https://tools.ietf.org/html/rfc7231#section-4.3.1 https://tools.ietf.org/html/rfc7231#section-4.3.1
    - 7y ago

NicoJuicy 7y ago

There is json batch https://docs.microsoft.com/en-us/graph/json-batching https://docs.microsoft.com/en-us/graph/json-batching

felixfbecker 7y ago

If you don’t need it to be atomic, you can just make parallel requests. Should not be an issue with HTTP/2

charrondev 7y ago
Except now your server has to handle a lot more connections. Was recently dealing with a list of 100+ simple text fields each with unique identifying meta info. Most usage of the form would only involve updating a few at a time, but on initial entry or a large update all of them could be updated or deleted. A RESTful approach would have me submit 100+ individual PUT requests. However for a smaller instance of our server it would likely be affected by 100s of requests at a time and it would significantly complicate the client code to make all of these requests at a time and do the error handling for it. It’s actually a small payload, even with 100 items, so it’s ridiculous to send them individual in order to be “correct” if it - leads to a worse user experience. - increases load on the server. - complicates the codebase.

deleted 7y ago

[deleted]