5 ms·

I think that a big part of the problem is that HTTP was initially designed for transfering static files and static data over potentially unauthenticated connect

I think that a big part of the problem is that HTTP was initially designed for transfering static files and static data over potentially unauthenticated connections. It wasn't designed to deal with application data that is complex and changes frequently with many concurrent users. I think that HTTP has been popular for so long that many people just refuse to admit that it's not suitable for handling complex dynamic data. They will go to desperate extremes to try to make it work; e.g. GraphQL over HTTP or Introspected REST. To deal with complex, fast changing data, we need to step away from HTTP and look towards simpler bidirectional connections as a starting point (e.g. WebSockets).

elcomet 8y ago
I'm not so sure about this. The methods POST and PUT for example reflect the intention of updating data on a server. Also, HTTP is just a transfer protocol, I see no problem building abstractions on top of it to handle data transfers for complex applications. Could you explain with more details what properties of HTTP makes it hard to deal with complex data?
- jondubois 8y ago
  > Could you explain with more details what properties of HTTP makes it hard to deal with complex data? There are a few but one of the first that comes to mind is what happens if two users modify different fields/properties of the same resource at the same time? If you want to support concurrent editing by multiple users, you need a way to either automatically resolve conflicts or to avoid conflicts altogether. With REST over HTTP, because updates typically involve overwriting the whole resource in a single POST or PUT, one user will fully overwrite the changes made by the other user; even if they were editing different fields of the same resource.
  - spinningarrow 8y ago
    But is that a failing of HTTP? “Updates typically involve overwriting the whole resource” because of how many APIs implement updates; they could for example use PATCH instead. I believe GraphQL mutations also allow updating individual fields.
  - lucideer 8y ago
    You seem to be describing a use-case for HTTP PATCH. What is there about what you're describing that's more difficult with PATCH than alternative protocols? Unless of course you have two users directly editing the same field simultaneously, but this is a direct conflict case that any underlying implementations will have to have their own strategy for—HTTP wouldn't be any different to anything else here.
    - jondubois 8y ago
      That's true, PATCH would solve that specific writing aspect of the problem, but there is still the issue that each user would not know that the other user has changed another property; we need some kind of server push mechanism to inform users that the underlying data has changed. Also, if users update one field at a time, it's more expensive to have one HTTP request per field (considering the HTTP headers overhead and potentially having to open a new connection) rather than using one WebSocket frame per update.
      - jungturk 8y ago
        Server-sent events might be a good fit here. Note that SSE belongs to the content layer (HTML) rather than the transfer layer (HTTP) so its specifically about notifications to active clients about content changes to resources they're observing. Here's someone describing their experience with it: https://medium.com/axiomzenteam/websockets-http-2-and-sse-5c24ae4d9d96 https://medium.com/axiomzenteam/websockets-http-2-and-sse-5c...
        jondubois 8y ago
        The fact that the SSE connection is unidirectional makes it impossible to quickly and efficiently detect disconnection from the server-side; this costs extra in terms of performance. Also, SSEs make it essentially impossible to implement features like real-time user online/offline presence and automatic renewal of authentication tokens (without closing the previous real-time/SSE connection).
        8y ago
  - drchickensalad 8y ago
    Other than patch, there's also etags and match headers that are frequently used and well supported.
  - rasteau 8y ago
    > what happens if two users modify different fields/properties of the same resource at the same time? https://tools.ietf.org/html/rfc7231#section-6.5.8 https://tools.ietf.org/html/rfc7231#section-6.5.8 6.5.8. 409 Conflict The 409 (Conflict) status code indicates that the request could not be completed due to a conflict with the current state of the target resource. This code is used in situations where the user might be able to resolve the conflict and resubmit the request. The server SHOULD generate a payload that includes enough information for a user to recognize the source of the conflict. Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the representation being PUT included changes to a resource that conflict with those made by an earlier (third-party) request, the origin server might use a 409 response to indicate that it can't complete the request. In this case, the response representation would likely contain information useful for merging the differences based on the revision history. Conflict resolution is inherently a consequence of the application-specific semantics of a "conflict". If the notion of "conflict" exists -- i.e., it is not obviated by application design[1] -- then conflict resolution is pushed to the caller. That said, it may be reasonable to develop reusable conflict resolution mechanisms for reusable definitions of "conflict", allowing for composition of separate concerns as needed. This seems preferable to inventing a new system that binds concerns together with a single, irrevocable notion of "conflict". [1] Using PATCH does not obviate conflicts that the application design otherwise allows.
- e12e 8y ago
  HTTP is inherently stateless. I think it was probably a mistake to not more clearly change this with the move to encrypted transport - or rather, I guess web sockets are a better evolution of http/1.1 than just wrapping http/1.1 in tls. Without state there isn't any great way to do transactional changes, without some added state; if you say get me a list of users, then update user x, y and z; there's no way to make sure you're not working on stale data. You could run the logic on the server (send js to execute in a transaction), or you could have protocol level support (like with sql: begin...commit/rollback). But rest/http will always be a document store, and it will always be challenging to maintain data integrity accross documents/resources. It's not what it was designed for.
  - LolNoGenerics 8y ago
    If you want transaction like behaviour you can simulate it. Not sure if that is even remotely a point with Web APIs.
    - e12e 8y ago
      Web api's are a lot of things. Some are transactional, like booking systems, inventory systems, banking systems, web shops... Ed: however the thing with tls is that it is session based, and it's probably a good idea to surface that state to the application, so you had a connection-based transport, and you could say: "I know this session, it's encrypted, and I've flagged it as authenticated to this user, and can map that to authorization" - rather than have a cookie that can get stolen rather easily. You might still hijack an encrypted session of course, but it should be a bit more tricky.
yenwel 8y ago
this guy agrees with you https://www.tbray.org/ongoing/When/201x/2018/11/18/Post-REST https://www.tbray.org/ongoing/When/201x/2018/11/18/Post-REST
e12e 8y ago
> I think that HTTP has been popular for so long that many people just refuse to admit that it's not suitable for handling complex dynamic data. This exactly why I keep trying to get people to go and read Fielding's REST thesis - it starts off with describing different architectures, and lands on REST for hypertext specifically because it enables things like multi level caching etc. When you throw that out, other, different architectures make more sense! It's all right there in chapter 3 (but seriously, do read the whole thing - it's surprisingly accessible for a PhD. Thesis) : https://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch_styles.htm#sec_3_9 https://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch...
rasteau 8y ago
>HTTP was initially designed for transfering static files and static data over potentially unauthenticated connections. It may be more useful to view "transfering static files and static data over potentially unauthenticated connections" as the first application of HTTP. While there are parts of HTTP that are useful for static data transfer (e.g., Cache-Control), there are parts of HTTP that would have no relevance if that were the only use case (e.g., 409 Conflict). >it's not suitable for handling complex dynamic data I would agree that HTTP was not designed to support applications pushing data from server to client, but that is a communication directionality issue separate from the issue of whether the data itself is complex or dynamic.
jayd16 8y ago
How does a websocket solve this? You still have multiple commands going through your websocket which could still conflict with another user's actions. If you make the actions atomic, you could use REST. If you use a transaction, you could use REST even still. Letting client code consume locks is frowned upon, but doing it through a websocket doesn't make it better. My point is REST is the worst solution except for all the other solutions.

elcomet 8y ago

I'm not so sure about this. The methods POST and PUT for example reflect the intention of updating data on a server. Also, HTTP is just a transfer protocol, I see no problem building abstractions on top of it to handle data transfers for complex applications. Could you explain with more details what properties of HTTP makes it hard to deal with complex data?

jondubois 8y ago
> Could you explain with more details what properties of HTTP makes it hard to deal with complex data? There are a few but one of the first that comes to mind is what happens if two users modify different fields/properties of the same resource at the same time? If you want to support concurrent editing by multiple users, you need a way to either automatically resolve conflicts or to avoid conflicts altogether. With REST over HTTP, because updates typically involve overwriting the whole resource in a single POST or PUT, one user will fully overwrite the changes made by the other user; even if they were editing different fields of the same resource.
- spinningarrow 8y ago
  But is that a failing of HTTP? “Updates typically involve overwriting the whole resource” because of how many APIs implement updates; they could for example use PATCH instead. I believe GraphQL mutations also allow updating individual fields.
- lucideer 8y ago
  You seem to be describing a use-case for HTTP PATCH. What is there about what you're describing that's more difficult with PATCH than alternative protocols? Unless of course you have two users directly editing the same field simultaneously, but this is a direct conflict case that any underlying implementations will have to have their own strategy for—HTTP wouldn't be any different to anything else here.
  - jondubois 8y ago
    That's true, PATCH would solve that specific writing aspect of the problem, but there is still the issue that each user would not know that the other user has changed another property; we need some kind of server push mechanism to inform users that the underlying data has changed. Also, if users update one field at a time, it's more expensive to have one HTTP request per field (considering the HTTP headers overhead and potentially having to open a new connection) rather than using one WebSocket frame per update.
    - jungturk 8y ago
      Server-sent events might be a good fit here. Note that SSE belongs to the content layer (HTML) rather than the transfer layer (HTTP) so its specifically about notifications to active clients about content changes to resources they're observing. Here's someone describing their experience with it: https://medium.com/axiomzenteam/websockets-http-2-and-sse-5c24ae4d9d96 https://medium.com/axiomzenteam/websockets-http-2-and-sse-5c...
      - jondubois 8y ago
        The fact that the SSE connection is unidirectional makes it impossible to quickly and efficiently detect disconnection from the server-side; this costs extra in terms of performance. Also, SSEs make it essentially impossible to implement features like real-time user online/offline presence and automatic renewal of authentication tokens (without closing the previous real-time/SSE connection).
        8y ago
- drchickensalad 8y ago
  Other than patch, there's also etags and match headers that are frequently used and well supported.
- rasteau 8y ago
  > what happens if two users modify different fields/properties of the same resource at the same time? https://tools.ietf.org/html/rfc7231#section-6.5.8 https://tools.ietf.org/html/rfc7231#section-6.5.8 6.5.8. 409 Conflict The 409 (Conflict) status code indicates that the request could not be completed due to a conflict with the current state of the target resource. This code is used in situations where the user might be able to resolve the conflict and resubmit the request. The server SHOULD generate a payload that includes enough information for a user to recognize the source of the conflict. Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the representation being PUT included changes to a resource that conflict with those made by an earlier (third-party) request, the origin server might use a 409 response to indicate that it can't complete the request. In this case, the response representation would likely contain information useful for merging the differences based on the revision history. Conflict resolution is inherently a consequence of the application-specific semantics of a "conflict". If the notion of "conflict" exists -- i.e., it is not obviated by application design[1] -- then conflict resolution is pushed to the caller. That said, it may be reasonable to develop reusable conflict resolution mechanisms for reusable definitions of "conflict", allowing for composition of separate concerns as needed. This seems preferable to inventing a new system that binds concerns together with a single, irrevocable notion of "conflict". [1] Using PATCH does not obviate conflicts that the application design otherwise allows.
e12e 8y ago
HTTP is inherently stateless. I think it was probably a mistake to not more clearly change this with the move to encrypted transport - or rather, I guess web sockets are a better evolution of http/1.1 than just wrapping http/1.1 in tls. Without state there isn't any great way to do transactional changes, without some added state; if you say get me a list of users, then update user x, y and z; there's no way to make sure you're not working on stale data. You could run the logic on the server (send js to execute in a transaction), or you could have protocol level support (like with sql: begin...commit/rollback). But rest/http will always be a document store, and it will always be challenging to maintain data integrity accross documents/resources. It's not what it was designed for.
- LolNoGenerics 8y ago
  If you want transaction like behaviour you can simulate it. Not sure if that is even remotely a point with Web APIs.
  - e12e 8y ago
    Web api's are a lot of things. Some are transactional, like booking systems, inventory systems, banking systems, web shops... Ed: however the thing with tls is that it is session based, and it's probably a good idea to surface that state to the application, so you had a connection-based transport, and you could say: "I know this session, it's encrypted, and I've flagged it as authenticated to this user, and can map that to authorization" - rather than have a cookie that can get stolen rather easily. You might still hijack an encrypted session of course, but it should be a bit more tricky.

yenwel 8y ago

this guy agrees with you https://www.tbray.org/ongoing/When/201x/2018/11/18/Post-REST https://www.tbray.org/ongoing/When/201x/2018/11/18/Post-REST

e12e 8y ago

> I think that HTTP has been popular for so long that many people just refuse to admit that it's not suitable for handling complex dynamic data. This exactly why I keep trying to get people to go and read Fielding's REST thesis - it starts off with describing different architectures, and lands on REST for hypertext specifically because it enables things like multi level caching etc. When you throw that out, other, different architectures make more sense! It's all right there in chapter 3 (but seriously, do read the whole thing - it's surprisingly accessible for a PhD. Thesis) : https://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch_styles.htm#sec_3_9 https://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch...

rasteau 8y ago

>HTTP was initially designed for transfering static files and static data over potentially unauthenticated connections. It may be more useful to view "transfering static files and static data over potentially unauthenticated connections" as the first application of HTTP. While there are parts of HTTP that are useful for static data transfer (e.g., Cache-Control), there are parts of HTTP that would have no relevance if that were the only use case (e.g., 409 Conflict). >it's not suitable for handling complex dynamic data I would agree that HTTP was not designed to support applications pushing data from server to client, but that is a communication directionality issue separate from the issue of whether the data itself is complex or dynamic.

jayd16 8y ago

How does a websocket solve this? You still have multiple commands going through your websocket which could still conflict with another user's actions. If you make the actions atomic, you could use REST. If you use a transaction, you could use REST even still. Letting client code consume locks is frowned upon, but doing it through a websocket doesn't make it better. My point is REST is the worst solution except for all the other solutions.