5 ms·

REST for the control plane works because there are few enough "resources" that you can have a URI for each and CRUD works. Much more importantly, the pressure

REST for the control plane works because there are few enough "resources" that you can have a URI for each and CRUD works. Much more importantly, the pressure to have complex multi-resource transactions is low because the control plane tends to have simple schemas and there is no need to do complex multi-resource transactions with any kind of atomicity. The data plane is not at all like that. There can be many many resources, and very complex transactions. PostgREST, a REST interface to PostgreSQL, shows you can have REST for the data plane, but it achieves that by forcing one to have simple transactions with any and all complexity wrapped into PG functions so that the REST interface can remain simple. The place where REST falls down, really, is transactions. For simple enough schemas and transactions involving only one relation, it's easy enough to have URIs for resources and collections. But for anything much more complex than that REST simply fails. Instead one ends up writing an end-point that accepts POSTs whose request bodies encode complex transactions, and this ends up resembling SOAP (basically RPC over HTTP) rather than REST. Imagine an HTTP/1.1 verb named TRANSACT whose body consists of a concatenation of the requests that you really want to do, and with a header controlling / reflecting transaction control options (e.g., atomicity). If HTTP had such a thing, then it would be a lot easier to build a proper RESTful interface to the data plane. But HTTP has no such thing, and probably never will. Incidentally, it's useful to consider how NFSv3 had RPCs for every little thing, while NFSv4 has only one RPC (COMPOUND) that is... a lot like the TRANSACT I imagine above -- it's just not that crazy a thing to imagine. The transaction complexity problem in HTTP is not unique to HTTP!

dragonwriter 8y ago
> The place where REST falls down, really, is transactions. For simple enough schemas and transactions involving only one relation, it's easy enough to have URIs for resources and collections. But for anything much more complex than that REST simply fails. Instead one ends up writing an end-point that accepts POSTs whose request bodies encode complex transactions, and this ends up resembling SOAP (basically RPC over HTTP) rather than REST. It's perfectly RESTful; resources that represents actions are just as valid as any other resources. I feel like the mistake here extends a common mistake of object-oriented analysis where the valid idea that objects should correspond to nouns are somehow combined with forgetting that actions are nouns. If you have an API with complex transactions (especially if you need to expose state of those transactions), why wouldn't they be resources? Why wouldn't you post a representation to a collection endpoint? How is that not exactly REST?
- cryptonector 8y ago
  > It's perfectly RESTful; ... Yeah, so this is where just what the meaning of REST is comes in. There's no representation of the "transaction" resource that you can GET, therefore they are not like the other resources (the ones you can HEAD/GET). You can only POST representations of transactions that internally contain ad-hoc encodings of CRUD operations on multiple other resources. This really feels like it should have been a first-class operation in HTTP rather than constructed in an ad-hoc way as needed.
  - niftich 8y ago
    If you design in this model, 'GET transaction-uri' should get, y'know, the payload of the transaction, or a status resource, or whichever you asked for based on proactive or reactive content negotiation, or whichever the server wishes to make available. It's just such a mental departure from the last 20 years of 'do-the-verb-to-the-noun' object oriented design that it's not often done.
    - cryptonector 8y ago
      Sure, but it's still not like a resource, and you still have to write code to implement it. You could never have a user-agent construct a transaction because the construction is completely ad-hoc and will differ by application. It's just not generic.
      - dragonwriter 8y ago
        > Sure, but it's still not like a resource It is exactly a resource. > and you still have to write code to implement it. Well, someone, somewhere has to, just like any other resource. > You could never have a user-agent construct a transaction because the construction is completely ad-hoc and will differ by application. This is not true in any sense where the same is not true of lots of resources. > It's just not generic. While REST may favor using suitable generic representations where available, being generic isn't a requirement of REST. REST is DRY, not “Don’t write code for anything.”
denormalfloat 8y ago
> Imagine an HTTP/1.1 verb named TRANSACT Why not just use RPC at that point? If one of the concatenated messages has a failure, REST can only report back an error code for one of them. Also, the client can't really stop sending once it has started. The whole connection would have to be torn down to cancel out early. Also, transactions usually involve both read and writes. Doing a stream of writes that are not conditioned on some reads is often not that useful. (I.e. check if this account has money, and then transfer to some other account).
- cryptonector 8y ago
  For one thing because the CRUD structure enables a bunch of things. For example, suppose you wanted to front a service with a reverse proxy that performs authorization: if the verbs and resources encode all you need to authorize supplicants, then you could do it generically, otherwise your proxy would have to understand the application's resources in much more detail. Of course, you can do this with RPC as well. But HTTP is so... widely used, while RPC frameworks are a) a dime a dozen, b) none so widely used that you can find reverse proxies for them. > > Imagine an HTTP/1.1 verb named TRANSACT > > [...] If one of the concatenated messages has a failure, REST can only report back an error code for one of them. [...] REST as it is can report whatever you want because the encoding of transactions is ad-hoc, so you just have to make a provision for conveying all the error information that you want. A TRANSACT verb does not exist, but if it did it could easily returns a status code, message, and error response body for each operation in the transaction.
  - denormalfloat 8y ago
    > But HTTP is so... widely used, while RPC frameworks are a) a dime a dozen, b) none so widely used that you can find reverse proxies for them. The beauty of gRPC is that it is HTTP. All proxies that support HTTP/2 implicitly support gRPC. > REST as it is can report whatever you want because the encoding of transactions is ad-hoc, so you just have to make a provision for conveying all the error information that you want. But at that point it isn't really REST. The single request-single response semantics are lost. The error codes don't map cleanly to the operations. In some cases they do, but not generally.
hodgesrm 8y ago
> REST for the control plane works because there are few enough "resources" that you can have a URI for each and CRUD works. Isn't the issue also that entities on the control plane often have relatively simple state machines? You can reduce REST complexity by designing APIs that manipulate the state machine rather than thinking in terms of a pure CRUD model. For instance, the need to hold client-side state disappears in transaction models where you request a desired state and then have the option to wait for the actual state to correspond or just go away and check later to see how it's doing. The poll request (if there is one) translates from looking for transaction completion to waiting for the desired state. Edit: for clarity
- cryptonector 8y ago
  I referred to schema simplicity elsewhere in that comment, which is not the same thing, but similar enough to state machine simplicity.
ako 8y ago
Combining multiple changes in one request is actually part of the oData standard (which is REST): http://docs.oasis-open.org/odata/odata/v4.0/errata02/os/complete/part1-protocol/odata-v4.0-errata02-os-part1-protocol-complete.html#_Toc406398359 http://docs.oasis-open.org/odata/odata/v4.0/errata02/os/comp... You can send multiple retrieves and changes in one multipart request.
steve-chavez 8y ago
Just FYI, there's a pending enhancement in PostgREST[1] that would allow us to take advantage of HTTP/2 multiplexing and have multiple requests executed atomically in a single transaction. [1]:https://github.com/PostgREST/postgrest/issues/286 https://github.com/PostgREST/postgrest/issues/286
- cryptonector 8y ago
  Neat!
cryptonector 8y ago
Another thing, regarding eventing, is that you can use HTTP for that: just use chunked transfer encoding in never-ending responses. I've written a tailfhttpd that does this for tailing files a-la tail(1) -f, as you'd expect. You want to have some sort of application-level heartbeat though because HTTP/1.1 chunked transfer encoding sure doesn't have that, but otherwise you can absolutely sit there waiting for events that come along over HTTP, and this can be extremely fast (my tailfhttpd is C10K).
geezerjay 8y ago
> The place where REST falls down, really, is transactions. I don't agree. The standard way of supporting transactions with REST is to provide a transactions endpoint where clients start a transaction by POSTing a new transaction resource, POST their DTOs to build up a unit of work, and when they are done just commit the transaction. That's pretty much a one-size-fits-all solution as that's the standard way to implement transactions at the database level.
- dustfinger 8y ago
  I was going to say the same thing and was pleased to see that someone beat me to it. If you combine unit of work pattern with continuation tokens then you can handle bulk transactions involving very large numbers of records with a simple rest interface.
- cryptonector 8y ago
  Yes, and I've said as much. But the user-agents can't construct this without JS.
Dowwie 8y ago
Would you go through a concrete example?
- cryptonector 8y ago
  Sure. I'll use PostgREST, which has a RESTful data plane interface to PostgreSQL. There's no UI as such, though you can build one with, e.g., Admin-on-REST. There's only trivial transaction support, even though there's some non-trivial query support. All transactions consist of a POST to update one row or insert many rows all on the same table, or a DELETE to delete rows. All non-triviality has to be done via functions or views with INSTEAD OF triggers. But now imagine you could construct a transaction like this using curl as an example: $ curl -X DELETE https://pgrest.foo.example/sometable/somerowid -- -X PUT https://pgrest.foo.example/sometable/someotherrowid -d @file_containing_row_data -- ... If you use one curl(1) invocation per operation then you get no transactional capabilities. But you can't do it in one request because HTTP itself has no transactional capabilities. Instead you have to construct a single request whose body encodes all those requests in some PostgREST-specific manner (as opposed to some generic and standard manner that all applications could use).

dragonwriter 8y ago

> The place where REST falls down, really, is transactions. For simple enough schemas and transactions involving only one relation, it's easy enough to have URIs for resources and collections. But for anything much more complex than that REST simply fails. Instead one ends up writing an end-point that accepts POSTs whose request bodies encode complex transactions, and this ends up resembling SOAP (basically RPC over HTTP) rather than REST. It's perfectly RESTful; resources that represents actions are just as valid as any other resources. I feel like the mistake here extends a common mistake of object-oriented analysis where the valid idea that objects should correspond to nouns are somehow combined with forgetting that actions are nouns. If you have an API with complex transactions (especially if you need to expose state of those transactions), why wouldn't they be resources? Why wouldn't you post a representation to a collection endpoint? How is that not exactly REST?

cryptonector 8y ago
> It's perfectly RESTful; ... Yeah, so this is where just what the meaning of REST is comes in. There's no representation of the "transaction" resource that you can GET, therefore they are not like the other resources (the ones you can HEAD/GET). You can only POST representations of transactions that internally contain ad-hoc encodings of CRUD operations on multiple other resources. This really feels like it should have been a first-class operation in HTTP rather than constructed in an ad-hoc way as needed.
- niftich 8y ago
  If you design in this model, 'GET transaction-uri' should get, y'know, the payload of the transaction, or a status resource, or whichever you asked for based on proactive or reactive content negotiation, or whichever the server wishes to make available. It's just such a mental departure from the last 20 years of 'do-the-verb-to-the-noun' object oriented design that it's not often done.
  - cryptonector 8y ago
    Sure, but it's still not like a resource, and you still have to write code to implement it. You could never have a user-agent construct a transaction because the construction is completely ad-hoc and will differ by application. It's just not generic.
    - dragonwriter 8y ago
      > Sure, but it's still not like a resource It is exactly a resource. > and you still have to write code to implement it. Well, someone, somewhere has to, just like any other resource. > You could never have a user-agent construct a transaction because the construction is completely ad-hoc and will differ by application. This is not true in any sense where the same is not true of lots of resources. > It's just not generic. While REST may favor using suitable generic representations where available, being generic isn't a requirement of REST. REST is DRY, not “Don’t write code for anything.”

denormalfloat 8y ago

> Imagine an HTTP/1.1 verb named TRANSACT Why not just use RPC at that point? If one of the concatenated messages has a failure, REST can only report back an error code for one of them. Also, the client can't really stop sending once it has started. The whole connection would have to be torn down to cancel out early. Also, transactions usually involve both read and writes. Doing a stream of writes that are not conditioned on some reads is often not that useful. (I.e. check if this account has money, and then transfer to some other account).

cryptonector 8y ago
For one thing because the CRUD structure enables a bunch of things. For example, suppose you wanted to front a service with a reverse proxy that performs authorization: if the verbs and resources encode all you need to authorize supplicants, then you could do it generically, otherwise your proxy would have to understand the application's resources in much more detail. Of course, you can do this with RPC as well. But HTTP is so... widely used, while RPC frameworks are a) a dime a dozen, b) none so widely used that you can find reverse proxies for them. > > Imagine an HTTP/1.1 verb named TRANSACT > > [...] If one of the concatenated messages has a failure, REST can only report back an error code for one of them. [...] REST as it is can report whatever you want because the encoding of transactions is ad-hoc, so you just have to make a provision for conveying all the error information that you want. A TRANSACT verb does not exist, but if it did it could easily returns a status code, message, and error response body for each operation in the transaction.
- denormalfloat 8y ago
  > But HTTP is so... widely used, while RPC frameworks are a) a dime a dozen, b) none so widely used that you can find reverse proxies for them. The beauty of gRPC is that it is HTTP. All proxies that support HTTP/2 implicitly support gRPC. > REST as it is can report whatever you want because the encoding of transactions is ad-hoc, so you just have to make a provision for conveying all the error information that you want. But at that point it isn't really REST. The single request-single response semantics are lost. The error codes don't map cleanly to the operations. In some cases they do, but not generally.

hodgesrm 8y ago

> REST for the control plane works because there are few enough "resources" that you can have a URI for each and CRUD works. Isn't the issue also that entities on the control plane often have relatively simple state machines? You can reduce REST complexity by designing APIs that manipulate the state machine rather than thinking in terms of a pure CRUD model. For instance, the need to hold client-side state disappears in transaction models where you request a desired state and then have the option to wait for the actual state to correspond or just go away and check later to see how it's doing. The poll request (if there is one) translates from looking for transaction completion to waiting for the desired state. Edit: for clarity

cryptonector 8y ago
I referred to schema simplicity elsewhere in that comment, which is not the same thing, but similar enough to state machine simplicity.

ako 8y ago

Combining multiple changes in one request is actually part of the oData standard (which is REST): http://docs.oasis-open.org/odata/odata/v4.0/errata02/os/complete/part1-protocol/odata-v4.0-errata02-os-part1-protocol-complete.html#_Toc406398359 http://docs.oasis-open.org/odata/odata/v4.0/errata02/os/comp... You can send multiple retrieves and changes in one multipart request.

steve-chavez 8y ago

Just FYI, there's a pending enhancement in PostgREST[1] that would allow us to take advantage of HTTP/2 multiplexing and have multiple requests executed atomically in a single transaction. [1]:https://github.com/PostgREST/postgrest/issues/286 https://github.com/PostgREST/postgrest/issues/286

cryptonector 8y ago
Neat!

cryptonector 8y ago

Another thing, regarding eventing, is that you can use HTTP for that: just use chunked transfer encoding in never-ending responses. I've written a tailfhttpd that does this for tailing files a-la tail(1) -f, as you'd expect. You want to have some sort of application-level heartbeat though because HTTP/1.1 chunked transfer encoding sure doesn't have that, but otherwise you can absolutely sit there waiting for events that come along over HTTP, and this can be extremely fast (my tailfhttpd is C10K).

geezerjay 8y ago

> The place where REST falls down, really, is transactions. I don't agree. The standard way of supporting transactions with REST is to provide a transactions endpoint where clients start a transaction by POSTing a new transaction resource, POST their DTOs to build up a unit of work, and when they are done just commit the transaction. That's pretty much a one-size-fits-all solution as that's the standard way to implement transactions at the database level.

dustfinger 8y ago
I was going to say the same thing and was pleased to see that someone beat me to it. If you combine unit of work pattern with continuation tokens then you can handle bulk transactions involving very large numbers of records with a simple rest interface.
cryptonector 8y ago
Yes, and I've said as much. But the user-agents can't construct this without JS.

Dowwie 8y ago

Would you go through a concrete example?

cryptonector 8y ago
Sure. I'll use PostgREST, which has a RESTful data plane interface to PostgreSQL. There's no UI as such, though you can build one with, e.g., Admin-on-REST. There's only trivial transaction support, even though there's some non-trivial query support. All transactions consist of a POST to update one row or insert many rows all on the same table, or a DELETE to delete rows. All non-triviality has to be done via functions or views with INSTEAD OF triggers. But now imagine you could construct a transaction like this using curl as an example: $ curl -X DELETE https://pgrest.foo.example/sometable/somerowid -- -X PUT https://pgrest.foo.example/sometable/someotherrowid -d @file_containing_row_data -- ... If you use one curl(1) invocation per operation then you get no transactional capabilities. But you can't do it in one request because HTTP itself has no transactional capabilities. Instead you have to construct a single request whose body encodes all those requests in some PostgREST-specific manner (as opposed to some generic and standard manner that all applications could use).