4 ms·

But now the remote client has an ID of something that may or may not exist the next time they try to use it depending on whether or not it actually made its way

But now the remote client has an ID of something that may or may not exist the next time they try to use it depending on whether or not it actually made its way into the database. I've seen this kind of architecture before. It sounds nice but is loaded with consistency problems.

NicoJuicy 5y ago
That's why the pattern is eventual consistency. You receive a message "{entity}Created" and it contains the Id of the full object.
- VWWHFSfQ 5y ago
  It's like, _maybe_ eventual consistency. Hopefully the client doesn't try to do anything important with the ID/new object
  - nchi3 5y ago
    If you get a "Created" back, it is possible for it to be guaranteed that it'll be created eventually.
    - VWWHFSfQ 5y ago
      I don't think you would ever get a 201 Created in an eventual consistency scenario. You would get a 202 Accepted. Unless the API is lying to the clients.
      - nchi3 5y ago
        Fair enough, though what I said remains true for an "Accepted" response.
        NicoJuicy 5y ago
        But Accepted doesn't mean created. The feedback flow is async from the creation flow. If you get created through the feedback flow it's created.
        nchi3 5y ago
        Accepted can mean that it will eventually be created though. In the sense that it's validated and guaranteed to be created at a later point ("eventually").
      - nijave 5y ago
        Created can mean different things to different people. Maybe an ecommerce order is created in the sense it's been updated in the inventory database and a record added to some sort of fulfilment system but there's still outstanding data warehouse sync jobs, some order analysis/product recommender job, and some anti fraud jobs (that just need to complete before the order picking in the warehouse begins)
    - zelphirkalt 5y ago
      What if the object is deleted again? You could still have an inconsistent picture, no? And you could create and delete over and over again, so that a client has an inconsistent picture all the time.
      - mafuy 5y ago
        That's a different design issue then - it means lack of synchronization. It is orthogonal to using uuids at a client. In fact, (properly) using uuids are a reasonable way to resolve this.
        zelphirkalt 5y ago
        Not arguing against using UUID. Just exploring that async concept further and the challenges in that. OK 2 separate issues : )
      - nchi3 5y ago
        That's what eventual consistency is, isn't it? It's definitely not the easiest thing to design systems around, but at a big enough scale you sometimes don't have a choice. There's also many ways to "work around it", so that it doesn't seem inconsistent to the user. I'm definitely not arguing that it's how most systems should be designed though.
  - piaste 5y ago
    If the client needs to use the object right there and then, it can start polling the GET for that ID until it returns 200. A slightly more sophisticated pattern is to have the initial creation command return an ID of the creation event. Then the client can poll that ID and just check that the event has been processed successfully. This requires the client to trust the server a little, but it means it can just poll a small, temporary, probably in-memory "recent events" table instead of hammering the main API. A more efficient design is to YOLO, submit the second command while the first is still being processed, and trust the server to handle the commands in the right order. This is fine for backend services, unfortunately human users get annoyed at their frontend when it says "sorry you have to do the purchase finalization again because an earlier 'add to cart' command failed and I only noticed it now".
globular-toast 5y ago
Well, you would have some kind of synchronisation protocol where the database confirms it has received that record and it now exists.
- deleted 5y ago
  [deleted]
nijave 5y ago
There's patterns for implementing it. It's not really any different than using more than 1 database which is unavoidable in many scenarios (interacting with 3rd parties) https://chrisrichardson.net/post/sagas/2019/08/04/developing-sagas-part-2.html https://chrisrichardson.net/post/sagas/2019/08/04/developing...

NicoJuicy 5y ago

That's why the pattern is eventual consistency. You receive a message "{entity}Created" and it contains the Id of the full object.

VWWHFSfQ 5y ago
It's like, _maybe_ eventual consistency. Hopefully the client doesn't try to do anything important with the ID/new object
- nchi3 5y ago
  If you get a "Created" back, it is possible for it to be guaranteed that it'll be created eventually.
  - VWWHFSfQ 5y ago
    I don't think you would ever get a 201 Created in an eventual consistency scenario. You would get a 202 Accepted. Unless the API is lying to the clients.
    - nchi3 5y ago
      Fair enough, though what I said remains true for an "Accepted" response.
      - NicoJuicy 5y ago
        But Accepted doesn't mean created. The feedback flow is async from the creation flow. If you get created through the feedback flow it's created.
        nchi3 5y ago
        Accepted can mean that it will eventually be created though. In the sense that it's validated and guaranteed to be created at a later point ("eventually").
    - nijave 5y ago
      Created can mean different things to different people. Maybe an ecommerce order is created in the sense it's been updated in the inventory database and a record added to some sort of fulfilment system but there's still outstanding data warehouse sync jobs, some order analysis/product recommender job, and some anti fraud jobs (that just need to complete before the order picking in the warehouse begins)
  - zelphirkalt 5y ago
    What if the object is deleted again? You could still have an inconsistent picture, no? And you could create and delete over and over again, so that a client has an inconsistent picture all the time.
    - mafuy 5y ago
      That's a different design issue then - it means lack of synchronization. It is orthogonal to using uuids at a client. In fact, (properly) using uuids are a reasonable way to resolve this.
      - zelphirkalt 5y ago
        Not arguing against using UUID. Just exploring that async concept further and the challenges in that. OK 2 separate issues : )
    - nchi3 5y ago
      That's what eventual consistency is, isn't it? It's definitely not the easiest thing to design systems around, but at a big enough scale you sometimes don't have a choice. There's also many ways to "work around it", so that it doesn't seem inconsistent to the user. I'm definitely not arguing that it's how most systems should be designed though.
- piaste 5y ago
  If the client needs to use the object right there and then, it can start polling the GET for that ID until it returns 200. A slightly more sophisticated pattern is to have the initial creation command return an ID of the creation event. Then the client can poll that ID and just check that the event has been processed successfully. This requires the client to trust the server a little, but it means it can just poll a small, temporary, probably in-memory "recent events" table instead of hammering the main API. A more efficient design is to YOLO, submit the second command while the first is still being processed, and trust the server to handle the commands in the right order. This is fine for backend services, unfortunately human users get annoyed at their frontend when it says "sorry you have to do the purchase finalization again because an earlier 'add to cart' command failed and I only noticed it now".

globular-toast 5y ago

Well, you would have some kind of synchronisation protocol where the database confirms it has received that record and it now exists.

deleted 5y ago
[deleted]

nijave 5y ago

There's patterns for implementing it. It's not really any different than using more than 1 database which is unavoidable in many scenarios (interacting with 3rd parties) https://chrisrichardson.net/post/sagas/2019/08/04/developing-sagas-part-2.html https://chrisrichardson.net/post/sagas/2019/08/04/developing...