4 ms·

This is really cool as a demonstration of how distributed systems can be built on top of Durable Objects. In particular I was really happy to see this: > What

This is really cool as a demonstration of how distributed systems can be built on top of Durable Objects. In particular I was really happy to see this: > What about replication? What about leadership election? > There is none, at least not in the application code. The point of using Durable Objects here is that we can offload these complexities onto the infrastructure layer, and keep our application code focused and minimal. Yes, exactly. With Durable Objects, these common problems are offloaded to the underlying system, and you can instead focus on the higher-level distributed systems algorithms problems that are unique to the particular infrastructure you want to build. (I'm the tech lead for Cloudflare Workers and helped design Durable Objects.)

sammy2255 4y ago
Thank you lord Kenton ^_^ I’m honestly looking forward to Cloudflare queues which is probably just built on top of DOs but its sooo nice having it as a service
klabb3 4y ago
The abstraction is great, just a single threaded object with strong consistency. It's an awesome building block for a managed service. I really wish DOs had bundled pricing, or something like it. I found myself thinking about how to reduce # objects to be cost effective :/
m00dy 4y ago
What can I do with this ?
maxwellpeterson 4y ago
Thanks Kenton!!
cjdoc29 4y ago
I feel like this is a dumb question. But how exactly does Durable Objects help you avoid replication and leadership election? I've read the high level overview of Durable Objects, but sadly, the answer isn't clicking for me...
- ec109685 4y ago
  The concurrency with Durable Objects when it comes to mutating state is effectively 1: https://developers.cloudflare.com/workers/learning/using-durable-objects/ https://developers.cloudflare.com/workers/learning/using-dur... Parallel requests can be served, but they are all through a single instance and the abstraction gives you effectively exclusive access to the data store while the stateful part of a request is being processed.
  - MuffinFlavored 4y ago
    what kind of “transactions per second” / latency is possible given this fact?
    - kentonv 4y ago
      This does imply a hard limit on transactions per second on a particular object, since those transactions must be serialized in a thread. But, this is true of all databases: there is a limit on transactions per second operating on the same "chunk" of the database, since those transactions by definition have to be serialized. In fact, you may find a traditional database behaves much worse when everyone is trying to modify the same row: if many clients do a read-modify-write at the same time, all but one of them will fail with transaction conflicts and will have to start over. If you aren't careful this can easily be O(n^2) total work done. With Durable Objects, because your application code itself runs serialized in the object thread with zero storage latency (assuming cache), it's easier to make each operation "atomic" and avoid transaction conflicts entirely.
EdwardDiego 4y ago
So every topic has a (Kafka) replication factor of 1, but how do producers find their partition leader if the DO fails? From my understanding, all the DOs are effectively the same broker so they'd never lose the leader? With regards to Kafka 0.8, I inferred it was due to the protocol being simpler back then, what issues did you hit with newer protocol versions? Nice way to demo the tech :)
- maxwellpeterson 4y ago
  Producers (and all clients) can only see and interact with the gateway worker. DOs aren’t accessible from the outside world, and all communication with them happens through the gateway worker. In the case of DO failure, it’s the gateway worker’s job to reconnect with the failed DO. Depending on the nature of the failure, this reconnection request might be routed to a DO instance located on a different machine from the original, failed instance (in which case a leadership change happened internally). From the client perspective, nothing changes, and it can keep using the same gateway worker connection as before. Regarding 0.8, that’s right, I wanted to start with something as simple as possible. This project wasn’t intended for serious, practical usage, so spending time on extra features added in later versions didn’t seem useful. I’d imagine that newer protocol versions could be implemented without too much difficulty, but whether this thing is fast enough to ever be practical is another larger question.
  - EdwardDiego 4y ago
    Ah yep, so it's effectively a proxy in front of the broker(s) - like Pulsar's proxy (that Kafka really needs to copy IMO). And kudos mate, it's a very cool way to prove the concept. :)

sammy2255 4y ago

Thank you lord Kenton ^_^ I’m honestly looking forward to Cloudflare queues which is probably just built on top of DOs but its sooo nice having it as a service

klabb3 4y ago

The abstraction is great, just a single threaded object with strong consistency. It's an awesome building block for a managed service. I really wish DOs had bundled pricing, or something like it. I found myself thinking about how to reduce # objects to be cost effective :/

m00dy 4y ago

What can I do with this ?

maxwellpeterson 4y ago

Thanks Kenton!!

cjdoc29 4y ago

I feel like this is a dumb question. But how exactly does Durable Objects help you avoid replication and leadership election? I've read the high level overview of Durable Objects, but sadly, the answer isn't clicking for me...

ec109685 4y ago
The concurrency with Durable Objects when it comes to mutating state is effectively 1: https://developers.cloudflare.com/workers/learning/using-durable-objects/ https://developers.cloudflare.com/workers/learning/using-dur... Parallel requests can be served, but they are all through a single instance and the abstraction gives you effectively exclusive access to the data store while the stateful part of a request is being processed.
- MuffinFlavored 4y ago
  what kind of “transactions per second” / latency is possible given this fact?
  - kentonv 4y ago
    This does imply a hard limit on transactions per second on a particular object, since those transactions must be serialized in a thread. But, this is true of all databases: there is a limit on transactions per second operating on the same "chunk" of the database, since those transactions by definition have to be serialized. In fact, you may find a traditional database behaves much worse when everyone is trying to modify the same row: if many clients do a read-modify-write at the same time, all but one of them will fail with transaction conflicts and will have to start over. If you aren't careful this can easily be O(n^2) total work done. With Durable Objects, because your application code itself runs serialized in the object thread with zero storage latency (assuming cache), it's easier to make each operation "atomic" and avoid transaction conflicts entirely.

EdwardDiego 4y ago

So every topic has a (Kafka) replication factor of 1, but how do producers find their partition leader if the DO fails? From my understanding, all the DOs are effectively the same broker so they'd never lose the leader? With regards to Kafka 0.8, I inferred it was due to the protocol being simpler back then, what issues did you hit with newer protocol versions? Nice way to demo the tech :)

maxwellpeterson 4y ago
Producers (and all clients) can only see and interact with the gateway worker. DOs aren’t accessible from the outside world, and all communication with them happens through the gateway worker. In the case of DO failure, it’s the gateway worker’s job to reconnect with the failed DO. Depending on the nature of the failure, this reconnection request might be routed to a DO instance located on a different machine from the original, failed instance (in which case a leadership change happened internally). From the client perspective, nothing changes, and it can keep using the same gateway worker connection as before. Regarding 0.8, that’s right, I wanted to start with something as simple as possible. This project wasn’t intended for serious, practical usage, so spending time on extra features added in later versions didn’t seem useful. I’d imagine that newer protocol versions could be implemented without too much difficulty, but whether this thing is fast enough to ever be practical is another larger question.
- EdwardDiego 4y ago
  Ah yep, so it's effectively a proxy in front of the broker(s) - like Pulsar's proxy (that Kafka really needs to copy IMO). And kudos mate, it's a very cool way to prove the concept. :)