5 ms·

Can you talk more about your specific use case?

sudhirj 8y ago
The simplest example is liveblogging - someone is sitting at the next iPhone launch, and sending out photos and commentary that's being updated as they send on a large number of readers' screens worldwide. The Verge has a good system that they do with polling S3 for new data, but there's really much more effective ways to do it. The more specific cases include opening ticket sales for movies in India - for some big movies tickets are sold out really quickly, so there'll be hundreds or thousands of people with browsers / phones open on a seat layout and they all need to see which seats are being sold in real-time. Then there's broadcasting stock market movements and tips to thousands of traders in real-time. The only existing solutions that seem really scalable are fanout.io - others like pusher.com and pubnub.com have pricing that's not conducive to serving a large number of simultaneous users / broadcasting. Fanout.io gets this right, but would be nice to have cheaper alternatives / make it a commodity. Given that the CF nodes do support websockets, I assume they terminate and re-create new connections to origins as well - so the nodes are capable of holding a large number of websockets open, right? Can Workers intercept websockets / stream data into them? And given that the KV store is capable of propagating new values to all the nodes, there could be a way expose incoming change notifications on keys, which would handle this use case. Could also implement as continuous addition of new timestamps keys, prefixed with a topic.
- zackbloom 8y ago
  Fundamentally what I think we want to do is build the primitives you would need to create your own fanout. That means we need the ability to push events to Worker nodes, and for Workers to be able to terminate WebSockets for you. If you can accept up to 10s of latency, I agree that KV could solve the 'pushing events' problem for you. (A tuple store is semantically equivalent to message passing, and it will get easier when we support range queries).
  - sudhirj 8y ago
    Yes, both points make sense. And I assume the 10s latency is also geographically related - i.e. it's worst case 10s for the longest distance on the network topology graph? That should be fine. Low latency requirements also tend to be geographical, I think - stock markets are mostly specific to a single country, examples like ticketing are usually specific to a single city / edge. For general news being broadcast globally, that kind of latency is fine.
- jkarneges 8y ago
  > Fanout.io gets this right, but would be nice to have cheaper alternatives / make it a commodity. Hi! Fanout founder here. I’m glad you like our way of doing things. If you need better pricing for high volume we are always willing to discuss. I’m not sure you’d want to maintain a WebSocket inside of a CF worker. At least I can’t see that helping with cost. Execution time limits would get you too. IMO connection management belongs as a separate layer in front of the worker, so workers are only woken when things happen. And on that note, of course you can use Fanout together with Cloudflare Workers today. At the moment this requires making raw API calls but we plan to update our server libs for CF compatibility soon.
  - sudhirj 8y ago
    > Hi! Fanout founder here. I’m glad you like our way of doing things. If you need better pricing for high volume we are always willing to discuss. Hi! Wasn't expecting you to show up like this. Re. pricing for high volume, I'm sure enterprises and publishers could afford it, I'm trying to see if it's possible to offer free or very cheap service to individuals with large audiences. So not so much that I want a high volume discount, I'm trying to figure out a way to make it one or two orders of magnitude cheaper, enough that pricing is no longer relevant. Self hosting Pushpin is an option, but CF might be easier to work with. > I’m not sure you’d want to maintain a WebSocket inside of a CF worker. At least I can’t see that helping with cost. Execution time limits would get you too. The CF edges already hold / passthrough websockets - whether the Workers API is conducive to controlling them is the question, but we know it already allows streaming - doesn't seem like big jump from there. And for my purposes one-way streaming is enough, so might not even need to change the API. And based on the current billing model it seems like only CPU time is billed (?) - so if the socket is waiting most of the time it's still cost effective. And connection management itself would still be outside the scope of the worker - I'm saying the worker's event loop should behave exactly the way it would if I using the streaming feature right now to make a 1000 slow network requests from the worker and streamed the responses out as the worker received them. Only the protocol would change.
    - kentonv 8y ago
      I actually have an open PR to add WebSocket endpoint support to CF Workers (so a Worker can act as a WebSocket client or server, instead of just a proxy). The main hangup though is that we need to figure out how to charge for long-lived connections. We obviously don't want to apply the normal time limits in this case. Hopefully we'll find time to work through this soon. :)

The simplest example is liveblogging - someone is sitting at the next iPhone launch, and sending out photos and commentary that's being updated as they send on a large number of readers' screens worldwide. The Verge has a good system that they do with polling S3 for new data, but there's really much more effective ways to do it. The more specific cases include opening ticket sales for movies in India - for some big movies tickets are sold out really quickly, so there'll be hundreds or thousands of people with browsers / phones open on a seat layout and they all need to see which seats are being sold in real-time. Then there's broadcasting stock market movements and tips to thousands of traders in real-time. The only existing solutions that seem really scalable are fanout.io - others like pusher.com and pubnub.com have pricing that's not conducive to serving a large number of simultaneous users / broadcasting. Fanout.io gets this right, but would be nice to have cheaper alternatives / make it a commodity. Given that the CF nodes do support websockets, I assume they terminate and re-create new connections to origins as well - so the nodes are capable of holding a large number of websockets open, right? Can Workers intercept websockets / stream data into them? And given that the KV store is capable of propagating new values to all the nodes, there could be a way expose incoming change notifications on keys, which would handle this use case. Could also implement as continuous addition of new timestamps keys, prefixed with a topic.

zackbloom 8y ago
Fundamentally what I think we want to do is build the primitives you would need to create your own fanout. That means we need the ability to push events to Worker nodes, and for Workers to be able to terminate WebSockets for you. If you can accept up to 10s of latency, I agree that KV could solve the 'pushing events' problem for you. (A tuple store is semantically equivalent to message passing, and it will get easier when we support range queries).
- sudhirj 8y ago
  Yes, both points make sense. And I assume the 10s latency is also geographically related - i.e. it's worst case 10s for the longest distance on the network topology graph? That should be fine. Low latency requirements also tend to be geographical, I think - stock markets are mostly specific to a single country, examples like ticketing are usually specific to a single city / edge. For general news being broadcast globally, that kind of latency is fine.
jkarneges 8y ago
> Fanout.io gets this right, but would be nice to have cheaper alternatives / make it a commodity. Hi! Fanout founder here. I’m glad you like our way of doing things. If you need better pricing for high volume we are always willing to discuss. I’m not sure you’d want to maintain a WebSocket inside of a CF worker. At least I can’t see that helping with cost. Execution time limits would get you too. IMO connection management belongs as a separate layer in front of the worker, so workers are only woken when things happen. And on that note, of course you can use Fanout together with Cloudflare Workers today. At the moment this requires making raw API calls but we plan to update our server libs for CF compatibility soon.
- sudhirj 8y ago
  > Hi! Fanout founder here. I’m glad you like our way of doing things. If you need better pricing for high volume we are always willing to discuss. Hi! Wasn't expecting you to show up like this. Re. pricing for high volume, I'm sure enterprises and publishers could afford it, I'm trying to see if it's possible to offer free or very cheap service to individuals with large audiences. So not so much that I want a high volume discount, I'm trying to figure out a way to make it one or two orders of magnitude cheaper, enough that pricing is no longer relevant. Self hosting Pushpin is an option, but CF might be easier to work with. > I’m not sure you’d want to maintain a WebSocket inside of a CF worker. At least I can’t see that helping with cost. Execution time limits would get you too. The CF edges already hold / passthrough websockets - whether the Workers API is conducive to controlling them is the question, but we know it already allows streaming - doesn't seem like big jump from there. And for my purposes one-way streaming is enough, so might not even need to change the API. And based on the current billing model it seems like only CPU time is billed (?) - so if the socket is waiting most of the time it's still cost effective. And connection management itself would still be outside the scope of the worker - I'm saying the worker's event loop should behave exactly the way it would if I using the streaming feature right now to make a 1000 slow network requests from the worker and streamed the responses out as the worker received them. Only the protocol would change.
  - kentonv 8y ago
    I actually have an open PR to add WebSocket endpoint support to CF Workers (so a Worker can act as a WebSocket client or server, instead of just a proxy). The main hangup though is that we need to figure out how to charge for long-lived connections. We obviously don't want to apply the normal time limits in this case. Hopefully we'll find time to work through this soon. :)