4 ms·

Azure Event Hubs – Cloud-scale telemetry ingestion

28 points by vyrotek 12y ago

chton 12y ago
I've been looking into this, and while it's an impressive piece of engineering, I'm disappointed at the lack of flexibility there is. You can determine your budget with Througput Units, but you lose out on a a very large group of cases by needing to configure Partitions up-front. This, and a few other guidelines (like having 1 worker process per partition), make it a much more specialized service than it would seem at first glance.
- ghawkescs 12y ago
  Where did you find the more in-depth technical details? I was looking through the articles on MSDN but they looked incomplete or brand-new.
  - chton 12y ago
    There's a great blog post for an overview at http://fabriccontroller.net/blog/posts/getting-started-azure-service-bus-event-hubs-building-a-real-time-log-stream/ http://fabriccontroller.net/blog/posts/getting-started-azure... You can learn quite a bit from the pricing page too: http://azure.microsoft.com/en-us/pricing/details/event-hubs/ http://azure.microsoft.com/en-us/pricing/details/event-hubs/ With these details you can get a pretty good idea how it works if you know how the rest of Azure is built. Other than that, it helps to know people that know people :) Information has been trickling in through various sources, slowly completing the picture.
kwallace 12y ago
404 on the first link for tutorial. Wow. https://azure.microsoft.com/en-us/documentation/services/event-hubs/ https://azure.microsoft.com/en-us/documentation/services/eve...
- chton 12y ago
  I've notified them, that is quite embarrassing.
  - maslam 12y ago
    They're fixing it now.
pjlegato 12y ago
Is this signficantly different than http://aws.amazon.com/kinesis/ http://aws.amazon.com/kinesis/ ?
- chton 12y ago
  Not significantly, but they differ in the finer details. Event Hubs is less elastic (since you need to choose your number of partitions on create), but it has a few more advanced options like filtering and fine-grained message replay. Kinesis has no requirement to chose a number of shards on creation, but is closer to a simple fat pipe than Event Hubs. In intention, basic structure and pricing, they are nearly identical.

28 points by vyrotek 12y ago

chton 12y ago

I've been looking into this, and while it's an impressive piece of engineering, I'm disappointed at the lack of flexibility there is. You can determine your budget with Througput Units, but you lose out on a a very large group of cases by needing to configure Partitions up-front. This, and a few other guidelines (like having 1 worker process per partition), make it a much more specialized service than it would seem at first glance.

ghawkescs 12y ago
Where did you find the more in-depth technical details? I was looking through the articles on MSDN but they looked incomplete or brand-new.
- chton 12y ago
  There's a great blog post for an overview at http://fabriccontroller.net/blog/posts/getting-started-azure-service-bus-event-hubs-building-a-real-time-log-stream/ http://fabriccontroller.net/blog/posts/getting-started-azure... You can learn quite a bit from the pricing page too: http://azure.microsoft.com/en-us/pricing/details/event-hubs/ http://azure.microsoft.com/en-us/pricing/details/event-hubs/ With these details you can get a pretty good idea how it works if you know how the rest of Azure is built. Other than that, it helps to know people that know people :) Information has been trickling in through various sources, slowly completing the picture.

kwallace 12y ago

404 on the first link for tutorial. Wow. https://azure.microsoft.com/en-us/documentation/services/event-hubs/ https://azure.microsoft.com/en-us/documentation/services/eve...

chton 12y ago
I've notified them, that is quite embarrassing.
- maslam 12y ago
  They're fixing it now.

pjlegato 12y ago

Is this signficantly different than http://aws.amazon.com/kinesis/ http://aws.amazon.com/kinesis/ ?

chton 12y ago
Not significantly, but they differ in the finer details. Event Hubs is less elastic (since you need to choose your number of partitions on create), but it has a few more advanced options like filtering and fine-grained message replay. Kinesis has no requirement to chose a number of shards on creation, but is closer to a simple fat pipe than Event Hubs. In intention, basic structure and pricing, they are nearly identical.