3 ms·

can you give some examples why i'd want a data stream for a 'small scale project' ?

by abledon 5y ago

debrice 5y ago
Integration and webhooks are very well suited for streams. Having your core product emitting messages as event have many benefits: - You can have a team working on the core product sending messages and another working on integrations triggering actions. - If your integrations fails, your core product is not impacted. You can also replay old messages once you've fixed the stream consumer. - Having streams allows you to do all kind of experiments too, you can connect a new project to a stream and go through a week of data almost instantly and see the result... rince and repeat as much as necessary
qbasic_forever 5y ago
Event-based architectures, for example: https://martinfowler.com/eaaDev/EventSourcing.html https://martinfowler.com/eaaDev/EventSourcing.html For some types of applications moving to a model with a stream of events as the source of truth can solve a ton of hairy distributed computing problems.
- bjt 5y ago
  How is that not possible in the exchanges + queues model that Rabbit has supported forever?
codeisawesome 5y ago
I was recently thinking about this: let’s say you built a chrome extension and wants to collect some basic usage analytics (with explicit user consent and knowledge, while preserving privacy) - you could batch-send activities at intervals to a REST-style API, but would be nicer to handle as a stream (eg to respond in real-time somehow).
- atentaten 5y ago
  > eg to respond in real-time somehow Make the api call in real-time. No batching. Pipe the api request into your streaming service.
- vbsteven 5y ago
  Something lightweight like MQTT is also well suited for this. It was originally designed for telemetry messages in IoT situations but it also supports websockets so it can be used in web applications or browser extensions.

3 ms·

can you give some examples why i'd want a data stream for a 'small scale project' ?

by abledon 5y ago

can you give some examples why i'd want a data stream for a 'small scale project' ?

debrice 5y ago
Integration and webhooks are very well suited for streams. Having your core product emitting messages as event have many benefits: - You can have a team working on the core product sending messages and another working on integrations triggering actions. - If your integrations fails, your core product is not impacted. You can also replay old messages once you've fixed the stream consumer. - Having streams allows you to do all kind of experiments too, you can connect a new project to a stream and go through a week of data almost instantly and see the result... rince and repeat as much as necessary
qbasic_forever 5y ago
Event-based architectures, for example: https://martinfowler.com/eaaDev/EventSourcing.html https://martinfowler.com/eaaDev/EventSourcing.html For some types of applications moving to a model with a stream of events as the source of truth can solve a ton of hairy distributed computing problems.
- bjt 5y ago
  How is that not possible in the exchanges + queues model that Rabbit has supported forever?
codeisawesome 5y ago
I was recently thinking about this: let’s say you built a chrome extension and wants to collect some basic usage analytics (with explicit user consent and knowledge, while preserving privacy) - you could batch-send activities at intervals to a REST-style API, but would be nicer to handle as a stream (eg to respond in real-time somehow).
- atentaten 5y ago
  > eg to respond in real-time somehow Make the api call in real-time. No batching. Pipe the api request into your streaming service.
- vbsteven 5y ago
  Something lightweight like MQTT is also well suited for this. It was originally designed for telemetry messages in IoT situations but it also supports websockets so it can be used in web applications or browser extensions.