6 ms·

I often find myself wishing for some sort of small scale durable log, so that this pattern was easy to implement in a small app backend without standing up some

by dminor 5y ago

I often find myself wishing for some sort of small scale durable log, so that this pattern was easy to implement in a small app backend without standing up something like Kafka.

emerongi 5y ago
I recently refactored a small set of services to use events instead of depending directly on eachother. I initially had Kafka in mind, but it would've been an absolute overkill. A "simple Kafka" would definitely be nice for these cases. I ended up using Redis Streams [0], which was good enough for my small-scale need. We already had Redis in our stack too, so it was a very simple integration. [0] https://redis.io/topics/streams-intro https://redis.io/topics/streams-intro
uDontKnowMe 5y ago
Have you considered rabbitmq or nats?
dharmaturtle 5y ago
What's wrong with EventStore? https://www.eventstore.com/ https://www.eventstore.com/
- dminor 5y ago
  Nice! Hadn't heard of it, will check it out.
  - dharmaturtle 5y ago
    They have a channel in this slack that's pretty good for getting answers to questions: https://github.com/ddd-cqrs-es/slack-community https://github.com/ddd-cqrs-es/slack-community
dimitrios1 5y ago
try jetstream
xfer 5y ago
Last time i looked into it there weren't that many i could find. There is https://github.com/tikv/tikv https://github.com/tikv/tikv which uses rocksdb with raft. and there is faster https://github.com/microsoft/FASTER/ https://github.com/microsoft/FASTER/ .
adamors 5y ago
Check out MessageDB https://github.com/message-db/message-db https://github.com/message-db/message-db. It's basically Postgres and can be installed into any Postgres DB.
pyrale 5y ago
For a small app, you may simply create a `events` table in a pg, with a json field storing the payload. It works pretty well this way, not everything needs to be streamed.
thibauts 5y ago
Styx was built with this kind of use in mind, among other things. If you’re using Go you can even use the storage engine standalone. It’s pretty robust and very fast (millions of fsynced writes/sec). https://github.com/dataptive/styx https://github.com/dataptive/styx
- dminor 5y ago
  I think this may be exactly what I was hoping for :)
  - thibauts 5y ago
    I’ll be more than happy to get your feedback if you test it ! Especially potential shortcomings preventing its use in production settings.
dsies 5y ago
Hi there, I started a company specifically centered around event sourcing - it's a saas platform for capturing all of your events, gaining the ability to granularly search them and then replay them to whatever destination. I'd be happy to walk you through the platform - there's no lock-in, since we don't require the use of any SDK's, just run an event-relaying container which will pipe all of your events to our stores. One big piece is that our platform is message bus agnostic meaning, we are able to hook into any message bus, be it kafka, rabbit, nats, etc.; same goes for replays - we can replay into any destination. Check it out: https://batch.sh https://batch.sh The relayer is open source - https://github.com/batchcorp/plumber https://github.com/batchcorp/plumber - if anything, the relayer can also be used for working with message busses which could improve your dev workflow for reading/writing messages, etc.

by dminor 5y ago

I often find myself wishing for some sort of small scale durable log, so that this pattern was easy to implement in a small app backend without standing up something like Kafka.

emerongi 5y ago

I recently refactored a small set of services to use events instead of depending directly on eachother. I initially had Kafka in mind, but it would've been an absolute overkill. A "simple Kafka" would definitely be nice for these cases. I ended up using Redis Streams [0], which was good enough for my small-scale need. We already had Redis in our stack too, so it was a very simple integration. [0] https://redis.io/topics/streams-intro https://redis.io/topics/streams-intro

uDontKnowMe 5y ago

Have you considered rabbitmq or nats?

dharmaturtle 5y ago

What's wrong with EventStore? https://www.eventstore.com/ https://www.eventstore.com/

dminor 5y ago
Nice! Hadn't heard of it, will check it out.
- dharmaturtle 5y ago
  They have a channel in this slack that's pretty good for getting answers to questions: https://github.com/ddd-cqrs-es/slack-community https://github.com/ddd-cqrs-es/slack-community

dimitrios1 5y ago

try jetstream

xfer 5y ago

Last time i looked into it there weren't that many i could find. There is https://github.com/tikv/tikv https://github.com/tikv/tikv which uses rocksdb with raft. and there is faster https://github.com/microsoft/FASTER/ https://github.com/microsoft/FASTER/ .

adamors 5y ago

Check out MessageDB https://github.com/message-db/message-db https://github.com/message-db/message-db. It's basically Postgres and can be installed into any Postgres DB.

pyrale 5y ago

For a small app, you may simply create a `events` table in a pg, with a json field storing the payload. It works pretty well this way, not everything needs to be streamed.

thibauts 5y ago

Styx was built with this kind of use in mind, among other things. If you’re using Go you can even use the storage engine standalone. It’s pretty robust and very fast (millions of fsynced writes/sec). https://github.com/dataptive/styx https://github.com/dataptive/styx

dminor 5y ago
I think this may be exactly what I was hoping for :)
- thibauts 5y ago
  I’ll be more than happy to get your feedback if you test it ! Especially potential shortcomings preventing its use in production settings.

dsies 5y ago

Hi there, I started a company specifically centered around event sourcing - it's a saas platform for capturing all of your events, gaining the ability to granularly search them and then replay them to whatever destination. I'd be happy to walk you through the platform - there's no lock-in, since we don't require the use of any SDK's, just run an event-relaying container which will pipe all of your events to our stores. One big piece is that our platform is message bus agnostic meaning, we are able to hook into any message bus, be it kafka, rabbit, nats, etc.; same goes for replays - we can replay into any destination. Check it out: https://batch.sh https://batch.sh The relayer is open source - https://github.com/batchcorp/plumber https://github.com/batchcorp/plumber - if anything, the relayer can also be used for working with message busses which could improve your dev workflow for reading/writing messages, etc.