6 ms·

> It uses Facebook's high-performance RocksDB embedded datastore internally to persist all job data, queues, error state, etc This breaks the devops story for

> It uses Facebook's high-performance RocksDB embedded datastore internally to persist all job data, queues, error state, etc This breaks the devops story for me. The difference in featureset between Rocksdb and redis is not that big.. However redis is hugely supported on the cloud and in high-availability fully-managed mode. Its so convenient to use sidekiq on heroku or aws. I really hope you build this on redis rather than a new persistence server.

deedubaya 9y ago
Correct me if I'm wrong, but RocksDB is embedded withink faktory, so this would actually be one less service to manage than using an external redis server. Isn't that a good thing? From my experience, when running high throughput, quickly executing sidekiq workers on heroku, the expense often doesn't come from dynos, but the redis instance as the limiting scale factor usually comes down to connections. That won't be a problem with Faktory and an embedded datastore.
- wilde 9y ago
  It depends on how comfortable you are developing a monitoring and failover system for something stateful. Right now AWS handles failover of our redis instances. With Faktory, I’m not sure what that looks like yet.
  - deedubaya 9y ago
    Indeed. It will be interesting to see how it handles fail over in the case of an outage.
- meritt 9y ago
  Can you provide a bit more details about the limits you're encountering? Connections? As in the sheer quantity? Or speed to connect? Are you hitting linux open files limits? I have a tough time understanding how Redis would be the bottleneck among a bunch of Ruby workers.
  - deedubaya 9y ago
    Specifically on Heroku, the different tiers of redis provide limits for the number of connections to it that can be made. This means that if you hav 5x100 low-memory, low-cpu workers (arbitrary), you'll probably be playing $750/month just to have a redis instance that supports 5,000 connections. It has less to do with technical limitations and more to do with pricing of other services, I think. https://elements.heroku.com/addons/heroku-redis https://elements.heroku.com/addons/heroku-redis
    - meritt 9y ago
      Holy hell. You need to get off Heroku right away. This is insane.
- sandGorgon 9y ago
  This is not the state of mind right now - and this is the same thing that blockchain devs seem to think. Everybody uses the cloud for anything reasonably important. And if it is not important, the difficulty delta between an embedded datastore and "./redis" is miniscule. Which means that leveldb/rocksdb gives zero path to scalability at the cost of saving a maximum of a few minutes of effort. At worst, this could have been done in redis with embedded lua packaged together. That would have solved both your external redis problem as well as a reasonable path to scale.

deedubaya 9y ago

Correct me if I'm wrong, but RocksDB is embedded withink faktory, so this would actually be one less service to manage than using an external redis server. Isn't that a good thing? From my experience, when running high throughput, quickly executing sidekiq workers on heroku, the expense often doesn't come from dynos, but the redis instance as the limiting scale factor usually comes down to connections. That won't be a problem with Faktory and an embedded datastore.

wilde 9y ago
It depends on how comfortable you are developing a monitoring and failover system for something stateful. Right now AWS handles failover of our redis instances. With Faktory, I’m not sure what that looks like yet.
- deedubaya 9y ago
  Indeed. It will be interesting to see how it handles fail over in the case of an outage.
meritt 9y ago
Can you provide a bit more details about the limits you're encountering? Connections? As in the sheer quantity? Or speed to connect? Are you hitting linux open files limits? I have a tough time understanding how Redis would be the bottleneck among a bunch of Ruby workers.
- deedubaya 9y ago
  Specifically on Heroku, the different tiers of redis provide limits for the number of connections to it that can be made. This means that if you hav 5x100 low-memory, low-cpu workers (arbitrary), you'll probably be playing $750/month just to have a redis instance that supports 5,000 connections. It has less to do with technical limitations and more to do with pricing of other services, I think. https://elements.heroku.com/addons/heroku-redis https://elements.heroku.com/addons/heroku-redis
  - meritt 9y ago
    Holy hell. You need to get off Heroku right away. This is insane.
sandGorgon 9y ago
This is not the state of mind right now - and this is the same thing that blockchain devs seem to think. Everybody uses the cloud for anything reasonably important. And if it is not important, the difficulty delta between an embedded datastore and "./redis" is miniscule. Which means that leveldb/rocksdb gives zero path to scalability at the cost of saving a maximum of a few minutes of effort. At worst, this could have been done in redis with embedded lua packaged together. That would have solved both your external redis problem as well as a reasonable path to scale.