4 ms·

I'm a little concerned with the added complexity of decoupling the datastore into two different systems that are relied on for application logic. What are good

by stephth 15y ago

I'm a little concerned with the added complexity of decoupling the datastore into two different systems that are relied on for application logic. What are good strategies to maintain consistency between the two, in the event of a failure? In the first example, what if the SQL transaction succeeds but the redis one fails? Would you rollback the SQL transaction?

joevandyk 15y ago
Depends on your application. If you're using ActiveRecord, if the save to redis fails, raise an exception and it'll rollback the transaction. Note that you'll always want a way to resync the redis copy of the data with what's in your database.
- stephth 15y ago
  Thanks. Note that you'll always want a way to resync the redis copy of the data with what's in your database. Why is that? Isn't Redis persistent?
  - tednaleid 15y ago
    Yes, you can tell it to either make a copy to disk ever N seconds or use an AOF mechanism, but that doesn't mean it's in sync with your database. Depending on how your DB can get updated, you'll want to think about how the cache can get stale and whether you're expiring data in redis periodically, or removing keys/using different hashes for keys. The specific way to do this will depend on the app that you're using.
tednaleid 15y ago
In most systems, you're probably already doing it. Any ORM that has a level 1/2 cache built in (like Hibernate and EHCache/terracotta/etc or SQL caching in Rails) is storing the data in more than one place. If the data is in the cache (or if the cache fails for whatever reason), you're in the same boat you are now. If you're just using redis for write-through caching/memoizatoin and there's some failure, you still have the answer in hand and can return it to the user, you just lose the benefit of the speedup from the cache.
edanm 15y ago
It's a big problem, at least for our application (chattybar.com - a chat plugin). We have 2 copies of every chat message sent, one in the DB (for persistency) and one in Redis for quick retrieval. Keeping them synced has a lot of edge cases, especially since multithreading means that adding things to the DB and then to Redis doesn't necessarily guarantee they'll end up in the same order.
- dolinsky 15y ago
  If all you're doing is keeping a copy of objects in a memory store for quick retrieval you might want to implement a read-through caching pattern using Memcached (or Redis but I'd turn any persistence off). Most chat clients don't allow for editing of posts, so you could just use Redis as your persistent storage using lists and RPUSH.
- deleted 15y ago
  [deleted]

by stephth 15y ago

joevandyk 15y ago

Depends on your application. If you're using ActiveRecord, if the save to redis fails, raise an exception and it'll rollback the transaction. Note that you'll always want a way to resync the redis copy of the data with what's in your database.

stephth 15y ago
Thanks. Note that you'll always want a way to resync the redis copy of the data with what's in your database. Why is that? Isn't Redis persistent?
- tednaleid 15y ago
  Yes, you can tell it to either make a copy to disk ever N seconds or use an AOF mechanism, but that doesn't mean it's in sync with your database. Depending on how your DB can get updated, you'll want to think about how the cache can get stale and whether you're expiring data in redis periodically, or removing keys/using different hashes for keys. The specific way to do this will depend on the app that you're using.

tednaleid 15y ago

In most systems, you're probably already doing it. Any ORM that has a level 1/2 cache built in (like Hibernate and EHCache/terracotta/etc or SQL caching in Rails) is storing the data in more than one place. If the data is in the cache (or if the cache fails for whatever reason), you're in the same boat you are now. If you're just using redis for write-through caching/memoizatoin and there's some failure, you still have the answer in hand and can return it to the user, you just lose the benefit of the speedup from the cache.

edanm 15y ago

It's a big problem, at least for our application (chattybar.com - a chat plugin). We have 2 copies of every chat message sent, one in the DB (for persistency) and one in Redis for quick retrieval. Keeping them synced has a lot of edge cases, especially since multithreading means that adding things to the DB and then to Redis doesn't necessarily guarantee they'll end up in the same order.

dolinsky 15y ago
If all you're doing is keeping a copy of objects in a memory store for quick retrieval you might want to implement a read-through caching pattern using Memcached (or Redis but I'd turn any persistence off). Most chat clients don't allow for editing of posts, so you could just use Redis as your persistent storage using lists and RPUSH.
deleted 15y ago
[deleted]