4 ms·
Redis drops data semi randomly when under memory pressure. If you use Redis for queue tasks (this is popular in Rails and Django/Python web services), that mea
by jitl 3y ago
Redis drops data semi randomly when under memory pressure.
If you use Redis for queue tasks (this is popular in Rails and Django/Python web services), that means that during an incident where your queue jobs are getting added faster than they’re removed, you’re going to lose jobs if the incident goes on long enough.
- byroot 3y agoThat depends on how the `maxmemory-policy` is configured, and queue systems based on Redis will tell you not to allow eviction. https://github.com/sidekiq/sidekiq/wiki/Using-Redis#memory https://github.com/sidekiq/sidekiq/wiki/Using-Redis#memory (it even logs a warnings if it detects your Redis is misconfigured IIRC).
- kiitos 3y agoWell, of course! Redis is not (and has never been) a database, it's a data structure server, at best described as a cache. If jobs are added faster than they're removed, this is straight queueing theory 101 -- ideally you'd reject jobs at add-time, but otherwise you have to drop them.
- gnarbarian 3y agoRight. I think Redis hitting the disk would be a terrible tradeoff compared to making a new backend call. it probably wouldn't save you much time and I imagine it would lead to very strange and unpredictable behavior on the front end or trying to debug latency or data issues downstream
- emseetech 3y agoSince Redis is an in-memory cache, and already doesn't guarantee the data, would it make sense to set PRAGMA SYNCHRONOUS on nalgeon to OFF to boost performance to something closer to standard Redis?