4 ms·

With a 2.6 system this problems will be simpler to detect, because at this point we have explicit (configurable) limits for clients, of type normal, pubsub, sla

by antirez 14y ago

With a 2.6 system this problems will be simpler to detect, because at this point we have explicit (configurable) limits for clients, of type normal, pubsub, slaves, so the connection is dropped if the amount of pending data goes over a given limit (and the event logged). This added a bit of complexity to a few code paths that now require to estimate the size of objects, data, and so forth, but I think this will pay back in the long run. This blog post makes me wonder if we should implement compression in the replication link as a feature in the future, or at least binary encoding. In short there is a plan for a binary version of AOF / replication-link that can save a lot of bandwidth.

solso 14y ago
woops! @antirez in person :-) amazing job you have done with Redis. Initially we didn't add compression to save bandwidth but to solve the network bandwidth issues between amazon and rackspace. However, it turns out that you can save on the bill quite a lot, in our case about ~ $700. Regarding the binary, would be very nice to have. The first though is that compression would work better for us than binary since our keys are quite long (50/100 bytes) and very regular on their patterns, so compression works very well in our case. Probably can be extrapolated for other cases where the keys are typically heavier than the values.
Smrchy 14y ago
Wouldn't compression between redis and hiredis make sense as well? How much load would compression put on a Redis instance that talks to n clients where each client sends/receives medium sized JSON docs. And how much would be gained in terms of faster responses from Redis.
- stock_toaster 14y ago
  Something like lzf or snappy maybe?
  - iand675 14y ago
    In my experience, snappy with very small messages (< 200 bytes or so) can increase bandwidth usage, so I'd say the way in which one uses redis can impact if compression is useful or not.

by antirez 14y ago

solso 14y ago

woops! @antirez in person :-) amazing job you have done with Redis. Initially we didn't add compression to save bandwidth but to solve the network bandwidth issues between amazon and rackspace. However, it turns out that you can save on the bill quite a lot, in our case about ~ $700. Regarding the binary, would be very nice to have. The first though is that compression would work better for us than binary since our keys are quite long (50/100 bytes) and very regular on their patterns, so compression works very well in our case. Probably can be extrapolated for other cases where the keys are typically heavier than the values.

Smrchy 14y ago

Wouldn't compression between redis and hiredis make sense as well? How much load would compression put on a Redis instance that talks to n clients where each client sends/receives medium sized JSON docs. And how much would be gained in terms of faster responses from Redis.

stock_toaster 14y ago
Something like lzf or snappy maybe?
- iand675 14y ago
  In my experience, snappy with very small messages (< 200 bytes or so) can increase bandwidth usage, so I'd say the way in which one uses redis can impact if compression is useful or not.