3 ms·

What replaced VM? I've seen antirez saying recently that diskstore isn't necessarily the perfect replacement it was suppose to be either. Just a big confused a

by rkalla 15y ago

What replaced VM? I've seen antirez saying recently that diskstore isn't necessarily the perfect replacement it was suppose to be either. Just a big confused about what the best-practice is moving forward for bigger-than-memory data sets.

deleted 15y ago
[deleted]
pjscott 15y ago
For larger-than-memory data sets, your current options seem to be either to shard your data, or to use something other than Redis. Redis doesn't pretend to be the answer to every problem, and fortunately there are a lot of options out there. Personally, I'm using Redis right now for queueing and API request rate limiting. The portion of my data that's larger than memory is in MySQL. This is working pretty well.
- rkalla 15y ago
  Thank you PJ for the clarification. I have no problem with "the right tool for the right job", just wasn't sure how to fit all the pieces together. If redis wants to focus on ultra-fast/in memory I can take advantage of that in the right ways. Just wanted to make sure I understood the constraints correctly.
  - alnayyir 15y ago
    Redis is in general, just an excellent pillow to put between your API, app servers, and frontend servers and your main backing stores for various purposes. Just don't plan to keep anything important in it for very long. It's not even a great idea for queuing anything important as you're better of logging or using transactions for things that might cost you money if they fail. Personally, I think it's the memcached I always wanted. :)
    - antirez 15y ago
      You can surely use Redis as a caching solution, but if your data fits in memory you can use it as a full featured and absolutely safe database: 1) with AOF you have innodb-level durability. 2) with replication you can do your HA setups. 3) RDB files are self-contained files holding all the dataset that can be copied and backed-up while Redis is running. 4) with the upcoming cluster you can have fault tolerance without too much problems, but for this there is to wait this fall.
      - alnayyir 15y ago
        Is AOF the preferred persistence method than I take it? After seeing some pain that is required to resuscitate Redis from the persisted files if it goes down, I was wary of relying on that persistence. I'm pretty excited about the cluster functionality. At my current company sadly I'm working with big data so I can't really use Redis as a main backing store, but it's always in my head for personal projects.
- LeafStorm 15y ago
  Though in the long run, antirez wants to implement a system whereby data structures are actually manipulated in-place on disk, which sounds pretty amazing. The main loop and memory operations would still be single-threaded, but there would be threads in the background handling the disk IO. This is post-cluster though.
  - bretthoerner 15y ago
    Last I heard "diskstore" was also scrapped.

by rkalla 15y ago

deleted 15y ago

[deleted]

pjscott 15y ago

For larger-than-memory data sets, your current options seem to be either to shard your data, or to use something other than Redis. Redis doesn't pretend to be the answer to every problem, and fortunately there are a lot of options out there. Personally, I'm using Redis right now for queueing and API request rate limiting. The portion of my data that's larger than memory is in MySQL. This is working pretty well.

rkalla 15y ago
Thank you PJ for the clarification. I have no problem with "the right tool for the right job", just wasn't sure how to fit all the pieces together. If redis wants to focus on ultra-fast/in memory I can take advantage of that in the right ways. Just wanted to make sure I understood the constraints correctly.
- alnayyir 15y ago
  Redis is in general, just an excellent pillow to put between your API, app servers, and frontend servers and your main backing stores for various purposes. Just don't plan to keep anything important in it for very long. It's not even a great idea for queuing anything important as you're better of logging or using transactions for things that might cost you money if they fail. Personally, I think it's the memcached I always wanted. :)
  - antirez 15y ago
    You can surely use Redis as a caching solution, but if your data fits in memory you can use it as a full featured and absolutely safe database: 1) with AOF you have innodb-level durability. 2) with replication you can do your HA setups. 3) RDB files are self-contained files holding all the dataset that can be copied and backed-up while Redis is running. 4) with the upcoming cluster you can have fault tolerance without too much problems, but for this there is to wait this fall.
    - alnayyir 15y ago
      Is AOF the preferred persistence method than I take it? After seeing some pain that is required to resuscitate Redis from the persisted files if it goes down, I was wary of relying on that persistence. I'm pretty excited about the cluster functionality. At my current company sadly I'm working with big data so I can't really use Redis as a main backing store, but it's always in my head for personal projects.
LeafStorm 15y ago
Though in the long run, antirez wants to implement a system whereby data structures are actually manipulated in-place on disk, which sounds pretty amazing. The main loop and memory operations would still be single-threaded, but there would be threads in the background handling the disk IO. This is post-cluster though.
- bretthoerner 15y ago
  Last I heard "diskstore" was also scrapped.