4 ms·

I have a trading bot that consolidates a lot of price information in order to make decisions. I'm currently using Redis due to data structures (lists and scores

by haolez 7y ago

I have a trading bot that consolidates a lot of price information in order to make decisions. I'm currently using Redis due to data structures (lists and scores) and Pub/Sub. I wonder if memcached would give me performance benefits and lower my CPU costs.

manigandham 7y ago
CPU is cheap and memcached has no advanced data structures. If you're using them then you'll probably have to reimplement everything yourself in memcached with less efficiency.
- henriquez 7y ago
  You can always just JSON-encode whatever you’re putting in there and decode on the way out. Though I couldn’t imagine using it for pub/sub like the parent commenter is doing with redis.
  - manigandham 7y ago
    Which is inefficient and pointless compared to just using Redis with data structures that you can manipulate directly.
dormando 7y ago
Mind giving some examples in more detail about how you make use of lists/scores?
- haolez 7y ago
  Pub/Sub is easily replaceable. I use lists and scores to store orders from orderbooks, where the prices are the scores. Then I can order by price. It's simple, but I have like 400 updates per second.
jasonwatkinspdx 7y ago
You'll get a bunch of garbage answers here to be frank, from people who never put more than trivial load through both. Memcache blows redis out of the water in terms of raw throughput and latency, like it's not even remotely close on modern lots of cores and big memory machines. Redis has a more feature rich api, but has fundamentally hobbled itself on a root hash table and concurrency implementation that is circa 1990. Give memcached a try, even if it means you implement some stuff you currently defer to redis ops. Measure it. Look at the code bases. Make up your own mind about performance and which actually makes your code simpler.
- dijit 7y ago
  http://antirez.com/news/94 http://antirez.com/news/94 TL;DR: Redis is just doing different things, and the slab allocator is pretty efficient in regards to performance but not memory usage.
  - dormando 7y ago
    Fwiw this post comparison is pretty old. The memory usage should be the same or better these days (automatic slab rebalancing, LRU expired item crawler, chunked items, etc).
- haolez 7y ago
  Thanks. I'll make sure to create a PoC and post back here.

by haolez 7y ago

manigandham 7y ago

CPU is cheap and memcached has no advanced data structures. If you're using them then you'll probably have to reimplement everything yourself in memcached with less efficiency.

henriquez 7y ago
You can always just JSON-encode whatever you’re putting in there and decode on the way out. Though I couldn’t imagine using it for pub/sub like the parent commenter is doing with redis.
- manigandham 7y ago
  Which is inefficient and pointless compared to just using Redis with data structures that you can manipulate directly.

dormando 7y ago

Mind giving some examples in more detail about how you make use of lists/scores?

haolez 7y ago
Pub/Sub is easily replaceable. I use lists and scores to store orders from orderbooks, where the prices are the scores. Then I can order by price. It's simple, but I have like 400 updates per second.

jasonwatkinspdx 7y ago

You'll get a bunch of garbage answers here to be frank, from people who never put more than trivial load through both. Memcache blows redis out of the water in terms of raw throughput and latency, like it's not even remotely close on modern lots of cores and big memory machines. Redis has a more feature rich api, but has fundamentally hobbled itself on a root hash table and concurrency implementation that is circa 1990. Give memcached a try, even if it means you implement some stuff you currently defer to redis ops. Measure it. Look at the code bases. Make up your own mind about performance and which actually makes your code simpler.

dijit 7y ago
http://antirez.com/news/94 http://antirez.com/news/94 TL;DR: Redis is just doing different things, and the slab allocator is pretty efficient in regards to performance but not memory usage.
- dormando 7y ago
  Fwiw this post comparison is pretty old. The memory usage should be the same or better these days (automatic slab rebalancing, LRU expired item crawler, chunked items, etc).
haolez 7y ago
Thanks. I'll make sure to create a PoC and post back here.