4 ms·

Er, I really feel like you’re not understanding how simple AWS makes managing a Redis instance. I’ve been using Redis for nearly 10 years and it’s been a seaml

Er, I really feel like you’re not understanding how simple AWS makes managing a Redis instance. I’ve been using Redis for nearly 10 years and it’s been a seamless and pleasant experience. Honestly it sounds like you’re taking your specific experiences and overgeneralizing.

maxbond 4y ago
Partly I'm sharing war stories because that's a fun thing to do, I'm not being entirely serious. Partly it's that you said that wasn't your experience of reality, so I broke off a little piece of mine and offered it to you. I'd suggest we're both generalizing, and as long as it's not taken too seriously, that's fine; it's how shop talk works. I don't know the nature of the applications you've been working on those last 10 years, but it was more or less the main database for a high bandwidth, low latency service I was working on, also using Elasticache. Problem spaces vary. If you're using it as a cache with modest load and consistency requirements, maybe you never need to understand it. But those sorts of requirements often creep & change out from under you. So if you're saying, Elasticache did a good job of abstracting Redis, sure, I agree. If you're saying, there is no additional cognitive load to adopting a new service in your data path, because you don't even need to understand it - that puts a shiver up my spine, and makes me hear Pagerduty alerts in my head.
- datalopers 4y ago
  Keep in mind the context of this thread. It's some shitty blog suggesting you do every fuck thing on postgres. Myself and the other guy are simply suggesting that running a single-node instance of redis is an infinitely better and simpler choice than implementing a cache or a job queue on a rdbms. I don't feel designing for a guaranteed high-availability application was part of the discussion at all.
  - maxbond 4y ago
    If you don't find it applicable, feel free to discard what I'm saying, I take no offense. But I just don't quite understand your perspective. Maybe I have oncall firefighter brain rot, but a distributed job queue is exactly the sort of thing I'd want to be available, and a cache is something I regard as being very dangerous and requiring utmost care.
    - tekknik 4y ago
      You use SQS, couple clicks and you’re done. You shouldn’t be using a cache for a queue. You use a queue for a queue.
      - maxbond 4y ago
        Not disagreeing, but purely for interest, Redis contains a queue-like primitive. https://redis.io/docs/data-types/streams/ https://redis.io/docs/data-types/streams/ You can make good job queues out of this, combined with sharding or consistent hashing, for low(ish) latency applications. Each shard has a stream, they operate on data stored in Redis, and you pass them the key to this data over their stream. But SQS is great, and a great rebuttal to the article. Totally easier to prototype a job queue that way than with pg, and you probably won't need to move off of it.
        tekknik 4y ago
        Sure, but you still need to run redis vs click a couple of buttons and create a queue, offloading the entire management to AWS staff.
        julik 4y ago
        I have used both SQS and PG-based queues and for the smaller workloads/smaller systems (read: "not very very large systems") I now prefer the latter. There is also a non-trivial amount of stuff that we turned out to need for operating SQS at scale on the application side, basically to compensate for the things SQS does not have. It is great it doesn't have those things, but if you have a smaller application you might want to have those things instead and sacrifice a bit of scalability.
        maxbond 4y ago
        The advantage being that you could sort things to implement priorities and such? Did you use listen()/notify() at all? ETA: seeing your list of missing features now, that all makes a lot of sense. In my mind the biggest advantage of SQS is that it glues together all the other AWS offerings, so you can go AWS -> Lambda for an instant job queue (with concurrency limits, etc. so you don't blow your hand off - perhaps undermining the simplicity argument). But everything you're saying makes sense if your job queue needs any degree of sophistication.

maxbond 4y ago

Partly I'm sharing war stories because that's a fun thing to do, I'm not being entirely serious. Partly it's that you said that wasn't your experience of reality, so I broke off a little piece of mine and offered it to you. I'd suggest we're both generalizing, and as long as it's not taken too seriously, that's fine; it's how shop talk works. I don't know the nature of the applications you've been working on those last 10 years, but it was more or less the main database for a high bandwidth, low latency service I was working on, also using Elasticache. Problem spaces vary. If you're using it as a cache with modest load and consistency requirements, maybe you never need to understand it. But those sorts of requirements often creep & change out from under you. So if you're saying, Elasticache did a good job of abstracting Redis, sure, I agree. If you're saying, there is no additional cognitive load to adopting a new service in your data path, because you don't even need to understand it - that puts a shiver up my spine, and makes me hear Pagerduty alerts in my head.

datalopers 4y ago
Keep in mind the context of this thread. It's some shitty blog suggesting you do every fuck thing on postgres. Myself and the other guy are simply suggesting that running a single-node instance of redis is an infinitely better and simpler choice than implementing a cache or a job queue on a rdbms. I don't feel designing for a guaranteed high-availability application was part of the discussion at all.
- maxbond 4y ago
  If you don't find it applicable, feel free to discard what I'm saying, I take no offense. But I just don't quite understand your perspective. Maybe I have oncall firefighter brain rot, but a distributed job queue is exactly the sort of thing I'd want to be available, and a cache is something I regard as being very dangerous and requiring utmost care.
  - tekknik 4y ago
    You use SQS, couple clicks and you’re done. You shouldn’t be using a cache for a queue. You use a queue for a queue.
    - maxbond 4y ago
      Not disagreeing, but purely for interest, Redis contains a queue-like primitive. https://redis.io/docs/data-types/streams/ https://redis.io/docs/data-types/streams/ You can make good job queues out of this, combined with sharding or consistent hashing, for low(ish) latency applications. Each shard has a stream, they operate on data stored in Redis, and you pass them the key to this data over their stream. But SQS is great, and a great rebuttal to the article. Totally easier to prototype a job queue that way than with pg, and you probably won't need to move off of it.
      - tekknik 4y ago
        Sure, but you still need to run redis vs click a couple of buttons and create a queue, offloading the entire management to AWS staff.
      - julik 4y ago
        I have used both SQS and PG-based queues and for the smaller workloads/smaller systems (read: "not very very large systems") I now prefer the latter. There is also a non-trivial amount of stuff that we turned out to need for operating SQS at scale on the application side, basically to compensate for the things SQS does not have. It is great it doesn't have those things, but if you have a smaller application you might want to have those things instead and sacrifice a bit of scalability.
        maxbond 4y ago
        The advantage being that you could sort things to implement priorities and such? Did you use listen()/notify() at all? ETA: seeing your list of missing features now, that all makes a lot of sense. In my mind the biggest advantage of SQS is that it glues together all the other AWS offerings, so you can go AWS -> Lambda for an instant job queue (with concurrency limits, etc. so you don't blow your hand off - perhaps undermining the simplicity argument). But everything you're saying makes sense if your job queue needs any degree of sophistication.