3 ms·

Yeah, the article was like "I always need a DB anyway" but then sets up an extra cronjob to expire keys, plus more code. I get YAGNI and avoiding deps, but this

Yeah, the article was like "I always need a DB anyway" but then sets up an extra cronjob to expire keys, plus more code. I get YAGNI and avoiding deps, but this is really extra stuff to deal with. Maybe Postgres could use a caching feature. Until then, I'm gonna drop in Redis or memcached instead of reinventing the wheel.

maxbond 1y ago
Expiring keys in Postgres with a created_at column and a pg_cron job is very easy (at least, if you're comfortable in Postgres). Redis is world class though of course, and can be deployed turn-key in basically any environment. If you're more comfortable in Redis than Postgres, more power to you. Different choices can be pragmatic to different people. Personally for a greenfield project, my thinking would be that I am paying for Postgres already. So I would want to avoid paying for Redis too. My Postgres database is likely to be underutilized until (and unless) I get any real scale. So adding caching to it is free in terms of dollars.
- frollogaston 1y ago
  I'm comfy with Postgres though, like I'll center my entire backend around it and do the heavy lifting in SQL (never ORM). It's more that I don't want to depend on a cronjob for something as fundamental as a cache. Usually Postgres costs a lot more than Redis if you're paying for a platform. Like a decent Redis or memcached in Heroku is free. And I don't want to waste precious Postgres connections or risk bogging down the whole DB if there's lots of cache usage, which actually happened last time I tried skipping Redis.
  - maxbond 1y ago
    I can understand being nervous about some cron job running on some other service, but what's concerning about a cron job managed inside of Postgres with pg_cron? If that doesn't run, your database is probably down anyway. Postgres might cost more but I'm probably already paying. I agree that exhausting connections and writing at a high rate are easy ways to bring down Postgres, but I'm personally not going to worry about exhausting connections to Postgres until I have at least a thousand of them. Everything has to be considered within the actual problem you are solving, there are definitely situations to start out with a cache.
    - frollogaston 1y ago
      I might be ok with this if it were built in, but pg_cron is an extension, so first off you might not even have access to it. And then you still have to monitor it.
      - maxbond 1y ago
        Seems like it's available on all the major providers.
        frollogaston 1y ago
        Heroku doesn't have it. That's actually kinda annoying that they don't, cause the others do. AND they no longer have free Redis, so that changes things a bit. Edit: well a tiny bit, max $3/mo
  - motorest 1y ago
    > Usually Postgres costs a lot more than Redis if you're paying for a platform. You need to back up your unbelievable assertion with facts. Memory cache is typically far more expensive than a simple database, specially as provisioning the same memory capacity as RAM is orders of magnitude more expensive than storing the equivalent data in a database.
    - frollogaston 1y ago
      I didn't say it's cheaper for the same cache size. But yeah a base tier Redis that will carry a small project tends to be a lot cheaper than the base tier Postgres.
      - motorest 1y ago
        > I didn't say it's cheaper for the same cache size. So be specific. What exactly did you wanted to say? > But yeah a base tier Redis that will carry a small project tends to be a lot cheaper than the base tier Postgres. This is patently false. I mean,some cloud providers offer nosql databases with sub-20ms performance as part of their free tier. Just go ahead and provide any evidence, any at all,that support the idea that Redis is cheaper than Postgres. Any concrete data will do.
        frollogaston 1y ago
        Look at the Heroku pricing. If you don't like Heroku then look at AWS pricing. Specifically for Postgres, not a NoSQL DB (which Redis can be too)
    - MangoToupe 1y ago
      Why would you cache the entire database though? Seems like an apples to oranges comparison.
      - motorest 1y ago
        > Why would you cache the entire database though? I have no ideas where did you got that from.
motorest 1y ago
> Yeah, the article was like "I always need a DB anyway" but then sets up an extra cronjob to expire keys, plus more code. You do not need cron jobs to do cache. Sometimes you don't even need a TTL. All you need is a way to save data in a way that is easy and cheaper to retrieve. I feel these comments just misinterpret what a cache is by confusing it with what some specific implementation does. Perhaps that's why we see expensive and convoluted strategies using Redis and the like when they are absolutely not needed at all.
- maxbond 1y ago
  If we don't use a TTL, aren't we going to have to either accept that our cache will grow without bounds or take an even more sophisticated approach (like tracking access times instead of creation times)? Is there something simpler I'm not seeing?
  - frollogaston 1y ago
    I've tried it this way. You can get away with no TTL if your keys are constrained. Sometimes there are enough keys to be a problem. I'd rather just set up a TTL and not worry about this.
    - maxbond 1y ago
      Agreed, simple and widely applicable heuristics are great, and you can think deeply on it when and if it proves to be an issue worthy of such attention.
  - motorest 1y ago
    > If we don't use a TTL, aren't we going to have to either accept that our cache will grow without bounds (...) Do you have a bound? I mean, with Redis you do, but that's primarily a cost-driven bound. Nevertheless, I think you're confusing the point of a TTL. TTLs are not used to limit how much data you cache. The whole point of a TTL is to be able to tell whether a cache entry is still fresh or it is stale and must be revalidated. Just because some cache strategies use TTL to determine what entry they should evict, that is just a scenario that takes place when memory is at full capacity.

maxbond 1y ago

Expiring keys in Postgres with a created_at column and a pg_cron job is very easy (at least, if you're comfortable in Postgres). Redis is world class though of course, and can be deployed turn-key in basically any environment. If you're more comfortable in Redis than Postgres, more power to you. Different choices can be pragmatic to different people. Personally for a greenfield project, my thinking would be that I am paying for Postgres already. So I would want to avoid paying for Redis too. My Postgres database is likely to be underutilized until (and unless) I get any real scale. So adding caching to it is free in terms of dollars.

frollogaston 1y ago
I'm comfy with Postgres though, like I'll center my entire backend around it and do the heavy lifting in SQL (never ORM). It's more that I don't want to depend on a cronjob for something as fundamental as a cache. Usually Postgres costs a lot more than Redis if you're paying for a platform. Like a decent Redis or memcached in Heroku is free. And I don't want to waste precious Postgres connections or risk bogging down the whole DB if there's lots of cache usage, which actually happened last time I tried skipping Redis.
- maxbond 1y ago
  I can understand being nervous about some cron job running on some other service, but what's concerning about a cron job managed inside of Postgres with pg_cron? If that doesn't run, your database is probably down anyway. Postgres might cost more but I'm probably already paying. I agree that exhausting connections and writing at a high rate are easy ways to bring down Postgres, but I'm personally not going to worry about exhausting connections to Postgres until I have at least a thousand of them. Everything has to be considered within the actual problem you are solving, there are definitely situations to start out with a cache.
  - frollogaston 1y ago
    I might be ok with this if it were built in, but pg_cron is an extension, so first off you might not even have access to it. And then you still have to monitor it.
    - maxbond 1y ago
      Seems like it's available on all the major providers.
      - frollogaston 1y ago
        Heroku doesn't have it. That's actually kinda annoying that they don't, cause the others do. AND they no longer have free Redis, so that changes things a bit. Edit: well a tiny bit, max $3/mo
- motorest 1y ago
  > Usually Postgres costs a lot more than Redis if you're paying for a platform. You need to back up your unbelievable assertion with facts. Memory cache is typically far more expensive than a simple database, specially as provisioning the same memory capacity as RAM is orders of magnitude more expensive than storing the equivalent data in a database.
  - frollogaston 1y ago
    I didn't say it's cheaper for the same cache size. But yeah a base tier Redis that will carry a small project tends to be a lot cheaper than the base tier Postgres.
    - motorest 1y ago
      > I didn't say it's cheaper for the same cache size. So be specific. What exactly did you wanted to say? > But yeah a base tier Redis that will carry a small project tends to be a lot cheaper than the base tier Postgres. This is patently false. I mean,some cloud providers offer nosql databases with sub-20ms performance as part of their free tier. Just go ahead and provide any evidence, any at all,that support the idea that Redis is cheaper than Postgres. Any concrete data will do.
      - frollogaston 1y ago
        Look at the Heroku pricing. If you don't like Heroku then look at AWS pricing. Specifically for Postgres, not a NoSQL DB (which Redis can be too)
  - MangoToupe 1y ago
    Why would you cache the entire database though? Seems like an apples to oranges comparison.
    - motorest 1y ago
      > Why would you cache the entire database though? I have no ideas where did you got that from.

motorest 1y ago

> Yeah, the article was like "I always need a DB anyway" but then sets up an extra cronjob to expire keys, plus more code. You do not need cron jobs to do cache. Sometimes you don't even need a TTL. All you need is a way to save data in a way that is easy and cheaper to retrieve. I feel these comments just misinterpret what a cache is by confusing it with what some specific implementation does. Perhaps that's why we see expensive and convoluted strategies using Redis and the like when they are absolutely not needed at all.

maxbond 1y ago
If we don't use a TTL, aren't we going to have to either accept that our cache will grow without bounds or take an even more sophisticated approach (like tracking access times instead of creation times)? Is there something simpler I'm not seeing?
- frollogaston 1y ago
  I've tried it this way. You can get away with no TTL if your keys are constrained. Sometimes there are enough keys to be a problem. I'd rather just set up a TTL and not worry about this.
  - maxbond 1y ago
    Agreed, simple and widely applicable heuristics are great, and you can think deeply on it when and if it proves to be an issue worthy of such attention.
- motorest 1y ago
  > If we don't use a TTL, aren't we going to have to either accept that our cache will grow without bounds (...) Do you have a bound? I mean, with Redis you do, but that's primarily a cost-driven bound. Nevertheless, I think you're confusing the point of a TTL. TTLs are not used to limit how much data you cache. The whole point of a TTL is to be able to tell whether a cache entry is still fresh or it is stale and must be revalidated. Just because some cache strategies use TTL to determine what entry they should evict, that is just a scenario that takes place when memory is at full capacity.