4 ms·

A reasonably configured Postgres is easily sub-millisecond for reads and writes on a small dataset like this. I suspect that the author is doing something quit

by fizx 7y ago

A reasonably configured Postgres is easily sub-millisecond for reads and writes on a small dataset like this. I suspect that the author is doing something quite wrong. People generally reach for Redis too quickly, IMO. Postgres is about as fast, has better tooling, and will slow down gracefully as the dataset increases, rather than evicting records as the limit is hit. That said, I guess if you're relatively junior, it's hard to grossly misconfigure Redis, so that's a point in its favor. Somewhat like Java vs Node.js--you can write higher performance network services in Java, but it's a lot easier to make mistakes.

Benjamin_Dobell 7y ago
> That said, I guess if you're relatively junior, it's hard to grossly misconfigure Redis, so that's a point in its favor. Redis persistence (or possibly lack-thereof) is probably quite complicated for a junior developer to understand. I'd actually expect a better outcome with PSQL.
- swsieber 7y ago
  I disagree. But anytime I've been exposed to redis, it's always been as an ephemeral data store.
- threeseed 7y ago
  Not sure what you are getting at here. Redis is one of the simplest data stores around to use and its persistence is solid.
  - erichocean 7y ago
    > Redis is one of the simplest data stores around to use and its persistence is solid. I've lost countless amount of data in production systems running Redis. Haven't ever lost anything in Postgres. (I still run both.)
throwawayy98121 7y ago
Isn’t the fundamental difference also use case? You would have to use Postgres if you need actual durability. AFAIK, one outage and your redis cache is gone... unless you’re running an HA cluster (which I dunno if that’s offered by redis). But even that doesn’t mean you have durability.
- iliketoworkhard 7y ago
  In general caching systems are volatile and not meant to be used to store data that cannot be afforded to be lost. Of course that doesn't stop clients from storing critical data on there in return for the simple GET/PUT APIs and low latencies. You can go for more complex caches such as Hazelcast which offers hot restarts by persisting data to disk.
- derefr 7y ago
  Redis can be configured as `appendonly yes; appendfsync always`, which the Redis docs say is “very very slow”, but really would just make Redis work the way Postgres does with its WAL, with probably equivalent performance (or still slightly better, since there’s no checkpointing going on.)
threeseed 7y ago
You have no evidence that: a) PostgreSQL has sub-millisecond read/writes for storing JSON blobs than Redis. b) Java results in buggier code than Javascript. If you're going to make bold claims the onus is on you to provide evidence to back it up.
Someone 7y ago
There’s class AmazonAffiliateLookup(models.Model, TotalCountMixin) I guess that TotalCountMixin adds some triggers that maintain a count of the records in the table in a separate table (could or could not be a good idea, depending on the table size, relative frequency of ‘give me the number of items’ calls relative to inserts and deletes, etc, but might introduce a bottleneck on updating that count) There also is: song = models.ForeignKey(Song, on_delete=models.CASCADE) I don’t understand why that foreign key is there, but it can’t speed up inserts and deletes. Then, we have: created = models.DateTimeField(auto_now_add=True, db_index=True) I think creating an index is done to implement the equivalent of Redis’ auto-expiry. Again, that may be suboptimal if there are few records in the database. DateTimeField also may be ‘too structured’ for the task at hand (it might needlessly parse year/month/day out of time stamp in nanoseconds, for example, or even do time zone calculations to convert a local time to UTC), but I don’t know. It would be helpful if the text showed the actual SQL DDL.
tracker1 7y ago
Probably, hard to tell, the biggest difference, is Redis is likely storing the JSON as a string, and Posgres is storing it as binary json (separated out into distinct values/collections). I'd guess if Postgres was using UTF8 text for the JSON it would be closer. Also, Postgres is probably a bit less efficient on persistance requirements in favor of higher stability and preventing data loss. In any case, Redis is probably a more natural and better fit (remote API result caching). That said, the comparison is awkward at best. If it were to cache localized results, where the dbms (PostgreSQL) is the source of truth, then adding a caching layer above the db may or may not be worth it. I think the performance comparisons come down to Granny Smith Apples vs Rome Apples... you can bake a pie with either, but they don't taste the same.

by fizx 7y ago

Benjamin_Dobell 7y ago

> That said, I guess if you're relatively junior, it's hard to grossly misconfigure Redis, so that's a point in its favor. Redis persistence (or possibly lack-thereof) is probably quite complicated for a junior developer to understand. I'd actually expect a better outcome with PSQL.

swsieber 7y ago
I disagree. But anytime I've been exposed to redis, it's always been as an ephemeral data store.
threeseed 7y ago
Not sure what you are getting at here. Redis is one of the simplest data stores around to use and its persistence is solid.
- erichocean 7y ago
  > Redis is one of the simplest data stores around to use and its persistence is solid. I've lost countless amount of data in production systems running Redis. Haven't ever lost anything in Postgres. (I still run both.)

throwawayy98121 7y ago

Isn’t the fundamental difference also use case? You would have to use Postgres if you need actual durability. AFAIK, one outage and your redis cache is gone... unless you’re running an HA cluster (which I dunno if that’s offered by redis). But even that doesn’t mean you have durability.

iliketoworkhard 7y ago
In general caching systems are volatile and not meant to be used to store data that cannot be afforded to be lost. Of course that doesn't stop clients from storing critical data on there in return for the simple GET/PUT APIs and low latencies. You can go for more complex caches such as Hazelcast which offers hot restarts by persisting data to disk.
derefr 7y ago
Redis can be configured as `appendonly yes; appendfsync always`, which the Redis docs say is “very very slow”, but really would just make Redis work the way Postgres does with its WAL, with probably equivalent performance (or still slightly better, since there’s no checkpointing going on.)

threeseed 7y ago

You have no evidence that: a) PostgreSQL has sub-millisecond read/writes for storing JSON blobs than Redis. b) Java results in buggier code than Javascript. If you're going to make bold claims the onus is on you to provide evidence to back it up.

Someone 7y ago

There’s class AmazonAffiliateLookup(models.Model, TotalCountMixin) I guess that TotalCountMixin adds some triggers that maintain a count of the records in the table in a separate table (could or could not be a good idea, depending on the table size, relative frequency of ‘give me the number of items’ calls relative to inserts and deletes, etc, but might introduce a bottleneck on updating that count) There also is: song = models.ForeignKey(Song, on_delete=models.CASCADE) I don’t understand why that foreign key is there, but it can’t speed up inserts and deletes. Then, we have: created = models.DateTimeField(auto_now_add=True, db_index=True) I think creating an index is done to implement the equivalent of Redis’ auto-expiry. Again, that may be suboptimal if there are few records in the database. DateTimeField also may be ‘too structured’ for the task at hand (it might needlessly parse year/month/day out of time stamp in nanoseconds, for example, or even do time zone calculations to convert a local time to UTC), but I don’t know. It would be helpful if the text showed the actual SQL DDL.

tracker1 7y ago

Probably, hard to tell, the biggest difference, is Redis is likely storing the JSON as a string, and Posgres is storing it as binary json (separated out into distinct values/collections). I'd guess if Postgres was using UTF8 text for the JSON it would be closer. Also, Postgres is probably a bit less efficient on persistance requirements in favor of higher stability and preventing data loss. In any case, Redis is probably a more natural and better fit (remote API result caching). That said, the comparison is awkward at best. If it were to cache localized results, where the dbms (PostgreSQL) is the source of truth, then adding a caching layer above the db may or may not be worth it. I think the performance comparisons come down to Granny Smith Apples vs Rome Apples... you can bake a pie with either, but they don't taste the same.