4 ms·

It doesn't make sense to me to architect these kinds of applications at the database level. What's wrong with using something like redis pubsub? I don't get th

It doesn't make sense to me to architect these kinds of applications at the database level. What's wrong with using something like redis pubsub? I don't get the obsession of evented databases, or implementing this kind of thing at the database level. I suppose its attractive to "listen" to a table for changes but the pattern can be implemented elsewhere and with better tools. Databases should be used for persistence, organization and schema of data, have flexible querying, and not much else.

gamesbrainiac 10y ago
I think that is is also a good idea to have a changefeed caching layer on top of PGNOTIFY, and you can definitely use redis for that.
danbruc 10y ago
I am explicitly not taking any side here, but that is certainly a matter of debate. Especially relational database management systems have all this functionality like views and triggers and stored procedures so that you can move a substantial part of your business logic into the database management system and only have very thin clients in order to minimize duplication of business logic across different clients. Treating a database management system as a dumb data store is only one possible philosophy, admittedly the one that seems to be the dominant one currently, at least as far as I can tell.
ankrgyl 10y ago
You can flip this argument and make a similar point: "What's wrong with using something like Postgres LISTEN? I don't get the obsession of redis pubsub, or implementing this kind of thing outside the database. I suppose its attractive to "subscribe" to a collection for changes but the pattern can be implemented inside a database which has better tools." In more constructive terms, databases are familiar to a lot of folks and have reliable guarantees (persistence, ACID) that are generally useful properties. If you're able to achieve the performance you need within the database, then you get a lot of operational benefits from keeping your workload running inside of it. If your workload for some reason is slow inside of a database, then it certainly makes sense to consider specialized alternatives (like Redis). In my experience, however, you can tune a database to perform better than these tools in pretty much every real-world case (i.e. moderate concurrency with realistic load).
- k__ 10y ago
  On the one hand, I'd say, because of decoupling. If one feature is in Redis and one in Postgress, I can replace each of them more easily. On the other hand, how often did someone replace Postgress in their stack?
- floatboth 10y ago
  Postgres LISTEN/NOTIFY is actually very similar to Redis pubsub. You can manually add triggers to send notification on changes, sure, but the real "in the database" stuff is in RethinkDB.
throwaway91111 10y ago
Aren't they both just the observer pattern? Seems like it falls into the same category as every other "should I implement this in the DB or app land" code. Also, moving your callbacks to the db means you can call into the db from multiple code bases without fear of replicating the callbacks inconsistently. That said, I'm not aware of many people who rely on that functionality.
mamcx 10y ago
In the moment you are out the DB, you are out of ACID. This is a reason that I don't wanna to do event sourcing outside PG.
zapov 10y ago
The great thing with LISTEN/NOTIFY is they work inside transaction boundry. This way you can get correct cache invalidation inside inside multiple table changes really easy. But if you cant use transactions that doesnt seem helpful.
- vosper 10y ago
  That is really interesting - thanks.
Sanddancer 10y ago
You can have both. Postgres has foreign data wrappers, with Redis as a first class citizen. So, you can push your data to Postgres, and have it trigger to publish the updates. Databases have a pretty big toolset to make your life easier when you need to manipulate your data.
bpicolo 10y ago
Code-level change data capture doesn't work well for e.g. many-row updates, one off queries... a ton of different stuff. That's not to say it doesn't have uses, but wanting global-DB-update monitoring is not one of those things where it fits cleanly

gamesbrainiac 10y ago

I think that is is also a good idea to have a changefeed caching layer on top of PGNOTIFY, and you can definitely use redis for that.

danbruc 10y ago

I am explicitly not taking any side here, but that is certainly a matter of debate. Especially relational database management systems have all this functionality like views and triggers and stored procedures so that you can move a substantial part of your business logic into the database management system and only have very thin clients in order to minimize duplication of business logic across different clients. Treating a database management system as a dumb data store is only one possible philosophy, admittedly the one that seems to be the dominant one currently, at least as far as I can tell.

ankrgyl 10y ago

You can flip this argument and make a similar point: "What's wrong with using something like Postgres LISTEN? I don't get the obsession of redis pubsub, or implementing this kind of thing outside the database. I suppose its attractive to "subscribe" to a collection for changes but the pattern can be implemented inside a database which has better tools." In more constructive terms, databases are familiar to a lot of folks and have reliable guarantees (persistence, ACID) that are generally useful properties. If you're able to achieve the performance you need within the database, then you get a lot of operational benefits from keeping your workload running inside of it. If your workload for some reason is slow inside of a database, then it certainly makes sense to consider specialized alternatives (like Redis). In my experience, however, you can tune a database to perform better than these tools in pretty much every real-world case (i.e. moderate concurrency with realistic load).

k__ 10y ago
On the one hand, I'd say, because of decoupling. If one feature is in Redis and one in Postgress, I can replace each of them more easily. On the other hand, how often did someone replace Postgress in their stack?
floatboth 10y ago
Postgres LISTEN/NOTIFY is actually very similar to Redis pubsub. You can manually add triggers to send notification on changes, sure, but the real "in the database" stuff is in RethinkDB.

throwaway91111 10y ago

Aren't they both just the observer pattern? Seems like it falls into the same category as every other "should I implement this in the DB or app land" code. Also, moving your callbacks to the db means you can call into the db from multiple code bases without fear of replicating the callbacks inconsistently. That said, I'm not aware of many people who rely on that functionality.

mamcx 10y ago

In the moment you are out the DB, you are out of ACID. This is a reason that I don't wanna to do event sourcing outside PG.

zapov 10y ago

The great thing with LISTEN/NOTIFY is they work inside transaction boundry. This way you can get correct cache invalidation inside inside multiple table changes really easy. But if you cant use transactions that doesnt seem helpful.

vosper 10y ago
That is really interesting - thanks.

Sanddancer 10y ago

You can have both. Postgres has foreign data wrappers, with Redis as a first class citizen. So, you can push your data to Postgres, and have it trigger to publish the updates. Databases have a pretty big toolset to make your life easier when you need to manipulate your data.

bpicolo 10y ago

Code-level change data capture doesn't work well for e.g. many-row updates, one off queries... a ton of different stuff. That's not to say it doesn't have uses, but wanting global-DB-update monitoring is not one of those things where it fits cleanly