3 ms·

> I never understood why everyone thought it was necessary to create HTTP API layers in front of every database Mostly because you usually need to do some kind

by s_kilk 10y ago

> I never understood why everyone thought it was necessary to create HTTP API layers in front of every database Mostly because you usually need to do some kind of logic that's either difficult or impossible to represent in SQL views or functions. For example, how would your SQL view access something like redis? or any kind of external cache? or an HTTP service? This is the kind of stuff that's pretty common in HTTP Controller functions, and would be a nightmare to move into the database. (Note: I'm generally in favor of using SQL views/functions for data-facing logic, just, y'know, not a silver bullet.)

derefr 10y ago
> For example, how would your SQL view access something like redis? or any kind of external cache? or an HTTP service? Smug comeback: https://wiki.postgresql.org/wiki/Foreign_data_wrappers https://wiki.postgresql.org/wiki/Foreign_data_wrappers But your point still stands; no other RDBMS has those, so it can't really be an assumption. Still, there is a mismatch of paradigms here. The people who built the RDBMSes assumed that everyone would import data into them from wherever else. You, the user (or you, the application developer) shouldn't need to query a third-party web service when doing your database query; something else was supposed to do that query in expectation of your need, and shove the results into the database, so you could just treat that data as another table of useful information to JOIN on. People who front databases with web APIs tend to treat the database itself as a dumb store—one among many—that's supposed to quickly return "raw" data from single tables so that the web service can chew it together with other data to compute answers to questions. But, by doing so, they're in effect creating their own bespoke Database Management System: the web service becomes the thing that does efficient OnLine Analytical Processing of manifold backing datasets. (Or, at least, attempts to.) A DBMS's job is to ingest data and process queries to spit out answers. It's not your job; it's not your app's job; it's the DBMS's job. The DBMS has decades of optimization for that use-case; and in fact, that's the meaning of the word "database management system", when defined in opposition to a "flat-file database" like BDB or LevelDB. The fact that there's a daemon sitting there between you and your databases allows the daemon to index and join and expose views and run checks and enforce rules and on and on and on. It allows the daemon to do asynchronous, streaming imports of data-sets from remote sources, so that its data is always up-to-date. It allows the daemon to be there for things like crawler agents to dump their results into, rather than dumping them to something like an S3 bucket where you later have to be the one to import them in. In other words, being a daemon allows the DBMS to be the service that you just come to and ask questions—arbitrarily-complex analytical queries—and get answers. But that only works if the database management system itself is given access methods to all the databases that contain the data to answer your question. And, in modern practice, we've essentially forgotten that that is a thing that we ever wanted to do. Who would teach a database management system anything? Database connector libraries don't belong in DBMSes; they belong scattered throughout our toppling edifice of RESTful API microservices! (And if you argue that we still need those RESTful APIs for things like browser-clients wanting to do queries using AJAX: there's no reason DBMSes—not just document stores but full-fledged OLAP DBMSes—can't speak HTTP as well. CouchDB is a great but strangely-unique example. Adapter gateways also exist, like PostgREST.)

by s_kilk 10y ago

derefr 10y ago

> For example, how would your SQL view access something like redis? or any kind of external cache? or an HTTP service? Smug comeback: https://wiki.postgresql.org/wiki/Foreign_data_wrappers https://wiki.postgresql.org/wiki/Foreign_data_wrappers But your point still stands; no other RDBMS has those, so it can't really be an assumption. Still, there is a mismatch of paradigms here. The people who built the RDBMSes assumed that everyone would import data into them from wherever else. You, the user (or you, the application developer) shouldn't need to query a third-party web service when doing your database query; something else was supposed to do that query in expectation of your need, and shove the results into the database, so you could just treat that data as another table of useful information to JOIN on. People who front databases with web APIs tend to treat the database itself as a dumb store—one among many—that's supposed to quickly return "raw" data from single tables so that the web service can chew it together with other data to compute answers to questions. But, by doing so, they're in effect creating their own bespoke Database Management System: the web service becomes the thing that does efficient OnLine Analytical Processing of manifold backing datasets. (Or, at least, attempts to.) A DBMS's job is to ingest data and process queries to spit out answers. It's not your job; it's not your app's job; it's the DBMS's job. The DBMS has decades of optimization for that use-case; and in fact, that's the meaning of the word "database management system", when defined in opposition to a "flat-file database" like BDB or LevelDB. The fact that there's a daemon sitting there between you and your databases allows the daemon to index and join and expose views and run checks and enforce rules and on and on and on. It allows the daemon to do asynchronous, streaming imports of data-sets from remote sources, so that its data is always up-to-date. It allows the daemon to be there for things like crawler agents to dump their results into, rather than dumping them to something like an S3 bucket where you later have to be the one to import them in. In other words, being a daemon allows the DBMS to be the service that you just come to and ask questions—arbitrarily-complex analytical queries—and get answers. But that only works if the database management system itself is given access methods to all the databases that contain the data to answer your question. And, in modern practice, we've essentially forgotten that that is a thing that we ever wanted to do. Who would teach a database management system anything? Database connector libraries don't belong in DBMSes; they belong scattered throughout our toppling edifice of RESTful API microservices! (And if you argue that we still need those RESTful APIs for things like browser-clients wanting to do queries using AJAX: there's no reason DBMSes—not just document stores but full-fledged OLAP DBMSes—can't speak HTTP as well. CouchDB is a great but strangely-unique example. Adapter gateways also exist, like PostgREST.)