5 ms·

As in, write your entire application in SQL? That sounds like a nightmare of epic proportions. Every time I've tried to use SQL for anything more general-purpos

As in, write your entire application in SQL? That sounds like a nightmare of epic proportions. Every time I've tried to use SQL for anything more general-purpose (like string manipulation, say) I've wanted to gouge my eyes out by the end.

dventimi 3y ago
Yes. Write your entire application in SQL, IF it's the kind of application for which this can be done. Is the application a flight simulator or a game engine or resource manager or a distributed task scheduler? No, that's probably not a good case for SQL. Neither would be a machine learning application that relies heavily on Python packages, and it's easy to think of other examples as well. But, is the application responding to user input to query and display data, perform transformations over those data, impose constraints and invariants over those data, and validate user input against those data? Those are tasks for which relational databases with SQL (and other kinds of databases as well, but here we're talking about SQL) are tailor-made. What then does, say, Python bring to the party?
- TheFlyingFish 3y ago
  The obvious thought that occurs to me is UI - how are users going to access this hypothetical application? If it's via the web, then you'll need a whole setup for generating HTML, and doing that in SQL sounds extremely painful. If it's a native app, then you've got to interact with the OS somehow to do things like draw to the screen, and I've never even heard of trying to do that from SQL. More broadly, this just seems like it would sharply limit how you could extend the application in the future. What if at some point you want to query a web API for some additional data with which to enrich what you're returning from your database? (Not an unusual situation, in my experience.) You'd be stuck trying to make web requests from SQL, which again seems needlessly painful. Note that I'm not against the basic idea of "do as much data-munging in SQL as possible" - in my experience that's a great way to ensure that your application stays fast and efficient. It' just all the ancillary things surrounding the data-munging for which I don't think SQL is the best fit.
  - dventimi 3y ago
    The OP didn't write about a UI and we don't know what their needs are, so for all we know they're writing a Python-based back-end API in REST or GraphQL which is being consumed by a mobile UI or a SPA UI in React or Vue.js or whatever, such that that back-end API doesn't have to provide a UI. If the existing Python back-end doesn't provide a UI then any proposed substitute--including one in SQL--shouldn't have to provide a UI either, just an API. In that case, there are some ready-made solutions already available: - PostgREST (https://postgrest.org/ https://postgrest.org/): REST API for PostgreSQL - PostGraphile (https://www.graphile.org/ https://www.graphile.org/) GraphQL API for PostgreSQL - pg_graphql (https://github.com/supabase/pg_graphql https://github.com/supabase/pg_graphql) GraphQL API for PostgreSQL - Hasura (https://hasura.io/ https://hasura.io/) GraphQL API for various databases If you want to blend data from a web API and you're content with GraphQL, for some use-cases (not all, but some), there are options: - Apollo Federation (https://www.apollographql.com/apollo-federation/ https://www.apollographql.com/apollo-federation/) - GraphQL Mesh (https://the-guild.dev/graphql/mesh https://the-guild.dev/graphql/mesh) - Hasura Remote Schema (https://hasura.io/blog/tagged/remote-schemas/ https://hasura.io/blog/tagged/remote-schemas/) If you want more control over the web API and you were going to fetch the data within your Python back-end and process it there, for some use-cases (not all, but some), there are options: - pg_http (https://github.com/pramsey/pgsql-http https://github.com/pramsey/pgsql-http) Life is about trade-offs. Doing the work in SQL is not without its drawbacks, but it's also not without its benefits, and that's true for doing the work in a general-purpose language as well. Whatever the drawbacks of doing it in SQL, one of the benefits has got to be eliminating the impedance mismatch (for people who regard that mismatch as a problem, and the OP seems to be one such person). What I claim is that doing the work directly in the database shouldn't be ruled out in general (the specifics of a given use-case may rule it out in particular) any more than the other common patterns (API hand-written in Python, for instance) shouldn't be ruled out in general.
    - cryptonector 3y ago
      > The OP didn't write about a UI and we don't know what their needs are, [...] > What I claim is that doing the work directly in the database shouldn't be ruled out in general (the specifics of a given use-case may rule it out in particular) any more than the other common patterns (API hand-written in Python, for instance) shouldn't be ruled out in general. Hear hear. With PG's COMMENT feature one can enrich SQL schema with the sorts of metadata one needs for UI generation. Here's a script that generates a JSON view of a PG SQL schema enriched with JSON from COMMENTs: https://github.com/twosigma/postgresql-contrib/blob/master/schema2json.sh https://github.com/twosigma/postgresql-contrib/blob/master/s... and https://github.com/twosigma/postgresql-contrib/blob/master/schema2json.sql https://github.com/twosigma/postgresql-contrib/blob/master/s...

dventimi 3y ago

Yes. Write your entire application in SQL, IF it's the kind of application for which this can be done. Is the application a flight simulator or a game engine or resource manager or a distributed task scheduler? No, that's probably not a good case for SQL. Neither would be a machine learning application that relies heavily on Python packages, and it's easy to think of other examples as well. But, is the application responding to user input to query and display data, perform transformations over those data, impose constraints and invariants over those data, and validate user input against those data? Those are tasks for which relational databases with SQL (and other kinds of databases as well, but here we're talking about SQL) are tailor-made. What then does, say, Python bring to the party?

TheFlyingFish 3y ago
The obvious thought that occurs to me is UI - how are users going to access this hypothetical application? If it's via the web, then you'll need a whole setup for generating HTML, and doing that in SQL sounds extremely painful. If it's a native app, then you've got to interact with the OS somehow to do things like draw to the screen, and I've never even heard of trying to do that from SQL. More broadly, this just seems like it would sharply limit how you could extend the application in the future. What if at some point you want to query a web API for some additional data with which to enrich what you're returning from your database? (Not an unusual situation, in my experience.) You'd be stuck trying to make web requests from SQL, which again seems needlessly painful. Note that I'm not against the basic idea of "do as much data-munging in SQL as possible" - in my experience that's a great way to ensure that your application stays fast and efficient. It' just all the ancillary things surrounding the data-munging for which I don't think SQL is the best fit.
- dventimi 3y ago
  The OP didn't write about a UI and we don't know what their needs are, so for all we know they're writing a Python-based back-end API in REST or GraphQL which is being consumed by a mobile UI or a SPA UI in React or Vue.js or whatever, such that that back-end API doesn't have to provide a UI. If the existing Python back-end doesn't provide a UI then any proposed substitute--including one in SQL--shouldn't have to provide a UI either, just an API. In that case, there are some ready-made solutions already available: - PostgREST (https://postgrest.org/ https://postgrest.org/): REST API for PostgreSQL - PostGraphile (https://www.graphile.org/ https://www.graphile.org/) GraphQL API for PostgreSQL - pg_graphql (https://github.com/supabase/pg_graphql https://github.com/supabase/pg_graphql) GraphQL API for PostgreSQL - Hasura (https://hasura.io/ https://hasura.io/) GraphQL API for various databases If you want to blend data from a web API and you're content with GraphQL, for some use-cases (not all, but some), there are options: - Apollo Federation (https://www.apollographql.com/apollo-federation/ https://www.apollographql.com/apollo-federation/) - GraphQL Mesh (https://the-guild.dev/graphql/mesh https://the-guild.dev/graphql/mesh) - Hasura Remote Schema (https://hasura.io/blog/tagged/remote-schemas/ https://hasura.io/blog/tagged/remote-schemas/) If you want more control over the web API and you were going to fetch the data within your Python back-end and process it there, for some use-cases (not all, but some), there are options: - pg_http (https://github.com/pramsey/pgsql-http https://github.com/pramsey/pgsql-http) Life is about trade-offs. Doing the work in SQL is not without its drawbacks, but it's also not without its benefits, and that's true for doing the work in a general-purpose language as well. Whatever the drawbacks of doing it in SQL, one of the benefits has got to be eliminating the impedance mismatch (for people who regard that mismatch as a problem, and the OP seems to be one such person). What I claim is that doing the work directly in the database shouldn't be ruled out in general (the specifics of a given use-case may rule it out in particular) any more than the other common patterns (API hand-written in Python, for instance) shouldn't be ruled out in general.
  - cryptonector 3y ago
    > The OP didn't write about a UI and we don't know what their needs are, [...] > What I claim is that doing the work directly in the database shouldn't be ruled out in general (the specifics of a given use-case may rule it out in particular) any more than the other common patterns (API hand-written in Python, for instance) shouldn't be ruled out in general. Hear hear. With PG's COMMENT feature one can enrich SQL schema with the sorts of metadata one needs for UI generation. Here's a script that generates a JSON view of a PG SQL schema enriched with JSON from COMMENTs: https://github.com/twosigma/postgresql-contrib/blob/master/schema2json.sh https://github.com/twosigma/postgresql-contrib/blob/master/s... and https://github.com/twosigma/postgresql-contrib/blob/master/schema2json.sql https://github.com/twosigma/postgresql-contrib/blob/master/s...