4 ms·

If there is going to be any system capable of supporting creating something like The Matrix, it’s going to be Postgres. Tries to eat all application logic by su

by mlthoughts2018 7y ago

If there is going to be any system capable of supporting creating something like The Matrix, it’s going to be Postgres. Tries to eat all application logic by suggested move in database, use language extensions. I’ve only ever found it to be a false promise. You get locked inside the db and can’t create application logic outside the db, and the internal language extensions are super poor solutions for this if you need multiple environments, multiple packages, different versions of the same extension-based app. I love Postgres as a db, but I feel like all the viral marketing for moving to internal db app logic, especially with extensions, is a big trap, and the tutorials & blog posts give you some really superficial use cases to motivate it, and in reality you’re speeding toward a cliff.

kid_atticus 7y ago
I'd say the real value of the 'application-like' capabilities of Postgres is for analytics and offline tasks where there may not even be an application to 'eat'. This article is a great example, anyone can get this working on their laptop easily and just feed new data into the structure they have created and trust that it will just work. Now, if this is powering some actual business logic then that's can cause problems since now your application is tightly coupled to a specific implementation in a specific database technology that may or may not be testable, versioned, or observable. But if you're a data hacker trying to answer questions, produce a report, or tease out features for a ML model having a such a robust easy to manage tool in your toolbelt is super useful. Application Postgres and Analytics Postgres are really different animals, it's pretty cool that one piece of software can do both so well and be so many different things to different people, but yeah I agree you can into real trouble if you use the wrong part of the tool for the job.
CuriouslyC 7y ago
If you have a problem that is never going to be google scale, and you have people who are familiar with postgres, you can deliver applications that are a fraction of the code with superior performance. The only problem with db-centric applications is the tooling, which is inferior to standard application tooling at this point, in terms of version control and testability. Given the hype train building around postgres as a platform, I expect this is going to change pretty quickly.
- mlthoughts2018 7y ago
  Disagree. For example, putting machine learning in the database is miserable. The same is true for other X besides ML. Does not matter what the scale is.
  - lootsauce 7y ago
    We have found in database ML to be great to productionalize basic machine-learning methods. We use BigQuery ML and it saved us a ton of fussing with various frameworks, and because we were already in BQ it saved us the moving of data from database to ML cluster and back into database. We simply author some SQL and get great results quickly.
    - mlthoughts2018 7y ago
      That strikes me as bad usage of ML algorithms for quite a few reasons. People who say this kind of thing usually are not ML practitioners and want to drink some Kool Aid that they can save money & avoid hiring them with techniques like this. How are you benchmarking the in-DB models against variations that can improve them? If they are basic regression models, how are you solving nuanced problems relating to coefficient estimates that cannot be solved with plug and chug libraries (such as from the article “Let’s Put the Garbage Can Regressions and Garbage Can Probits Where They Belong” by Achen)? I’d venture to guess you’re assuming one particular type of model just works, and either are unaware it is failing, or are unaware of wasted money left on the table by not improving or understanding it. This is extremely common especially with simpler use cases. For example, you might think A/B testing is easily solved as an abstract problem and you just need a framework. What could be hard? I’d argue that A/B testing is a great example of something nobody should be buying from a vendor or getting from unexamined library output, and it’s extremely easy to be losing a bunch of money on junk tests and not even understand your tests are junk. It only becomes much more true with OLS & logistic regression, decision trees, SVMs. You need customized systems operated by people who actually know statistics. Embedding this inside a DB where you cannot seriously develop software to customize the models & analysis just doesn’t work. The cost of serializing the data elsewhere is not the salient one.
ako 7y ago
Yes, almost every software or business presentation i see these days is repeating this sentiment: "Postgres is eating the world!" and "Every company is a Postgres company!"
- cryptonector 7y ago
  Is that a bad thing?

kid_atticus 7y ago

I'd say the real value of the 'application-like' capabilities of Postgres is for analytics and offline tasks where there may not even be an application to 'eat'. This article is a great example, anyone can get this working on their laptop easily and just feed new data into the structure they have created and trust that it will just work. Now, if this is powering some actual business logic then that's can cause problems since now your application is tightly coupled to a specific implementation in a specific database technology that may or may not be testable, versioned, or observable. But if you're a data hacker trying to answer questions, produce a report, or tease out features for a ML model having a such a robust easy to manage tool in your toolbelt is super useful. Application Postgres and Analytics Postgres are really different animals, it's pretty cool that one piece of software can do both so well and be so many different things to different people, but yeah I agree you can into real trouble if you use the wrong part of the tool for the job.

CuriouslyC 7y ago

If you have a problem that is never going to be google scale, and you have people who are familiar with postgres, you can deliver applications that are a fraction of the code with superior performance. The only problem with db-centric applications is the tooling, which is inferior to standard application tooling at this point, in terms of version control and testability. Given the hype train building around postgres as a platform, I expect this is going to change pretty quickly.

mlthoughts2018 7y ago
Disagree. For example, putting machine learning in the database is miserable. The same is true for other X besides ML. Does not matter what the scale is.
- lootsauce 7y ago
  We have found in database ML to be great to productionalize basic machine-learning methods. We use BigQuery ML and it saved us a ton of fussing with various frameworks, and because we were already in BQ it saved us the moving of data from database to ML cluster and back into database. We simply author some SQL and get great results quickly.
  - mlthoughts2018 7y ago
    That strikes me as bad usage of ML algorithms for quite a few reasons. People who say this kind of thing usually are not ML practitioners and want to drink some Kool Aid that they can save money & avoid hiring them with techniques like this. How are you benchmarking the in-DB models against variations that can improve them? If they are basic regression models, how are you solving nuanced problems relating to coefficient estimates that cannot be solved with plug and chug libraries (such as from the article “Let’s Put the Garbage Can Regressions and Garbage Can Probits Where They Belong” by Achen)? I’d venture to guess you’re assuming one particular type of model just works, and either are unaware it is failing, or are unaware of wasted money left on the table by not improving or understanding it. This is extremely common especially with simpler use cases. For example, you might think A/B testing is easily solved as an abstract problem and you just need a framework. What could be hard? I’d argue that A/B testing is a great example of something nobody should be buying from a vendor or getting from unexamined library output, and it’s extremely easy to be losing a bunch of money on junk tests and not even understand your tests are junk. It only becomes much more true with OLS & logistic regression, decision trees, SVMs. You need customized systems operated by people who actually know statistics. Embedding this inside a DB where you cannot seriously develop software to customize the models & analysis just doesn’t work. The cost of serializing the data elsewhere is not the salient one.

ako 7y ago

Yes, almost every software or business presentation i see these days is repeating this sentiment: "Postgres is eating the world!" and "Every company is a Postgres company!"

cryptonector 7y ago
Is that a bad thing?