9 ms·

I serve many, many millions of rows, collect real-time statistics, push real-time updates, and maintain data integrity and consistency... all from a single data

• pg_is_a_butt 10y ago
  because SELECT statements don't spawn UDP connections and AJAX service workers, because that is what a dataBASE is supposed to do. you're all idiots.
• dpark 10y ago
  The argument is not that DBs failed the web for scalability, but that they failed the web for features. I don't fully agree but it is an interesting argument. DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. DBs are great at storing and maintaining relations and relationships but much of the other functionality is little-used.
  • WrtCdEvrydy 10y ago
    Maybe if there was better support at the connection layer, this might be better. If ActiveRecord (Rails) allowed you to pass a free-form ruby function and built a procedure and executed it in SQL, I'd be all for it, but I'm not writing a one-off procedure and mantaining it.
    • tomc1985 10y ago
      What is there to maintain? It's not hard to directly emit SQL to ActiveRecord
      • WrtCdEvrydy 10y ago
        I'm not writing a one-off procedure in RAW SQL. If ActiveRecord could turn functions into SQL equivalents, it would be awesome.
        • tomc1985 10y ago
          I know, they're so scary aren't they?
          • WrtCdEvrydy 10y ago
            No, in terms of Return on Investment, it just isn't worth it for me. Noone cares if a large query takes 10 seconds or 11 seconds with ActiveRecord enough to blow developer time on it.
            • throwanem 10y ago
              No one cares until they do, because the dataset's grown nontrivial and the crap 10s query Active Record generates is now a crap 120s query and one of the server, the client, or the user times out before it finishes.
  • vog 10y ago
    The PostgrREST project is an interesting approach so fill this gap: https://postgrest.com/ https://postgrest.com/
    • deleted 10y ago
      [deleted]
  • zitterbewegung 10y ago
    It sounds like you would like to look at something like postgrest. See https://github.com/begriffs/postgrest https://github.com/begriffs/postgrest
  • deleted 10y ago
    [deleted]
  • daliwali 10y ago
    Many DBs weren't built to be exposed on the open web. The web didn't even exist at the time some popular databases were invented, and even after the fact, it's still not the purpose. I would even argue that DBs have overstepped their bounds already with what programming abilities some have built-in. They should be focused on reading and writing structured data efficiently and correctly, not much more than that. Which database to use should depend on access patterns and structural requirements. Let the application logic be handled at a higher layer instead of mixing responsibilities everywhere.
    • geon 10y ago
      > DBs have overstepped their bounds And they have a stringly typed interface! It sucks! When both the languages inside and outside the database are typed, why should the interface between them not be? It is just unfathomably bad.
      • majewsky 10y ago
        Uh, what? How is this stringly-typed: https://www.postgresql.org/docs/9.4/static/datatype.html https://www.postgresql.org/docs/9.4/static/datatype.html ? Postgres's type system is actually more sophisticated than any programming language that I know. The problem is that ORM developers are much more serious about interoperability than can be reasonably argued, so you cannot rely on much more than VARCHAR, INTEGER and maybe BOOLEAN.
        • dpark 10y ago
          Query input to SQL is literally a string. Output is also a string. Thus "stringly typed" interface. Yes, DB storage is strongly typed, but the interface is not. I'm not sure how much an issue this really is, but it's what the GP is talking about.
          • majewsky 10y ago
            So? Data in computers is all 0s an 1s, but you don't see anyone complaining about bitly-typed languages. Strong typing refers to the presence of type context wherever a value is used, not to the wire protocol.
  • okket 10y ago
    > DBs have auth systems already, but we mostly bypass those and reimplement auth in the stateless layer. This is sane and good design. The db auth layer is for db management, not frontend auth (are you seriously advocating this?). > DBs have programming capabilities but we mostly bypass that and build business logic in the stateless layer. This is just (initial) design / programmer laziness. I bet if you start to scale up, you will think a lot about moving your business logic further to the real data for safety and for performance reasons. This is the point when you realise that db systems are very different and switching from one to another is not that easy...
    • fragmede 10y ago
      > The db auth layer is for db management, not frontend auth Yes, but why? The database already has a table of database users, and the different permissions they're allowed. (Well, hopefully your webapp doesn't just do everything as the root DB user with * privs.) Is there that much to be gained from the database having a second table of users that you've created, and then you get to reimplement a bunch of logic in your app server on top of the database so that you have... users and permissions?
      • moron4hire 10y ago
        Because traditionally the database was managed by IT, not by engineering. IT came up with the idea that it should take forever to approve changes to the database, so engineering invented ways around their can-don't attitude. Can you imagine any CTO approving a web-based signup page being allowed to change the users to the database?!
        • mixmastamyk 10y ago
          If it only adds restricted users is that a big problem?
          • moron4hire 10y ago
            The problem is not an issue of technology.
• fragmede 10y ago
  The articles point is that there's a bit more to Stack Overflow than the four database servers it sits on, and because MS SQL is treated as a monolith,a stupid amount of development time, the world over, goes into doing things the database, arguably, could be configured to do itself. I click the accept answer button on my web browser, stack overflow stack reads the cookie my browser presented, does a bunch of logic in the app server, then does a database lookup. It then takes the web request, does a bunch of other logic with it in the app server again, then... runs some more SQL against the database. Stack Overflow, and I, and presumably you as well, have written tons of custom code that sits between the web browser and database, but the article's thesis is this is stupid because if the database "just" spoke http, then it could do the majority of that natively, no Ruby/Python/JavaScript/.Net or anything needed.
  • deleted 10y ago
    [deleted]
  • laumars 10y ago
    > the article's thesis is this is stupid because if the database "just" spoke http, then it could do the majority of that natively, no Ruby/Python/JavaScript/.Net or anything needed. The issue then becomes scaling. Instead of having half a dozen database nodes you need to keep in sync you could now dozens or hundreds. Possibly even thousands on some applications. The point of databases is they're meant to be a monolithic repository of persistent data. The point of web servers is that they're supposed to "dumb" and interchangeable. You meant to be able to spin up new instances to scale to demand. This gets greatly more complex if your application is also your database. It's also not true that databases haven't evolved for the web. We now have NoSQL databases, JSON support in many RDBMS and in-memory databases for caching. Running your web application inside your database is already possibly. There have been posted on HN discussing people doing this. But it's really not practical for much beyond a small scale blog or private (IP whitelisted) API server. ie the kind of thing that the application be hosted on a single node instance and potentially with the DB sharing the same VM too. The moment you need to scale sideways, even just to a small few nodes, it becomes more advantageous to have your database separated from your application servers.
• ruslan_talpa 10y ago
  Exactly! How about "developers have failed the databases" The majority of the developers don't use (or don't know how to) views/stored procedures/roles/triggers. Whenever you mention something like this you hear "omg, you put business logic in the db? everybody knows that does not scale" or "we deconstruct our request into a few simple queries instead of a join so we run super optimal"
  • sk5t 10y ago
    > deconstruct our request into a few simple queries Makes sense if you can cache only portions of the result, and/or the query optimizer or database layout actually struggle with the join.
  • cr0sh 10y ago
    I agree that more business logic needs to be "closer" to the DB - if not in the DB itself (using those facilities you mentioned), then at a level very nearby. There's a very good reason for this: Showing a consistent interface for data update and query. When there are multiple teams developing against the same DB server, they might be able to see most or all of the tables, but they each might implement updates/selects (SQL queries) that exhibit extremely different results - which either present or update the data in potentially incompatible or inconsistent ways. This ultimately manifests itself as errors/bugs which can be very difficult to figure out what went wrong where and why (and correcting the data becomes a nightmare). Things get even worse when you have people querying the data via ad-hoc methods (like ODBC to an Excel file or something weird like that). Essentially, it is possible (likely) that every team sees a slightly (or greatly) different view from each other. Coupling the BL closer to the DB level would prevent this; whether via the tools mentioned (views, triggers, etc) or something else. Ultimately, app developers shouldn't be hitting the DB tables directly, but going through these interfaces. Of course - that brings up a host of problems on its own (table updates become an issue of coordination and conflict between teams, etc) - and there's also the issue of implementing versioning of the business logic (and how that relates to current/past DB structure)... ...but implementing it would solve a lot of headaches (and create more I guess). /sigh
  • agentultra 10y ago