3 ms·
> Can anyone confirm if Postgres would die like MySQL did on too big a dataset --- selecting everything without paginating but fetching just one row at a time i
by anarazel 9y ago
> Can anyone confirm if Postgres would die like MySQL did on too big a dataset --- selecting everything without paginating but fetching just one row at a time in the application?
Depends on the client library. A lot of clientside SQL interfaces load all rows into arrays of hashes or such into memory.
- TheAceOfHearts 9y agoI'd seriously expect anyone working with such large datasets to be working with lower level libraries when interfacing with the database. Stuff like ActiveRecord is fine for simple operations and small datasets, but surely at the scale of hundreds of millions of records you can afford to hire an engineer capable of writing SQL and understanding the underlying database operations? Am I just being naïve?
- barrkel 9y agoIn my experience, if you're working with moderately large data sets and want to provide interesting UI with rich filtering operations, you'll want a lot more control over execution plans than straightforward declarative SQL gives you. You end up forcing the database's hand by nesting subtable queries; and since ORM tools don't generate queries in this style, you write your own SQL generation library.
- userpass 9y agoThat very same engineer capable of using ActiveRecord should also be capable of writing SQL.