3 ms·

My fun postgres story is the time I spent a week learning and using CouchDB to do some operations across five to eight million tweets. I mean, surely a relatio

My fun postgres story is the time I spent a week learning and using CouchDB to do some operations across five to eight million tweets. I mean, surely a relational db is ill-suited to something like that, right? After no small amount of frustration, I just threw a cleaned-up version of each tweet into postgres and did a `LIKE` query across all of them. It returned in, iirc, less than a second (vs nearly a minute for couch). The lesson, of course, is not that couch is bad and postgres is great, but rather as the OP said: "Human intuition about what might be slow is almost certainly wrong."

astine 11y ago
I once built a location aware application on MongoDB. After months of Mongo being the bottleneck in the application and periodically crashing, I switched to a naive implementation of the application with PostgreSQL and the application sped up five-fold. I've learned my lesson on Mongo. Postgres, however, has always been pretty awesome.
- DrJosiah 11y ago
  Postgres with its Geo extensions are also pretty solid.
theandrewbailey 11y ago
> Human intuition about what might be slow is almost certainly wrong. He got that right. Try it for yourself: https://computers-are-fast.github.io/ https://computers-are-fast.github.io/
gdulli 11y ago
Not only was that a good outcome because 5m-8m isn't "big", there's even a more performant way to do real text search within Postgres that would let you scale far beyond that number before outgrowing it.
- Johnny555 11y ago
  Whether or not full text search is more scalable than a full table scan depends on the use case - if he's continually inserting and deleting rows from his table while doing relatively few queries, it may very well be more scalable to do the full table scans to avoid the continual overhead of maintaining the full-text index on inserts and deletes.
  - gdulli 11y ago
    Good point.
coldtea 11y ago
>I mean, surely a relational db is ill-suited to something like that, right? Why would one assume that? Relational dbs have been optimized for 30+ years to be as fast as possible for exactly that, and with hundreds of millions of rows even -- 8 million rows is nothing. If you don't need complex JOINs and have indexes, they can do all that faster than the NoSQL flavor of the day (and even with JOINs they tend to be on par).
- fishtoaster 11y ago
  Yes, that was the point. It was the incorrect assumption of a younger me. Lesson learned. :)

astine 11y ago

I once built a location aware application on MongoDB. After months of Mongo being the bottleneck in the application and periodically crashing, I switched to a naive implementation of the application with PostgreSQL and the application sped up five-fold. I've learned my lesson on Mongo. Postgres, however, has always been pretty awesome.

DrJosiah 11y ago
Postgres with its Geo extensions are also pretty solid.

theandrewbailey 11y ago

> Human intuition about what might be slow is almost certainly wrong. He got that right. Try it for yourself: https://computers-are-fast.github.io/ https://computers-are-fast.github.io/

gdulli 11y ago

Not only was that a good outcome because 5m-8m isn't "big", there's even a more performant way to do real text search within Postgres that would let you scale far beyond that number before outgrowing it.

Johnny555 11y ago
Whether or not full text search is more scalable than a full table scan depends on the use case - if he's continually inserting and deleting rows from his table while doing relatively few queries, it may very well be more scalable to do the full table scans to avoid the continual overhead of maintaining the full-text index on inserts and deletes.
- gdulli 11y ago
  Good point.

coldtea 11y ago

>I mean, surely a relational db is ill-suited to something like that, right? Why would one assume that? Relational dbs have been optimized for 30+ years to be as fast as possible for exactly that, and with hundreds of millions of rows even -- 8 million rows is nothing. If you don't need complex JOINs and have indexes, they can do all that faster than the NoSQL flavor of the day (and even with JOINs they tend to be on par).

fishtoaster 11y ago
Yes, that was the point. It was the incorrect assumption of a younger me. Lesson learned. :)