4 ms·

Something is very wrong for the median read of what should be a moderate sized blob with an indexed lookup to be 8ms. The article doesn’t give enough details t

Something is very wrong for the median read of what should be a moderate sized blob with an indexed lookup to be 8ms. The article doesn’t give enough details to really understand what is that is happening here, but I would recommend everyone to take this 14/16x report and not expect that to be the general case.

mslot 7y ago
Looking up a row of JSONB data by a primary key in a 3GB PostgreSQL table on my laptop takes 0.4-0.5ms, since it easily fits in memory and Postgres caches the data.
- bufferoverflow 7y ago
  If it fits in RAM, you might as well put it in memcached. You will likely get a 10X speedup. EDIT: To be fair, RAM is cheap these days, and most real world projects fit into RAM without a crazy budget.
  - jjeaff 7y ago
    Ram is cheap everywhere but "cloud" instances. Was just noticing the other day, I can get about 128GB of ram on a dedicated instance for the same price I am paying in the cloud for 16GB.
    - bufferoverflow 7y ago
      Yeah, cloud prices are insane. I only rent individual servers. Very few real world projects actually do need the cloud features. It's nice to have, sure, but you're paying a lot for it.
  - mslot 7y ago
    Why even memcached? It's not going to be 10x faster than your typical LRU cache (e.g. Postgres' shared buffers). It might scale a little better, but you should only consider that when it's worth it.
zbentley 7y ago
I think the author suggested that the cause of some of the latency might be the processing in Python as data goes through the ORM. I like the Django ORM but it does not prioritize speed. It's possible that Python time plus the ORM doing something stupid (perhaps multiple round trips to the DB, reconnects, or weird instrumentation in the DB router or manager) could account for those extra milliseconds.
- tyingq 7y ago
  It's also in a json type in Postgres rather than a blob or varchar, while it's just raw data in redis.

mslot 7y ago

Looking up a row of JSONB data by a primary key in a 3GB PostgreSQL table on my laptop takes 0.4-0.5ms, since it easily fits in memory and Postgres caches the data.

bufferoverflow 7y ago
If it fits in RAM, you might as well put it in memcached. You will likely get a 10X speedup. EDIT: To be fair, RAM is cheap these days, and most real world projects fit into RAM without a crazy budget.
- jjeaff 7y ago
  Ram is cheap everywhere but "cloud" instances. Was just noticing the other day, I can get about 128GB of ram on a dedicated instance for the same price I am paying in the cloud for 16GB.
  - bufferoverflow 7y ago
    Yeah, cloud prices are insane. I only rent individual servers. Very few real world projects actually do need the cloud features. It's nice to have, sure, but you're paying a lot for it.
- mslot 7y ago
  Why even memcached? It's not going to be 10x faster than your typical LRU cache (e.g. Postgres' shared buffers). It might scale a little better, but you should only consider that when it's worth it.

zbentley 7y ago

I think the author suggested that the cause of some of the latency might be the processing in Python as data goes through the ORM. I like the Django ORM but it does not prioritize speed. It's possible that Python time plus the ORM doing something stupid (perhaps multiple round trips to the DB, reconnects, or weird instrumentation in the DB router or manager) could account for those extra milliseconds.

tyingq 7y ago
It's also in a json type in Postgres rather than a blob or varchar, while it's just raw data in redis.