3 ms·

In 2011 a company i contracted for was testing some new dell 1U servers with around 1-2TB of ram. There was a postgres database with 4000qps that could fit into

In 2011 a company i contracted for was testing some new dell 1U servers with around 1-2TB of ram. There was a postgres database with 4000qps that could fit into tmpfs, and so i restricted postgres to 640Kb of memory and we got replication working, it took about 6 hours of babysitting. We threw the switch and watched as postgres, with 640Kb of ram and a tmpfs backed store proceeded to handle all of the query traffic. There were some stored procedures or something that were long-querying or whatever - i'm not a DB person at all, so we switched back to the regular production server about 8 minutes later. Yes, we did it in production.

bagels 5y ago
For real, 640 kilobits?
- sigstoat 5y ago
  K isn't the abbreviation for kilo, so if you're going to rag on the fellow for the 'b', then you should at least be asking what a Kelvin*bit is.
  - iandinwoodie 5y ago
    “The binary meaning of the kilobyte for 1024 bytes typically uses the symbol KB, with an uppercase letter K.” [0] 0. https://en.m.wikipedia.org/wiki/Kilobyte https://en.m.wikipedia.org/wiki/Kilobyte
  - jmiserez 5y ago
    640KiB is very little and I'm wondering if it's a typo, given that the servers had 1-2TiB available. Postgres 9.0 released in 2010 already had 32MiB as the default for shared_buffers (with a minimum of 128KiB): https://www.postgresql.org/docs/9.0/runtime-config-resource.html https://www.postgresql.org/docs/9.0/runtime-config-resource.... and 8.1 released in 2005 used 8MB (1000*8KiB): https://www.postgresql.org/docs/8.1/runtime-config-resource.html https://www.postgresql.org/docs/8.1/runtime-config-resource....
    - sigstoat 5y ago
      i interpreted it as "we wanted to turn the shared buffers ~off, but in a hilarious way that would suggest to someone reading the configuration file that something was going on" (bill gates, mumble mumble) but, wtf do i know, i'm the crazy guy who tries to interpret comments generously.
      - genewitch 5y ago
        Yes, it was a direct reference to Bill Gates "640 kilobytes is enough for anyone" and i typed the comment right before i fell asleep.
  - bagels 5y ago
    The question was more about the kilo part, even though I didn't clarify. Seems orders of magnitude too small?
winrid 5y ago
Postgres handles low memory situations well. It'll kill memory intensive queries before it crashes. I wonder if your application was getting a lot of errors back instead of successful queries :)
- genewitch 5y ago
  the application performed fine, even though we made the switch around 15:00 PST. The DBA was concerned because of the few long queries. Obviously the tmpfs was doing the heavy lifting, there - and if i had to do a postmortem, i'd wager that filling the OS caches was the main reason the long queries took so long. We didn't do any sort of performance tracing. The main purpose was to show that these $35k servers could essentially replace the older machines if need be, even though the old ones had FusionIO. I just removed the middleman of the PCIe bus between the application and the memory. It was a near constant argument on the floor about whether or not we could feasibly switch to SSDs in some configuration over spinning rust or even FusionIO, i wanted a third option. Basically, serve out of registered, ECC memory in front, replicate to the fusionIO and let those handle the spindled backups, which iirc was a pain point.

bagels 5y ago

For real, 640 kilobits?

sigstoat 5y ago
K isn't the abbreviation for kilo, so if you're going to rag on the fellow for the 'b', then you should at least be asking what a Kelvin*bit is.
- iandinwoodie 5y ago
  “The binary meaning of the kilobyte for 1024 bytes typically uses the symbol KB, with an uppercase letter K.” [0] 0. https://en.m.wikipedia.org/wiki/Kilobyte https://en.m.wikipedia.org/wiki/Kilobyte
- jmiserez 5y ago
  640KiB is very little and I'm wondering if it's a typo, given that the servers had 1-2TiB available. Postgres 9.0 released in 2010 already had 32MiB as the default for shared_buffers (with a minimum of 128KiB): https://www.postgresql.org/docs/9.0/runtime-config-resource.html https://www.postgresql.org/docs/9.0/runtime-config-resource.... and 8.1 released in 2005 used 8MB (1000*8KiB): https://www.postgresql.org/docs/8.1/runtime-config-resource.html https://www.postgresql.org/docs/8.1/runtime-config-resource....
  - sigstoat 5y ago
    i interpreted it as "we wanted to turn the shared buffers ~off, but in a hilarious way that would suggest to someone reading the configuration file that something was going on" (bill gates, mumble mumble) but, wtf do i know, i'm the crazy guy who tries to interpret comments generously.
    - genewitch 5y ago
      Yes, it was a direct reference to Bill Gates "640 kilobytes is enough for anyone" and i typed the comment right before i fell asleep.
- bagels 5y ago
  The question was more about the kilo part, even though I didn't clarify. Seems orders of magnitude too small?

winrid 5y ago

Postgres handles low memory situations well. It'll kill memory intensive queries before it crashes. I wonder if your application was getting a lot of errors back instead of successful queries :)

genewitch 5y ago
the application performed fine, even though we made the switch around 15:00 PST. The DBA was concerned because of the few long queries. Obviously the tmpfs was doing the heavy lifting, there - and if i had to do a postmortem, i'd wager that filling the OS caches was the main reason the long queries took so long. We didn't do any sort of performance tracing. The main purpose was to show that these $35k servers could essentially replace the older machines if need be, even though the old ones had FusionIO. I just removed the middleman of the PCIe bus between the application and the memory. It was a near constant argument on the floor about whether or not we could feasibly switch to SSDs in some configuration over spinning rust or even FusionIO, i wanted a third option. Basically, serve out of registered, ECC memory in front, replicate to the fusionIO and let those handle the spindled backups, which iirc was a pain point.