5 ms·

You aren't running this in the standard language/caching/db setup, then? You're storing EVERYTHING in memory? Isn't this kind of.. well, stupid, frankly.

mishmash 18y ago
While the comment above may not have the best manners, it does bring up an interesting point. I certainly wouldn't mind hearing more about HN's setup.
- paulgb 18y ago
  If you're really curious and have the time, you can download the source: http://arclanguage.org/install http://arclanguage.org/install
  - catch404 18y ago
    It'd be interesting to read a write up on the concepts used (such as the in memory database) and the server restart mechanism. It'd be hard to tell from the source for non Lisp programmers.
    - lacker 18y ago
      An in-memory database is pretty simple - it's just a bunch of hash tables. A server restart mechanism is also pretty simple. Write a cron job that activates every few seconds to ps aux and grep for the server name, and if it's not there start the server. That won't catch server hangs rather than deaths, but that's exactly the problem that happened here.
      - catch404 18y ago
        Thanks, I understood it uses hash tables - I've done the same myself, but never on such a large scale. This post sums it up: http://news.ycombinator.com/item?id=513259 http://news.ycombinator.com/item?id=513259
DomesticMouse 18y ago
Dude, frack no. Having everything in a database means you are tied to the speed of disk, instead of the speed of ram. The biggest issue with being purely ram resident is handling multi-threaded updates across your ram resident dataset. You could always use Erlang...
- ajkirwin 18y ago
  That's why you use intelligent caching.
  - sounddust 18y ago
    For a site this small, there's absolutely no reason the entire database shouldn't be cached in RAM. 32GB of RAM costs about $800. That buys you plenty of time to not have to worry about caching, and instead gives you more time to work on interesting features. For a single-person operation (or even a few people), you have to spend your time wisely.
    - jrockway 18y ago
      But of course, any sane database will cache as much in RAM as possible anyway. But instead of dying when you run out of memory, it just evicts the least recently used page.
- nikblack 18y ago
  "Dude, frack no. Having everything in a database means you are tied to the speed of disk, instead of the speed of ram. " database are not tied to a storage medium. there is no reason why you can't run a DB (key-val or even a full RDBMS) in RAM.
- jcl 18y ago
  If everything is purely RAM resident, wouldn't you lose it all when the server gets wedged?
  - sounddust 18y ago

mishmash 18y ago

While the comment above may not have the best manners, it does bring up an interesting point. I certainly wouldn't mind hearing more about HN's setup.

paulgb 18y ago
If you're really curious and have the time, you can download the source: http://arclanguage.org/install http://arclanguage.org/install
- catch404 18y ago
  It'd be interesting to read a write up on the concepts used (such as the in memory database) and the server restart mechanism. It'd be hard to tell from the source for non Lisp programmers.
  - lacker 18y ago
    An in-memory database is pretty simple - it's just a bunch of hash tables. A server restart mechanism is also pretty simple. Write a cron job that activates every few seconds to ps aux and grep for the server name, and if it's not there start the server. That won't catch server hangs rather than deaths, but that's exactly the problem that happened here.
    - catch404 18y ago
      Thanks, I understood it uses hash tables - I've done the same myself, but never on such a large scale. This post sums it up: http://news.ycombinator.com/item?id=513259 http://news.ycombinator.com/item?id=513259

DomesticMouse 18y ago

Dude, frack no. Having everything in a database means you are tied to the speed of disk, instead of the speed of ram. The biggest issue with being purely ram resident is handling multi-threaded updates across your ram resident dataset. You could always use Erlang...

ajkirwin 18y ago
That's why you use intelligent caching.
- sounddust 18y ago
  For a site this small, there's absolutely no reason the entire database shouldn't be cached in RAM. 32GB of RAM costs about $800. That buys you plenty of time to not have to worry about caching, and instead gives you more time to work on interesting features. For a single-person operation (or even a few people), you have to spend your time wisely.
  - jrockway 18y ago
    But of course, any sane database will cache as much in RAM as possible anyway. But instead of dying when you run out of memory, it just evicts the least recently used page.
nikblack 18y ago
"Dude, frack no. Having everything in a database means you are tied to the speed of disk, instead of the speed of ram. " database are not tied to a storage medium. there is no reason why you can't run a DB (key-val or even a full RDBMS) in RAM.
jcl 18y ago
If everything is purely RAM resident, wouldn't you lose it all when the server gets wedged?
- sounddust 18y ago