4 ms·
First, relax. Look, you've got ~4 GHz processors available, and you're worried about a million hits a minute. That's nearly 240k clock cycles per hit. Next,
by lallysingh 18y ago
First, relax.
Look, you've got ~4 GHz processors available, and you're worried about a million hits a minute. That's nearly 240k clock cycles per hit.
Next, you've got 60 +/- 10ms of latency between you and your clients (unless there are details here you aren't covering, but no matter). You don't really have to respond to users terribly fast, as lots of your responsiveness would be lost over jitter/lag.
A single dell box could handle your load, if you wrote the whole thing in C on a modern quad-core desktop.
Ignore all the standard Apache/SQL/networking stuff until later until you know what you need. Get the core requirements up first, then add on the load of the side-stuff secondly.
E.g. doing all the heavy lifting in a private set of cpu cores and then providing a shared-mem plugin to apache may be enough for you. Save a CPU core to snapshot your data.
So, for advice:
1. Ignore database bullshit. You don't need it, it won't help. If you want a DB for other purposes, fine. A snapshot process writing to your DB is fine, just don't put it in the critical path.
2. Build a load simulator. A raw mode that just sends over the handful of bytes, and a cooked mode that bothers to printf' a GET request.
3. Start with a reasonable prototype, and work your way to something performant. Hell, you can probably do it in java if you don't mind buying a few more CPU cores.
4. Integrate as you need for the rest of your requirements. For example, have another box serve the rest of your webapp, and dedicate a stripped down apache box with a custom module for this stuff.
In essence, I'm telling you to treat it as a very smallish HPC problem, instead of some sort of nightmare webapp problem. It fits better, and suddenly you have lots of people/knowledge/COTS equipment available to you.
- lallysingh 18y agoAlso, if you're doing something game-ish, talk to me privately. Scalability of video games is my phd topic...
- mleonhard 18y agoThere's no need to build your own load simulator when good ones already exist: http://tsung.erlang-projects.org/ http://tsung.erlang-projects.org/
- owkaye 18y agoThanks mleonhard, I've created a new thread to continue this discussion since this one has become way too disorganized, so please post future replies here: http://news.ycombinator.com/item?id=362810 http://news.ycombinator.com/item?id=362810