4 ms·

Facebook's "Open Compute" Server tested

47 points by ssclafani 15y ago

heyrhett 15y ago
A Facebook memcached server uses 384 GB of RAM. That's a lot of poke requests.
- alexgartrell 15y ago
  That the machines are capable of containing 384 GB of RAM does not imply that they contain 384 GB of RAM[1] :) [1] I work on the memcached team at Facebook
  - leej 15y ago
    Alex public memcached development at Facebook seems to stopped for the last ~2 years. Is it really the case or still improving but not making it public? Also can we expect management tools for memcached in the future from Facebook?
    - alexgartrell 15y ago
      Facebook's version of memcached has diverged significantly from the public version, to the point where pushing it upstream is at the very least unproductive and likely disruptive. It is our intention to eventually open source the entire stack, but as it's still very much in active development it's relatively low priority at the moment (note that this is the conservative viewpoint, people with significantly more seniority than myself want to see it released sooner rather than later, and no one above them disagrees).
serverascode 15y ago
This is timely b/c I was just looking at a quote today for a dl380 g7. :)
- wmf 15y ago
  Unfortunately it doesn't look so easy to order Open Compute servers.
misterbwong 15y ago
These motherboards have no BMC, very few USB (2) and NIC ports (2), one expansion slot, and are headless (no videochip).The only thing that an administrator can do remotely is "reboot over LAN". This is impressive. I assume that all their servers are imaged and provisioned via a system build script. That is no small task to get just right
- wmf 15y ago
  But when you have over 10,000 servers you have to get it right anyway; you can't afford to have people doing stuff by hand.
- devicenull 15y ago
  It's actually not as hard as you might think. It's a lot simpler for Facebook then it would be for most people, as they control their own hardware. The problem with automated builds comes in when you have lots of different hardware configs, and lots of different software configs. If you just have a few hardware configs, then your task is much, much easier.
WestCoastJustin 15y ago
While searching around I found this Synnex page (http://synnex.com/opencompute/index.html http://synnex.com/opencompute/index.html) where they give some info, picture gallery, and pricing info! Looks like you could actually order these puppies through them ;) Price List: http://synnex.com/opencompute/images/pricing.jpg http://synnex.com/opencompute/images/pricing.jpg Here is a quick summary: $266k + tax gets you 90 machines in three racks and the supporting power / networking infrastructure. 72 intel boxes @ 3.5TB RAM, 72TB storage, 576 cores 18 amd boxes @ 1.3TB RAM, 9TB storage, 144 cores Total: 90 boxes @ 4.7TB RAM, 81TB storage, 720 cores

47 points by ssclafani 15y ago

heyrhett 15y ago

A Facebook memcached server uses 384 GB of RAM. That's a lot of poke requests.

alexgartrell 15y ago
That the machines are capable of containing 384 GB of RAM does not imply that they contain 384 GB of RAM[1] :) [1] I work on the memcached team at Facebook
- leej 15y ago
  Alex public memcached development at Facebook seems to stopped for the last ~2 years. Is it really the case or still improving but not making it public? Also can we expect management tools for memcached in the future from Facebook?
  - alexgartrell 15y ago
    Facebook's version of memcached has diverged significantly from the public version, to the point where pushing it upstream is at the very least unproductive and likely disruptive. It is our intention to eventually open source the entire stack, but as it's still very much in active development it's relatively low priority at the moment (note that this is the conservative viewpoint, people with significantly more seniority than myself want to see it released sooner rather than later, and no one above them disagrees).

serverascode 15y ago

This is timely b/c I was just looking at a quote today for a dl380 g7. :)

wmf 15y ago
Unfortunately it doesn't look so easy to order Open Compute servers.

misterbwong 15y ago

These motherboards have no BMC, very few USB (2) and NIC ports (2), one expansion slot, and are headless (no videochip).The only thing that an administrator can do remotely is "reboot over LAN". This is impressive. I assume that all their servers are imaged and provisioned via a system build script. That is no small task to get just right

wmf 15y ago
But when you have over 10,000 servers you have to get it right anyway; you can't afford to have people doing stuff by hand.
devicenull 15y ago
It's actually not as hard as you might think. It's a lot simpler for Facebook then it would be for most people, as they control their own hardware. The problem with automated builds comes in when you have lots of different hardware configs, and lots of different software configs. If you just have a few hardware configs, then your task is much, much easier.

WestCoastJustin 15y ago

While searching around I found this Synnex page (http://synnex.com/opencompute/index.html http://synnex.com/opencompute/index.html) where they give some info, picture gallery, and pricing info! Looks like you could actually order these puppies through them ;) Price List: http://synnex.com/opencompute/images/pricing.jpg http://synnex.com/opencompute/images/pricing.jpg Here is a quick summary: $266k + tax gets you 90 machines in three racks and the supporting power / networking infrastructure. 72 intel boxes @ 3.5TB RAM, 72TB storage, 576 cores 18 amd boxes @ 1.3TB RAM, 9TB storage, 144 cores Total: 90 boxes @ 4.7TB RAM, 81TB storage, 720 cores