5 ms·

Brawny cores still beat wimpy cores, most of the time [pdf]

53 points by gruseom 15y ago

zdw 15y ago
I wonder what the massively parallel CPU vendors like the Opteron 6xxx series or Sparc T-series would have to say about this. Also, overall power usage isn't given much consideration, just response time.
- rhizome 15y ago
  I'd say they'd at least be able to rely on the assertion from this paper that I/O beats CPU. Fascinating and useful read, thanks for posting.
- codedivine 15y ago
  I would say a core in Opteron 6xxx series at least still fits the defintiion of a brawny core. Its more about Atom/current ARM cores type of wimpy cores that we are talking about.
gwern 15y ago
> Third, smaller servers can also lead to lower utilization. Consider the task of allocating a set of applications across a pool of servers as a bin-packing problem—each of the servers is a bin, and we try to fit as many applications as possible into each bin. Clearly that task is harder when the bins are small, because many applications might not completely fill a server and yet use too much of its CPU or RAM to allow a second application to coexist on the same server. Thus, larger bins (combined with resource containers or virtual machines to achieve performance isolation between individual applications) might offer a lower total cost to run a given workload. Hey, that's a pretty good analogy! 'When you have lots of small processors, it's like packing for a trip with a lot of small bags - where do you fit your skis?'
justincormack 15y ago
I think the space of applications that could run on wimpy machines is a reasonable size, although of course not all. Google though do have a huge installed base of software so the cost of moving could be very high. The examples are based on stuff that is at the limit for sequential execution speeds, but loads of software is not, has scope to run slower.
pjscott 15y ago
The meta-problem here is automatically determining how to allocate tasks among different types of processors in a large, heterogeneous data center. There are workloads for which wimpy cores have a serious advantage, and ones where they don't fare as well. Also, assigning jobs to cores that aren't perfectly suited for them may still be a good idea if you need to meet load spikes. The profiling and scheduling could be tricky, but for a company the size of Google, it could save a lot of power and money.
- miratrix 15y ago
  I think you may have missed one of the central point that Urs was trying to make: "Software development costs often dominate a company’s overall technical expenses, so forcing programmers to parallelize more code can cost more than we’d save on the hardware side." There could be savings to be had by having a very smart scheduler that knows all about what kind of workload a given piece of task is by analyzing the code and dispatching it correctly. As you mentioned, though, that's not an easy problem to solve, and will also be a part of the software development cost. From the programmer's point of view, that'll be yet another variable out of their control that may break the computing abstraction that makes these massive parallel machines possible.
  - pjscott 15y ago
    That's part of the big win of having job abstraction systems, like MapReduce: having the jobs wrapped in higher-level abstractions that make automated profiling and fancy scheduling feasible. It doesn't even have to be anything that fancy. It could be along the lines of "run the job a hundred times on each type of machine, and measure throughput, latency, and energy usage; then try to allocate jobs to the best kind of processor for it." That shouldn't break anything.
    - gruseom 15y ago
      How do you tell what kind of job you have without running it a hundred times? Or do you mean run small portions of a large job before deciding where to commit the rest?
ldar15 15y ago
Takeaway: parallel programming is hard even for Googlers.
- kennystone 15y ago
  Google is a lot C++, Java, and Python, so indeed it is. Use Erlang (or similar) and parallel is easy.
  - lukesandberg 15y ago
    I don't know a lot about erlang but i doubt it makes parallelism easy all of the time. I'm sure there are many cases when the default actor model works great, but that's not a reasonable model for _every_ problem. Shared memory multithreading (for all of its faults) can be very effective for certain kinds of problems. So it can be worth the difficulty.
  - rbanffy 15y ago
    Writing parallel code is easy only when compared to writing Erlang (or similar) code ;-)
spullara 15y ago
One thing you can do to combat the issue is use a templating framework that is multithreaded like mine: http://github.com/spullara/mustache.java http://github.com/spullara/mustache.java It spreads all the requests work over multiple cores so single request latency doesn't suffer when you have a lot of cores.
- kevingadd 15y ago
  How do you address the scheduling and synchronization issues described in the paper to keep request latencies low?

53 points by gruseom 15y ago

zdw 15y ago

I wonder what the massively parallel CPU vendors like the Opteron 6xxx series or Sparc T-series would have to say about this. Also, overall power usage isn't given much consideration, just response time.

rhizome 15y ago
I'd say they'd at least be able to rely on the assertion from this paper that I/O beats CPU. Fascinating and useful read, thanks for posting.
codedivine 15y ago
I would say a core in Opteron 6xxx series at least still fits the defintiion of a brawny core. Its more about Atom/current ARM cores type of wimpy cores that we are talking about.

gwern 15y ago

> Third, smaller servers can also lead to lower utilization. Consider the task of allocating a set of applications across a pool of servers as a bin-packing problem—each of the servers is a bin, and we try to fit as many applications as possible into each bin. Clearly that task is harder when the bins are small, because many applications might not completely fill a server and yet use too much of its CPU or RAM to allow a second application to coexist on the same server. Thus, larger bins (combined with resource containers or virtual machines to achieve performance isolation between individual applications) might offer a lower total cost to run a given workload. Hey, that's a pretty good analogy! 'When you have lots of small processors, it's like packing for a trip with a lot of small bags - where do you fit your skis?'

justincormack 15y ago

I think the space of applications that could run on wimpy machines is a reasonable size, although of course not all. Google though do have a huge installed base of software so the cost of moving could be very high. The examples are based on stuff that is at the limit for sequential execution speeds, but loads of software is not, has scope to run slower.

pjscott 15y ago

The meta-problem here is automatically determining how to allocate tasks among different types of processors in a large, heterogeneous data center. There are workloads for which wimpy cores have a serious advantage, and ones where they don't fare as well. Also, assigning jobs to cores that aren't perfectly suited for them may still be a good idea if you need to meet load spikes. The profiling and scheduling could be tricky, but for a company the size of Google, it could save a lot of power and money.

miratrix 15y ago
I think you may have missed one of the central point that Urs was trying to make: "Software development costs often dominate a company’s overall technical expenses, so forcing programmers to parallelize more code can cost more than we’d save on the hardware side." There could be savings to be had by having a very smart scheduler that knows all about what kind of workload a given piece of task is by analyzing the code and dispatching it correctly. As you mentioned, though, that's not an easy problem to solve, and will also be a part of the software development cost. From the programmer's point of view, that'll be yet another variable out of their control that may break the computing abstraction that makes these massive parallel machines possible.
- pjscott 15y ago
  That's part of the big win of having job abstraction systems, like MapReduce: having the jobs wrapped in higher-level abstractions that make automated profiling and fancy scheduling feasible. It doesn't even have to be anything that fancy. It could be along the lines of "run the job a hundred times on each type of machine, and measure throughput, latency, and energy usage; then try to allocate jobs to the best kind of processor for it." That shouldn't break anything.
  - gruseom 15y ago
    How do you tell what kind of job you have without running it a hundred times? Or do you mean run small portions of a large job before deciding where to commit the rest?

ldar15 15y ago

Takeaway: parallel programming is hard even for Googlers.

kennystone 15y ago
Google is a lot C++, Java, and Python, so indeed it is. Use Erlang (or similar) and parallel is easy.
- lukesandberg 15y ago
  I don't know a lot about erlang but i doubt it makes parallelism easy all of the time. I'm sure there are many cases when the default actor model works great, but that's not a reasonable model for _every_ problem. Shared memory multithreading (for all of its faults) can be very effective for certain kinds of problems. So it can be worth the difficulty.
- rbanffy 15y ago
  Writing parallel code is easy only when compared to writing Erlang (or similar) code ;-)

spullara 15y ago

One thing you can do to combat the issue is use a templating framework that is multithreaded like mine: http://github.com/spullara/mustache.java http://github.com/spullara/mustache.java It spreads all the requests work over multiple cores so single request latency doesn't suffer when you have a lot of cores.

kevingadd 15y ago
How do you address the scheduling and synchronization issues described in the paper to keep request latencies low?