5 ms·

The spark workflow is 10% cheaper (24.36 vs 21.88) over 3 years. Is that really worth having to support a non-mainstream architecture? I doubt it.

by mtanski 11y ago

RankingMember 11y ago
That was my thought too. The "pain in the ass" factor associated with supporting a non-mainstream architecture is missing from the charts, and moves the dial away from IBM. That's not to say things couldn't change, but Xeons are proven, and unless you've got a surplus of money to throw at trying a new architecture (e.g. Google), you're better off sticking with the known good unless the predicted gains are sizable.
vessenes 11y ago
I just did this math for our company a couple months ago. Power8 memory bandwidth is VERY appealing. And, it's not always just about cost per benchmark unit -- if you have some realtime requirements for your analysis tools, then scale-up speed can be really valuable as compared to dev time and management time. In the end, we made the call for Intel because golang runs some key parts of our tools, and the golang power8 story isn't there. But, as I gaze at our servers where we paid what feels like thousands for extra megabytes of L3 cache, I wouldn't say I'm happy about the decision. A good go story from IBM would have likely tipped things the other way.
- mtanski 11y ago
  I think you just kind of proved my point that real work load benchmarks would have to be much more attractive to offset the cost of supporting an architecture that's not tier one in many languages / software libraries / projects.
  - vessenes 11y ago
    Totally. That said, IBM isn't short on optimizing compiler folks. Open ecosystem support is a strategy to pick up small and mid-size buyers; it will be interesting to see if IBM gets there. I would look again next time we source hardware. And, if IBM put someone internally on a properly vectorized go compilation pathway for Power8, I would buy in a heartbeat, provided it ran some sort of debian variant.
    - mtanski 11y ago
      I don't think that go as people tend to use it really benefits from vectorized code. Most people who are using go that I interact with are not writing numerical processing code but network servers and business logic for high level web APIs. You might get minor speed ups in vectorized memcpy but I can't see much else. I imagine the most important CPU features for most go code would be a good branch predictor and fast atomics / synchronization primitives. If you're using go for numerical processing code I'd like to hear more about it. Mostly because it's kind of a PITA.
      - vessenes 11y ago
        Well, there are almost no real vectorizable primitives or functions in the core library so I'm not surprised that you don't run into people vectorizing much. And, go dev team compiler focus has been elsewhere the last year. And, so far the go team hasn't seemed to be able to interest Intel in doing the heavy lifting that they might do for some other compilers.. So, branch predictions and faster sync primitives would be great, not least because they would speed up channels in many cases, which would be cool; it would be nice to widen the use cases for channel-based communication significantly, but they're just VERY slow if you want to use them at scale in a large application. I am using go for some large scale numerical processing, although it's the sort with lots of logic attached, not just a giant matrix with some glue around it. It's kind of a PITA. We are picking and choosing some outside libraries, and spend a lot of time massaging the go code for speed and bitching about the garbage collector. (Did you know that for i, _ := range is often 3 to 4x faster than for _, v := range? Do you know how awful code written down four or five nested loops that uses indices looks?) But, the size codebase our team can manage with go is pretty great. We wouldn't be nearly so productive in many other cool (or .. experienced) languages when you add up the full life cycle costs including innovation, enhancement, bug fixes, maintenance and deployment. It's a win. I'd do it again in a heartbeat.
        mtanski 11y ago
        > So, branch predictions and faster sync primitives would be great, not least because they would speed up channels in many cases, which would be cool; it would be nice to widen the use cases for channel-based communication significantly, but they're just VERY slow if you want to use them at scale in a large application. These operations are already pretty good on IA* processors, at least in comparison to the less mainstream architectures. Other architectures focus on either bandwidth, parallelism (but often without a great synchronization story), and optimizing power usage. So I doubt that Go lang would benefit from moving to Power. Some choices the Go people made about how channels how limited their options for optimizing channels / increase the complexity of a lock-free implementation (don't have the mailing list link handy). If you don't need all these guarantees you can use pick a SPSC, SPMC, MPMC implementation that might work better for your use case. > (Did you know that for i, _ := range is often 3 to 4x faster than for _, v := range? Do you know how awful code written down four or five nested loops that uses indices looks?) Yes, in the second version you have to make a copy of v. Depending on how large v this is how large the impact will be. The first version just references the array cell via a[i] and for that there's no copy needed and it's one assembly instruction. Maybe the optimizer can become better here but I'm guessing it might break some language contract.

The spark workflow is 10% cheaper (24.36 vs 21.88) over 3 years. Is that really worth having to support a non-mainstream architecture? I doubt it.

by mtanski 11y ago

The spark workflow is 10% cheaper (24.36 vs 21.88) over 3 years. Is that really worth having to support a non-mainstream architecture? I doubt it.

RankingMember 11y ago

That was my thought too. The "pain in the ass" factor associated with supporting a non-mainstream architecture is missing from the charts, and moves the dial away from IBM. That's not to say things couldn't change, but Xeons are proven, and unless you've got a surplus of money to throw at trying a new architecture (e.g. Google), you're better off sticking with the known good unless the predicted gains are sizable.

vessenes 11y ago

I just did this math for our company a couple months ago. Power8 memory bandwidth is VERY appealing. And, it's not always just about cost per benchmark unit -- if you have some realtime requirements for your analysis tools, then scale-up speed can be really valuable as compared to dev time and management time. In the end, we made the call for Intel because golang runs some key parts of our tools, and the golang power8 story isn't there. But, as I gaze at our servers where we paid what feels like thousands for extra megabytes of L3 cache, I wouldn't say I'm happy about the decision. A good go story from IBM would have likely tipped things the other way.

mtanski 11y ago
I think you just kind of proved my point that real work load benchmarks would have to be much more attractive to offset the cost of supporting an architecture that's not tier one in many languages / software libraries / projects.
- vessenes 11y ago
  Totally. That said, IBM isn't short on optimizing compiler folks. Open ecosystem support is a strategy to pick up small and mid-size buyers; it will be interesting to see if IBM gets there. I would look again next time we source hardware. And, if IBM put someone internally on a properly vectorized go compilation pathway for Power8, I would buy in a heartbeat, provided it ran some sort of debian variant.
  - mtanski 11y ago
    I don't think that go as people tend to use it really benefits from vectorized code. Most people who are using go that I interact with are not writing numerical processing code but network servers and business logic for high level web APIs. You might get minor speed ups in vectorized memcpy but I can't see much else. I imagine the most important CPU features for most go code would be a good branch predictor and fast atomics / synchronization primitives. If you're using go for numerical processing code I'd like to hear more about it. Mostly because it's kind of a PITA.
    - vessenes 11y ago
      Well, there are almost no real vectorizable primitives or functions in the core library so I'm not surprised that you don't run into people vectorizing much. And, go dev team compiler focus has been elsewhere the last year. And, so far the go team hasn't seemed to be able to interest Intel in doing the heavy lifting that they might do for some other compilers.. So, branch predictions and faster sync primitives would be great, not least because they would speed up channels in many cases, which would be cool; it would be nice to widen the use cases for channel-based communication significantly, but they're just VERY slow if you want to use them at scale in a large application. I am using go for some large scale numerical processing, although it's the sort with lots of logic attached, not just a giant matrix with some glue around it. It's kind of a PITA. We are picking and choosing some outside libraries, and spend a lot of time massaging the go code for speed and bitching about the garbage collector. (Did you know that for i, _ := range is often 3 to 4x faster than for _, v := range? Do you know how awful code written down four or five nested loops that uses indices looks?) But, the size codebase our team can manage with go is pretty great. We wouldn't be nearly so productive in many other cool (or .. experienced) languages when you add up the full life cycle costs including innovation, enhancement, bug fixes, maintenance and deployment. It's a win. I'd do it again in a heartbeat.
      - mtanski 11y ago
        > So, branch predictions and faster sync primitives would be great, not least because they would speed up channels in many cases, which would be cool; it would be nice to widen the use cases for channel-based communication significantly, but they're just VERY slow if you want to use them at scale in a large application. These operations are already pretty good on IA* processors, at least in comparison to the less mainstream architectures. Other architectures focus on either bandwidth, parallelism (but often without a great synchronization story), and optimizing power usage. So I doubt that Go lang would benefit from moving to Power. Some choices the Go people made about how channels how limited their options for optimizing channels / increase the complexity of a lock-free implementation (don't have the mailing list link handy). If you don't need all these guarantees you can use pick a SPSC, SPMC, MPMC implementation that might work better for your use case. > (Did you know that for i, _ := range is often 3 to 4x faster than for _, v := range? Do you know how awful code written down four or five nested loops that uses indices looks?) Yes, in the second version you have to make a copy of v. Depending on how large v this is how large the impact will be. The first version just references the array cell via a[i] and for that there's no copy needed and it's one assembly instruction. Maybe the optimizer can become better here but I'm guessing it might break some language contract.