4 ms·

The march of faster and faster transistors seems to be headed for an interregnum until some new technology can replace CMOS. Price per compute might still cont

The march of faster and faster transistors seems to be headed for an interregnum until some new technology can replace CMOS. Price per compute might still continue to fall though. I hope that we'll see an era of more architectural diversity coming up as the end of the march of nodes mean that a design can take longer to pay off and still be economical.

ghaff 8y ago
>an era of more architectural diversity We're already seeing this in machine learning. I used to follow the processor space fairly closely and I'd see various companies come out with designs that were specialized for some workload or other. The problem was that you could instead wait for a couple years and x86 would be just as fast. You also have some other things going on like open source software, centralized computing in clouds, etc. that make having hardware that's optimized for some specific workload much more tenable.
- baybal2 8y ago
  >The problem was that you could instead wait for a couple years and x86 would be just as fast. This still holds for the majority of users. Only tier 1 dotcoms can run stuff on GPUs economically. For everybody else, just getting many times, possibly few magnitudes, more commodity x86 machine time still makes sense. I always remember the saying of my ops unit head once I worked in an advertising network. It was something to the meaning of: "a new coder like you who can find a fancy algo to do thing A and B faster pays off in two years, but a 10 new servers will pay off in just 2 month" And I am still convinced that fabs will continue mercilessly squeezing CMOS till the last drop of blood for at least a decade. There are tons and tons of avenues to make cells smaller, cooler, and more performant that don't involve further process down scaling or switching to a next gen semiconductor.
  - edraferi 8y ago
    > Only tier 1 dotcoms can run stuff on GPUs economically. For everybody else, just getting many times, possibly few magnitudes, more commodity x86 machine time still makes sense. Disagree. Anybody running appreciable ML workloads buys and racks their own GPUs.
    - baybal2 8y ago
      The fact that they buy and own their GPUs already says that they spent a whole magnitude more than if they just rented CPU time. Moreover, say, a tier 2 player like a major AdNet rents its CPU time at rates that make AWS users cry. For them, it makes even more sense to just "let the code be a horrid contraption written by 10$ per hour outsourcing shop, but we can run in on cheap hardware"
      - annywhey 8y ago
        There are other considerations beyond a simple cost per minute average here: * Uptime guarantees - AWS goes down. Running your own hardware locally, you at least have a recourse. * Storage and bandwidth - some of these datasets are huge and not amenable to streaming onto a temporary rental. * Security - the ever-present bogeyman of cloud services. It might be harder to lock down your own machines, but it's the only way to fulfill some contractual requirements. For commodity processing that doesn't involve these specialties, sure, fire away. That's what it's good at, and what most people need most of the time. But there's always a final TCO number and arriving at it requires accounting for all the risks.
- edraferi 8y ago
  > centralized computing in clouds [could] make having hardware that's optimized for some specific workload much more tenable. I think this is the key. GPUs are pretty much the only consumer-level specialized co-processors, and those have been selling like gang busters. Plot twist, it's the data centers that are buying them up... to the point that the manufacturers tried to prevent you from buying more than one at a time. Of course specialized processors need to have proper networking and storage proximity to really deliver value. May be more challenging to offer a specialized service like this than you'd hope. Perhaps the big cloud vendors will dominate again, so chip manufacturers need to just hope for a few big contracts to Amazon / Microsoft / Google.
  - jcranmer 8y ago
    > Plot twist, it's the data centers that are buying them up... to the point that the manufacturers tried to prevent you from buying more than one at a time. No, it's the cryptocurrency miners that caused that restriction to be introduced.
    - Dylan16807 8y ago
      Not sure why you're being downvoted. It's possible that nVidia's unsold GPUs are just a coincidence with crypto price drops, but I'd like to see some evidence.

ghaff 8y ago

>an era of more architectural diversity We're already seeing this in machine learning. I used to follow the processor space fairly closely and I'd see various companies come out with designs that were specialized for some workload or other. The problem was that you could instead wait for a couple years and x86 would be just as fast. You also have some other things going on like open source software, centralized computing in clouds, etc. that make having hardware that's optimized for some specific workload much more tenable.

baybal2 8y ago
>The problem was that you could instead wait for a couple years and x86 would be just as fast. This still holds for the majority of users. Only tier 1 dotcoms can run stuff on GPUs economically. For everybody else, just getting many times, possibly few magnitudes, more commodity x86 machine time still makes sense. I always remember the saying of my ops unit head once I worked in an advertising network. It was something to the meaning of: "a new coder like you who can find a fancy algo to do thing A and B faster pays off in two years, but a 10 new servers will pay off in just 2 month" And I am still convinced that fabs will continue mercilessly squeezing CMOS till the last drop of blood for at least a decade. There are tons and tons of avenues to make cells smaller, cooler, and more performant that don't involve further process down scaling or switching to a next gen semiconductor.
- edraferi 8y ago
  > Only tier 1 dotcoms can run stuff on GPUs economically. For everybody else, just getting many times, possibly few magnitudes, more commodity x86 machine time still makes sense. Disagree. Anybody running appreciable ML workloads buys and racks their own GPUs.
  - baybal2 8y ago
    The fact that they buy and own their GPUs already says that they spent a whole magnitude more than if they just rented CPU time. Moreover, say, a tier 2 player like a major AdNet rents its CPU time at rates that make AWS users cry. For them, it makes even more sense to just "let the code be a horrid contraption written by 10$ per hour outsourcing shop, but we can run in on cheap hardware"
    - annywhey 8y ago
      There are other considerations beyond a simple cost per minute average here: * Uptime guarantees - AWS goes down. Running your own hardware locally, you at least have a recourse. * Storage and bandwidth - some of these datasets are huge and not amenable to streaming onto a temporary rental. * Security - the ever-present bogeyman of cloud services. It might be harder to lock down your own machines, but it's the only way to fulfill some contractual requirements. For commodity processing that doesn't involve these specialties, sure, fire away. That's what it's good at, and what most people need most of the time. But there's always a final TCO number and arriving at it requires accounting for all the risks.
edraferi 8y ago
> centralized computing in clouds [could] make having hardware that's optimized for some specific workload much more tenable. I think this is the key. GPUs are pretty much the only consumer-level specialized co-processors, and those have been selling like gang busters. Plot twist, it's the data centers that are buying them up... to the point that the manufacturers tried to prevent you from buying more than one at a time. Of course specialized processors need to have proper networking and storage proximity to really deliver value. May be more challenging to offer a specialized service like this than you'd hope. Perhaps the big cloud vendors will dominate again, so chip manufacturers need to just hope for a few big contracts to Amazon / Microsoft / Google.
- jcranmer 8y ago
  > Plot twist, it's the data centers that are buying them up... to the point that the manufacturers tried to prevent you from buying more than one at a time. No, it's the cryptocurrency miners that caused that restriction to be introduced.
  - Dylan16807 8y ago
    Not sure why you're being downvoted. It's possible that nVidia's unsold GPUs are just a coincidence with crypto price drops, but I'd like to see some evidence.