4 ms·

Let me fuel your paranoia a little. It’s likely that cloud computing will continue to see decreases in costs, which means the availability of computing will co

by npo9 7y ago

Let me fuel your paranoia a little. It’s likely that cloud computing will continue to see decreases in costs, which means the availability of computing will continue to raise after Mores Law has stopped giving. Quantum computers might play a role in increasing computational power in the next decade. Right now a lot of AI research is low hanging fruit recently made available with advances in CPU and GPU. After the advances dry up maybe more fundamental research will produce much better results with our current technology.

_bxg1 7y ago
Thanks :P
martinpw 7y ago
> It’s likely that cloud computing will continue to see decreases in costs, which means the availability of computing will continue to raise after Mores Law has stopped giving. Why would cloud computing continue to see significant decreases in costs once Moore's law stops? Clearly there will be some ongoing decline in infrastructure costs, but if your main requirement is for lots of compute (for AI) then it seems hard to maintain an exponential rate of improvement overall if the silicon is no longer getting faster and cheaper, and the exponential is what you need, else your costs get out of control quickly.
- npo9 7y ago
  I was speaking about linear decreases. Cheaper power, more refined manufacturing processes, and economy of scale benefits are where I expect some of the linear benefits to come from. We can also expect CPU manufactures to compete for awhile, getting increasingly marginal improvements. Linear improvements will get us there in the end.
  - martinpw 7y ago
    > Linear improvements will get us there in the end. No, they won't. That's the point. To quote directly from the article: we’re talking about exponential rates of growth here, linear expense adjustments won’t move the needle.
  - dogma1138 7y ago
    Why would power will become cheaper? Also manufacturing refinement? Sure it happens but not to any meaningful degree these days it’s more about bribing a new process to the same cost/efficiency as your older manufacturing processes. As far as economy of scales goes it’s also not a given. Large cloud providers are much more adapt at reusing old hardware for their PAAS and weaker instances. I’ve had a chance to talk to a few reps from Dell once and while cloud buy a lot of servers they also upgrade them much less often than large enterprises that run their own DC this is because Amazon can always continue to make money on 3-5 year old hardware by shifting it off to cheaper tiers or by running things like SQS or Lambdas on it in the background. On the other hand if you have a 3-5 year old server and you need more horse power if you manage your own DC or rack space the hosting costs make it economical to keep underperforming hardware. Not to mention other costs such as support are quite different, if an Amazon server dies they don’t care. Traditional self hosted environments will require extended support because they can’t have the same redundancy as Amazon. If you delegate a 5 year old server to some back office applications while you use your new ones for client facing apps the old server is still a mission critical box even if it doesn’t directly run revenue generating services. Basically Amazon buys to grow, on-prem orgs buy to both grow and upgrade. If you have an organization the size of Amazon that is completely self hosted they will end up buying much more hardware over a decade than Amazon.
the8472 7y ago
> Quantum computers might play a role in increasing computational power in the next decade. It won't increase your FLOP/s in any way. It will mostly[0] help solving narrow problem classes, those outside P but in BQP[1]. To my understanding none of the problems encountered in NN training falls into that. [0] ignoring polynomial speedups such as grover's algorithm [1] https://en.wikipedia.org/wiki/BQP https://en.wikipedia.org/wiki/BQP
sgt101 7y ago
>cloud computing will continue to see decreases in costs cloud computing costs much more than on prem; it's available on demand, which is why it's attractive to some companies. For everyone else it's an outsourcing play that unlocks capex for revenue generation. >Quantum computers might play a role in increasing computational power in the next decade. Nope. No chance. Best case is that QC will provide improved molecular simulations in 10-20 years. But that will be a very specialised result. >Right now a lot of AI research is low hanging fruit recently made available with advances in CPU and GPU. After the advances dry up maybe more fundamental research will produce much better results with our current technology. 100%, but the key word is maybe. One problem I anticipate is that the current skill set of grad students and the current management mind set in academia and commercial research is not well fitted to the kind of fundamental research that includes maybe and risk. I think that a more likely turn is that there will be a focus on how to better exploit the tech for data science and ML that we have. Currently we're pretty hit and miss and also the results are rather unpredictable and brittle. When did you last find an ML project on Github with unit tests?
- jayd16 7y ago
  Its not that computers in the cloud are cheap, its that the capacity will continue to grow and they're fully networked such that our AI overlords can take full advantage of them.
- TomMarius 7y ago
  > cloud computing costs much more than on prem; Yeah, and that will continue to go down, exactly because of what you stated in the next sentence.

by npo9 7y ago

_bxg1 7y ago

Thanks :P

martinpw 7y ago

> It’s likely that cloud computing will continue to see decreases in costs, which means the availability of computing will continue to raise after Mores Law has stopped giving. Why would cloud computing continue to see significant decreases in costs once Moore's law stops? Clearly there will be some ongoing decline in infrastructure costs, but if your main requirement is for lots of compute (for AI) then it seems hard to maintain an exponential rate of improvement overall if the silicon is no longer getting faster and cheaper, and the exponential is what you need, else your costs get out of control quickly.

npo9 7y ago
I was speaking about linear decreases. Cheaper power, more refined manufacturing processes, and economy of scale benefits are where I expect some of the linear benefits to come from. We can also expect CPU manufactures to compete for awhile, getting increasingly marginal improvements. Linear improvements will get us there in the end.
- martinpw 7y ago
  > Linear improvements will get us there in the end. No, they won't. That's the point. To quote directly from the article: we’re talking about exponential rates of growth here, linear expense adjustments won’t move the needle.
- dogma1138 7y ago
  Why would power will become cheaper? Also manufacturing refinement? Sure it happens but not to any meaningful degree these days it’s more about bribing a new process to the same cost/efficiency as your older manufacturing processes. As far as economy of scales goes it’s also not a given. Large cloud providers are much more adapt at reusing old hardware for their PAAS and weaker instances. I’ve had a chance to talk to a few reps from Dell once and while cloud buy a lot of servers they also upgrade them much less often than large enterprises that run their own DC this is because Amazon can always continue to make money on 3-5 year old hardware by shifting it off to cheaper tiers or by running things like SQS or Lambdas on it in the background. On the other hand if you have a 3-5 year old server and you need more horse power if you manage your own DC or rack space the hosting costs make it economical to keep underperforming hardware. Not to mention other costs such as support are quite different, if an Amazon server dies they don’t care. Traditional self hosted environments will require extended support because they can’t have the same redundancy as Amazon. If you delegate a 5 year old server to some back office applications while you use your new ones for client facing apps the old server is still a mission critical box even if it doesn’t directly run revenue generating services. Basically Amazon buys to grow, on-prem orgs buy to both grow and upgrade. If you have an organization the size of Amazon that is completely self hosted they will end up buying much more hardware over a decade than Amazon.

the8472 7y ago

> Quantum computers might play a role in increasing computational power in the next decade. It won't increase your FLOP/s in any way. It will mostly[0] help solving narrow problem classes, those outside P but in BQP[1]. To my understanding none of the problems encountered in NN training falls into that. [0] ignoring polynomial speedups such as grover's algorithm [1] https://en.wikipedia.org/wiki/BQP https://en.wikipedia.org/wiki/BQP

sgt101 7y ago

>cloud computing will continue to see decreases in costs cloud computing costs much more than on prem; it's available on demand, which is why it's attractive to some companies. For everyone else it's an outsourcing play that unlocks capex for revenue generation. >Quantum computers might play a role in increasing computational power in the next decade. Nope. No chance. Best case is that QC will provide improved molecular simulations in 10-20 years. But that will be a very specialised result. >Right now a lot of AI research is low hanging fruit recently made available with advances in CPU and GPU. After the advances dry up maybe more fundamental research will produce much better results with our current technology. 100%, but the key word is maybe. One problem I anticipate is that the current skill set of grad students and the current management mind set in academia and commercial research is not well fitted to the kind of fundamental research that includes maybe and risk. I think that a more likely turn is that there will be a focus on how to better exploit the tech for data science and ML that we have. Currently we're pretty hit and miss and also the results are rather unpredictable and brittle. When did you last find an ML project on Github with unit tests?

jayd16 7y ago
Its not that computers in the cloud are cheap, its that the capacity will continue to grow and they're fully networked such that our AI overlords can take full advantage of them.
TomMarius 7y ago
> cloud computing costs much more than on prem; Yeah, and that will continue to go down, exactly because of what you stated in the next sentence.