5 ms·

It’s not 10x more expensive. For example, Cloud TPU costs $12 for ImageNet training. AWS is $42. Lambda GPU Cloud is $19. You also need to take into account th

It’s not 10x more expensive. For example, Cloud TPU costs $12 for ImageNet training. AWS is $42. Lambda GPU Cloud is $19. You also need to take into account the cost of changing and tuning your code to run on a TPU. https://dawn.cs.stanford.edu/benchmark/#imagenet-train-cost https://dawn.cs.stanford.edu/benchmark/#imagenet-train-cost

fxtentacle 7y ago
In my case it was 10x. We're training networks to estimate optical flow on Sintel and KITTI using TensorFlow. On GPU, we hit the 12GB limit which caused TensorFlow to log a message that convolution performance would be limited due to not having enough cache RAM. On TPU, we didn't hit the RAM limit. Oh and amazingly enough, I took the exact same Python file that I built on GPU on my workstation, ran it in Google Collab with a TPU connected, and I really only had to change the configuration to make it work. There was no need to change or tune anything. I was pleasantly surprised by how well that worked. And let me rant a bit longer, when we started this project I actually first tried to use AWS because our company already had billing set up with them. But AWS was using CUDA 10.0 for SageMaker, which has a known crash- and nan-inducing bug for 2D convolution. AWS support being non-existent as always, we couldn't make things work on AWS and then went bare metal for our GPU training with hetzner.com - which simultaneously also shaved like $30,000 monthly off our bill. AWS's EC2 GPU instances are really expensive when compared to a dedicated root server.
- scarface74 7y ago
  I’m not arguing either way. But that’s still not where the money or growth is. Cloud computing only makes up 5% of enterprise (big Fortune 500 companies) computer spend. MS recognized moving enterprises with boring workloads to Azure was where the money was along with hybrid computing. That’s how it was able to succeed so quickly. Besides,Microsoft has always been focused on the dark matter developers doing boring stuff. Their tooling is far better than Google’s and their IDEs that developers are already using integrate with their cloud offerings.
  - fxtentacle 7y ago
    From watching NVIDIA's presentations, it sure sounds like AI is growing exponentially. Plus Google needs AI to advance for Waymo to become safe and profitable. Looking at other companies in my area, all of them have plans for AI, too, even if it is trivial stuff like automating the most repetitive 5% of support inquiries. So if Google can beat AWS out of the AI waters early on, that might become very valuable in a few years.
    - scarface74 7y ago
      It’s easy to grow “exponentially” from a small base. Also, if the only advantage that GCP has is price, that’s not defensible. It’s not like it takes a great engineering effort to change a price. The market leader can usually afford to be a “fast follower”. That was MS’s playbook for decades. I posted a link earlier from a podcast interview of someone who works at GCP. She said that only a few small internal projects at Google run on GCP.
      - deleted 7y ago
        [deleted]
  - jeffshek 7y ago
    I may be wrong, but I recall reading Azure grew quickly because they already had MS contracts in place with many enterprise companies. They simply said, we’ll also hold your hand in the entire cloud process. This was critical because by then if you weren’t on AWS, you probably didn’t have the DevOps team to do so - MS was now willing to lend you enough consultants to do it for you. The number of spend shouldn’t be astronomically high. Most workflows in cloud computing are commoditized low-margin. It also explains why Google wants to do high-margin AI stuff, and why AWS is happy with low-margin areas.
    - scarface74 7y ago
      That’s kind of my point and what I’ve been saying. Microsoft doesn’t tell a company that they should be using their new cool technology because we are “smart people” (tm) who know what’s best. Microsoft would say: “If you have a workload that still requires Windows 2000 and SQL Server 2003. We will help you migrate it to Azure as is and we will extend your support contract.” While Google will talk about migrating your entire workload to K8s.

fxtentacle 7y ago

In my case it was 10x. We're training networks to estimate optical flow on Sintel and KITTI using TensorFlow. On GPU, we hit the 12GB limit which caused TensorFlow to log a message that convolution performance would be limited due to not having enough cache RAM. On TPU, we didn't hit the RAM limit. Oh and amazingly enough, I took the exact same Python file that I built on GPU on my workstation, ran it in Google Collab with a TPU connected, and I really only had to change the configuration to make it work. There was no need to change or tune anything. I was pleasantly surprised by how well that worked. And let me rant a bit longer, when we started this project I actually first tried to use AWS because our company already had billing set up with them. But AWS was using CUDA 10.0 for SageMaker, which has a known crash- and nan-inducing bug for 2D convolution. AWS support being non-existent as always, we couldn't make things work on AWS and then went bare metal for our GPU training with hetzner.com - which simultaneously also shaved like $30,000 monthly off our bill. AWS's EC2 GPU instances are really expensive when compared to a dedicated root server.

scarface74 7y ago
I’m not arguing either way. But that’s still not where the money or growth is. Cloud computing only makes up 5% of enterprise (big Fortune 500 companies) computer spend. MS recognized moving enterprises with boring workloads to Azure was where the money was along with hybrid computing. That’s how it was able to succeed so quickly. Besides,Microsoft has always been focused on the dark matter developers doing boring stuff. Their tooling is far better than Google’s and their IDEs that developers are already using integrate with their cloud offerings.
- fxtentacle 7y ago
  From watching NVIDIA's presentations, it sure sounds like AI is growing exponentially. Plus Google needs AI to advance for Waymo to become safe and profitable. Looking at other companies in my area, all of them have plans for AI, too, even if it is trivial stuff like automating the most repetitive 5% of support inquiries. So if Google can beat AWS out of the AI waters early on, that might become very valuable in a few years.
  - scarface74 7y ago
    It’s easy to grow “exponentially” from a small base. Also, if the only advantage that GCP has is price, that’s not defensible. It’s not like it takes a great engineering effort to change a price. The market leader can usually afford to be a “fast follower”. That was MS’s playbook for decades. I posted a link earlier from a podcast interview of someone who works at GCP. She said that only a few small internal projects at Google run on GCP.
    - deleted 7y ago
      [deleted]
- jeffshek 7y ago
  I may be wrong, but I recall reading Azure grew quickly because they already had MS contracts in place with many enterprise companies. They simply said, we’ll also hold your hand in the entire cloud process. This was critical because by then if you weren’t on AWS, you probably didn’t have the DevOps team to do so - MS was now willing to lend you enough consultants to do it for you. The number of spend shouldn’t be astronomically high. Most workflows in cloud computing are commoditized low-margin. It also explains why Google wants to do high-margin AI stuff, and why AWS is happy with low-margin areas.
  - scarface74 7y ago
    That’s kind of my point and what I’ve been saying. Microsoft doesn’t tell a company that they should be using their new cool technology because we are “smart people” (tm) who know what’s best. Microsoft would say: “If you have a workload that still requires Windows 2000 and SQL Server 2003. We will help you migrate it to Azure as is and we will extend your support contract.” While Google will talk about migrating your entire workload to K8s.