3 ms·

It is beyond porting, it is mentality of developers. Change is expensive and I'm not just talking about $ value. > Also, Google has been using TPUs for a long

It is beyond porting, it is mentality of developers. Change is expensive and I'm not just talking about $ value. > Also, Google has been using TPUs for a long time now, and __they__ never hit a brick wall for a lack of CUDA. That's exactly what I'm saying. __they__ is the keyword.

amelius 1y ago
Not sure what you mean. Google is a big company. Their TPUs have many users internally.
- latchkey 1y ago
  Very few developers outside of Google have ever written code for a TPU. In a similar way, far fewer have written code for AMD, compared to NVIDIA. If you're going to design a custom chip and deploy it in your data centers, you're also committing to hiring and training developers to build for it. That's a kind of moat, but with private chips. While you solve one problem (getting the compute you want), you create another: supporting and maintaining that ecosystem long term. NVIDIA was successful because they got their hardware into developers hands, which created a feedback loop, developers asked for fixes/features, NVIDIA built them, the software stack improved, and the hardware evolved alongside it. That developer flywheel is what made CUDA dominant and is extremely hard to replicate because the shortage of talented developers is real.
  - amelius 1y ago
    I mean it's all true to some extent. But that doesn't mean implementing the few primitives to get transformers running requires CUDA, or that it's an impossible task. Remember, we're talking about >$1B companies here who can easily assemble teams of 10s-100s of developers. You can compare CUDA to the first PC OS, DOS 1.0. Sure, DOS was viewed as a moat at the time, but it didn't keep others from kicking its ass.
    - latchkey 1y ago
      > You can compare CUDA to the first PC OS, DOS 1.0. Sorry, I don't understand this comparison at all. CUDA isn't some first version of an OS, not even close. It's been developed for almost 20 years now. Bucketloads of documentation, software and utility have been created around it. It won't have its ass kicked by any stretch of imagination.
      - amelius 1y ago
        Yes, CUDA has a history. And it shows. CUDA has very bad integration with the OS for example. It's time some other company (Microsoft sounds like a good contender) showed them how you do this the right way. Anyway, this all distracts from the fact that you don't need an entire "OS" just to run some arithmetic primitives to get transformers running.
        latchkey 1y ago
        > CUDA has very bad integration with the OS for example. If you want to cherry pick anything, you can. But in my eyes, you're just solidifying my point. Software is critical. Minimizing the surface is obviously a good thing (tinygrad for example), but you're still going to need people who are willing and able to write the code.
        amelius 1y ago
        OK, but Microsoft is a software company ...
  - VirusNewbie 1y ago
    >Very few developers outside of Google have ever written code for a TPU. Anthropic?

amelius 1y ago

Not sure what you mean. Google is a big company. Their TPUs have many users internally.

latchkey 1y ago
Very few developers outside of Google have ever written code for a TPU. In a similar way, far fewer have written code for AMD, compared to NVIDIA. If you're going to design a custom chip and deploy it in your data centers, you're also committing to hiring and training developers to build for it. That's a kind of moat, but with private chips. While you solve one problem (getting the compute you want), you create another: supporting and maintaining that ecosystem long term. NVIDIA was successful because they got their hardware into developers hands, which created a feedback loop, developers asked for fixes/features, NVIDIA built them, the software stack improved, and the hardware evolved alongside it. That developer flywheel is what made CUDA dominant and is extremely hard to replicate because the shortage of talented developers is real.
- amelius 1y ago
  I mean it's all true to some extent. But that doesn't mean implementing the few primitives to get transformers running requires CUDA, or that it's an impossible task. Remember, we're talking about >$1B companies here who can easily assemble teams of 10s-100s of developers. You can compare CUDA to the first PC OS, DOS 1.0. Sure, DOS was viewed as a moat at the time, but it didn't keep others from kicking its ass.
  - latchkey 1y ago
    > You can compare CUDA to the first PC OS, DOS 1.0. Sorry, I don't understand this comparison at all. CUDA isn't some first version of an OS, not even close. It's been developed for almost 20 years now. Bucketloads of documentation, software and utility have been created around it. It won't have its ass kicked by any stretch of imagination.
    - amelius 1y ago
      Yes, CUDA has a history. And it shows. CUDA has very bad integration with the OS for example. It's time some other company (Microsoft sounds like a good contender) showed them how you do this the right way. Anyway, this all distracts from the fact that you don't need an entire "OS" just to run some arithmetic primitives to get transformers running.
      - latchkey 1y ago
        > CUDA has very bad integration with the OS for example. If you want to cherry pick anything, you can. But in my eyes, you're just solidifying my point. Software is critical. Minimizing the surface is obviously a good thing (tinygrad for example), but you're still going to need people who are willing and able to write the code.
        amelius 1y ago
        OK, but Microsoft is a software company ...
- VirusNewbie 1y ago
  >Very few developers outside of Google have ever written code for a TPU. Anthropic?