12 ms·

Nvidia and Salesforce double down on AI startup Cohere in $450M round

52 points by iam_a_user 2y ago

behnamoh 2y ago
Man I hate Nvidia and their monopolistic behavior. Making 800% margins on H100s is just absolute greed imo. They make good GPUs but they don't deserve to be an almost $3T company...
- talldayo 2y ago
  I hate the industry for never learning how to get along and make a real CUDA competitor. Nvidia can make 1600% margins for all I care, this is exactly the sort of future we deserve when nobody can agree on a common GPU API. You want competition, you want vertical integration? Here it is. Compete.
  - riku_iki 2y ago
    > I hate the industry for never learning how to get along and make a real CUDA competitor. there are google TPUs. Do they provide better pefrormance/dollar, or google also charges high margin, or Nvidia is doing some unique optimizations?
    - deleted 2y ago
      [deleted]
    - betaby 2y ago
      Our company can buy NVIDIA gear. Google TPU, well, is google's property I have no control over. This site is overrepresented by clod folks. In reality ~90% of the workloads are NOT cloud based. Unless one can buy Google/Amazon/Meta T/N/AI/PUs that unique optimizations are irrelevant for the most of the workloads.
      - debbiedowner 2y ago
        Why 90%? Why don't you go on the cloud for training at least if it's not healthcare data? Is it because you don't need to buy many gpus to do your workload?
        betaby 2y ago
        Reasons why people don't go on the cloud were outlined many times, i.e. https://world.hey.com/dhh/we-have-left-the-cloud-251760fb https://world.hey.com/dhh/we-have-left-the-cloud-251760fb was discussed on HN many times. I could have written almost the same reasons for GPU workloads.
    - thebruce87m 2y ago
      Who would go with google for anything? No support and the product cancelled when they get distracted with something else.
    - talldayo 2y ago
      The TPU is basically an ASIC as far as I know; it competes against CUDA in a very small subset of it's featureset. CUDA is essentially a composition layer on top of multiple GPU features that optimizes them for general-purpose compute. In essence, nothing is stopping Apple or Google from making an Open Source CUDA replacement and undermining the demand for specialized GPGPU compute. The problem is that CUDA is massive, and nobody wants to re-implement it (especially not for free). So now Nvidia is in the privileged position of having both highly-flexible GPGPU compute hardware, as well as a highly-advanced software layer to use it with. TPUs and NPUs are neat, but fundamentally they are neither of these things; they have an extremely limited processing pipeline exposed by a high-level library, and that's usually it. CUDA is comparatively flexible, to the point that it doesn't even rely on AI to sell it's product. To me, hating on Nvidia feels like being mad that a well-bred horse with great odds beat out the jockey you were betting on. Why should we hate them, for their "monopoly" on features that Apple and Khronos gave up developing? Because they're blocking-out their competitors by... not having working MacOS drivers per Apple's request? This is the causal and obvious outcome of letting businesses commoditize specialized compute. This is what the industry wanted, and it's rich watching the customers protest like they were fooled into thinking everything was fine.
      - riku_iki 2y ago
        > The TPU is basically an ASIC as far as I know; it competes against CUDA in a very small subset of it's featureset. CUDA is essentially a composition layer on top of multiple GPU features that optimizes them for general-purpose compute. my understanding is that compilers can compile some straighforward JAX, TF, Pytorch programs to both Cuda and TPU, so they in direct competition in current hot topics (LLM, deep learning).
        talldayo 2y ago
        Right; but you can't cross-compile everything. This is really common in AI libraries, especially multi-target projects like ONNX: https://onnx.ai/ https://onnx.ai/ The math probably adds up in Google's favor with the TPUs, even if they end up being less efficient and slower per-unit than Nvidia hardware. They don't need to pay for the margins, and they can run them 24/7 for their intended purpose. The previous-generation TPUs can't be reused or resold for other purposes though, and if/when AI blows over as a trend you probably can't easily start mining crypto or doing HPC calculations like an Nvidia cluster would.
- philomath_mn 2y ago
  If they are selling something that everyone wants and there aren't enough GPUs to around, how else do you suggest allocating those GPUs to the public except by increasing the price?
- rodiger 2y ago
  "Your margin is my opportunity" A ton of companies are dumping R&D into designing their own chips with seeming success. Nvidia will be in tough company quite soon, but for now demand greatly outstrips supply.
- asadm 2y ago
  AMD had a chance, they dropped the ball and still won't fix their ways (on devexp, drivers, etc). I think Nvidia deserves the success.
- Ductapemaster 2y ago
  So say they mark them down to some reasonable level — they are still supply constrained and now the line just gets longer. It doesn’t really fix anything. Pricing allows them to prioritize their customers, which seems reasonable.
  - pie420 2y ago
    if the line gets long enough, it allows other companies to compete with Nvidia and creates a healthier, more competitive market, in the same way AMD has made intel a more honest, fairer, less price-gouging company. Competition is the entire thesis of american capitalism
    - talldayo 2y ago
      > Competition is the entire thesis of american capitalism Then why aren't people competing with Nvidia? Why is OpenCL on life support and unsupported on major operating systems? Why are we doing this song-and-dance routine refusing to adopt certain GPU APIs but embracing closed ones instead? I'd like to believe that a tipping point will be reached, but if not now then when? People have talked about upending Nvidia's GPGPU compute empire for years, but besides application-specific replacements and proofs-of-concept, we don't have a real CUDA-killer. Apple does not ship one, Google does not ship one, Microsoft does not ship one and AMD doesn't either. So... when? If we continue along the current path, I suspect Nvidia will continue to find markets where CUDA is demanded and OEMS will continue to chase them down with half-measure solutions. Unless OpenCL is revived or someone commits to a proprietary CUDA-like platform, I suspect we'll be spinning our wheels and digging ourselves deeper.
      - thorncorona 2y ago
        > Then why aren't people competing with Nvidia? Why is OpenCL on life support and unsupported on major operating systems? Why are we doing this song-and-dance routine refusing to adopt certain GPU APIs but embracing closed ones instead? Because Nvidia invested 20 years into its API platform, and this advantage is slowly getting realized.
- dcgudeman 2y ago
  Here is the definition of monopoly: "the exclusive possession or control of the supply of or trade in a commodity or service." There are other huge players: AMD and Intel. Additionally Google and Amazon are producing their own AI hardware. Now can you explain to me how Nvidia has a monopoly?
  - Spivak 2y ago
    Because monopoly means "wields monopoly power and engages in anticompetitive behavior that harms consumers." Having competition or not, having large market share or not doesn't matter one bit. One of ways of wielding monopoly power is no longer being a price taker and being able to set prices at in practice to whatever you want.
    - dcgudeman 2y ago
      Your definition of monopoly contains a reference to "monopoly", so I am still not sure what your definition of monopoly is. It sounds like it is something like "big company that charges a price you think is high". Why do you think you know what the "right" price is. Isn't it a good thing the company delivering the most value is capturing the most profits? They will likely continue to deliver breakthroughs with the profits they are generating. Why would you want to kneecap a company generating so much value?
- IncreasePosts 2y ago
  If they sold them at 30% margins or whatever you think is reasonable, then scalpers would just pocket the 770%. Not sure how that is better for the ultimate purchasers or Nvidia.
- talldatethrow 2y ago
  Tech hardware companies are the only people where I'm not annoyed at their prices. Want a fence built for you? $3000 for 2 days work with 96 pieces of wood. Want a basic android phone with 4 gb of memory, 128 gb of storage, a camera, speakers, touch screen, wifi, cell tower connection, GPS, battery, charger and cable? $60 on sale at Walmart for a Moto G Power Stylus edition. (My current phone and that's what I paid a few months ago)
- hi-v-rocknroll 2y ago
  Monetize while the iron is hot. It's because Microsoft/OpenAI and Meta are throwing multiple $10B's at NVIDIA. This is unsustainable and NVIDIA's stock will contract over 3-6 years unless Microsoft and Meta can translate hype into essential, profitable products which isn't a sure thing™.
mrweasel 2y ago
Best description ever of what AI companies produce: "software systems that are trained on large amounts of data and can generate text". No further explanation of what Cohere might provide to it's customer of value, just the ability to generate text.
- eamsen 2y ago
  Yet, here we are, generating text. Some of which has value.
  - andrei_says_ 2y ago
    But often it’s hard to know which.
- DebtDeflation 2y ago
  If we're going to waste electricity, I'd rather we waste it on AI than on mining Bitcoin. At least AI is providing some utility to its consumers and there's reason to believe it will provide even more in the future.
  - kaliqt 2y ago
    Bitcoin provides massive utility to customers. The banking cartels + global government massive taxes are destroying livelihoods daily. The compute is "wasted" to pick winners and secure the system, but it's not actually wasted, it can be made more efficient by actually using the compute to do something, but even if that was not the case, it pays back its value 100x.
  - ErikBjare 2y ago
    "waste it on AI" "at least AI is providing utility" Doesn't sound like a waste, does it? I'd go further and argue mining Bitcoin isn't either, but I digress.
- itkovian_ 2y ago
  I think there's a reasonable argument that cohere might not capture the value and it might all go to the big players. But I can't understand how anyone still has the view llms themselves are not valuable.
tuttyboy 2y ago
There are so many LLMs that they’re starting to feel cheap.
- hi-v-rocknroll 2y ago
  The shark will be completely jumped once McDonald's offers a Happy Meal prize that's a virtual pet powered by an LLM.
  - djmips 2y ago
    That sounds cool.
    - powerbroker 2y ago
      Its even cooler if the pet has fleas that are organized as a cellular automata that plays the game of 'life' on the body of the pet.
      - djmips 2y ago
        another great idea!
metadat 2y ago
Also Oracle is in on this one: https://www.oracle.com/customers/cohere/ https://www.oracle.com/customers/cohere/
anothername12 2y ago
What's the best way to make money of all this LLM stuff before it implodes?

52 points by iam_a_user 2y ago

Man I hate Nvidia and their monopolistic behavior. Making 800% margins on H100s is just absolute greed imo. They make good GPUs but they don't deserve to be an almost $3T company...

talldayo 2y ago
I hate the industry for never learning how to get along and make a real CUDA competitor. Nvidia can make 1600% margins for all I care, this is exactly the sort of future we deserve when nobody can agree on a common GPU API. You want competition, you want vertical integration? Here it is. Compete.
- riku_iki 2y ago
  > I hate the industry for never learning how to get along and make a real CUDA competitor. there are google TPUs. Do they provide better pefrormance/dollar, or google also charges high margin, or Nvidia is doing some unique optimizations?
  - deleted 2y ago
    [deleted]
  - betaby 2y ago
    Our company can buy NVIDIA gear. Google TPU, well, is google's property I have no control over. This site is overrepresented by clod folks. In reality ~90% of the workloads are NOT cloud based. Unless one can buy Google/Amazon/Meta T/N/AI/PUs that unique optimizations are irrelevant for the most of the workloads.
    - debbiedowner 2y ago
      Why 90%? Why don't you go on the cloud for training at least if it's not healthcare data? Is it because you don't need to buy many gpus to do your workload?
      - betaby 2y ago
        Reasons why people don't go on the cloud were outlined many times, i.e. https://world.hey.com/dhh/we-have-left-the-cloud-251760fb https://world.hey.com/dhh/we-have-left-the-cloud-251760fb was discussed on HN many times. I could have written almost the same reasons for GPU workloads.
  - thebruce87m 2y ago
    Who would go with google for anything? No support and the product cancelled when they get distracted with something else.
  - talldayo 2y ago
    The TPU is basically an ASIC as far as I know; it competes against CUDA in a very small subset of it's featureset. CUDA is essentially a composition layer on top of multiple GPU features that optimizes them for general-purpose compute. In essence, nothing is stopping Apple or Google from making an Open Source CUDA replacement and undermining the demand for specialized GPGPU compute. The problem is that CUDA is massive, and nobody wants to re-implement it (especially not for free). So now Nvidia is in the privileged position of having both highly-flexible GPGPU compute hardware, as well as a highly-advanced software layer to use it with. TPUs and NPUs are neat, but fundamentally they are neither of these things; they have an extremely limited processing pipeline exposed by a high-level library, and that's usually it. CUDA is comparatively flexible, to the point that it doesn't even rely on AI to sell it's product. To me, hating on Nvidia feels like being mad that a well-bred horse with great odds beat out the jockey you were betting on. Why should we hate them, for their "monopoly" on features that Apple and Khronos gave up developing? Because they're blocking-out their competitors by... not having working MacOS drivers per Apple's request? This is the causal and obvious outcome of letting businesses commoditize specialized compute. This is what the industry wanted, and it's rich watching the customers protest like they were fooled into thinking everything was fine.
    - riku_iki 2y ago
      > The TPU is basically an ASIC as far as I know; it competes against CUDA in a very small subset of it's featureset. CUDA is essentially a composition layer on top of multiple GPU features that optimizes them for general-purpose compute. my understanding is that compilers can compile some straighforward JAX, TF, Pytorch programs to both Cuda and TPU, so they in direct competition in current hot topics (LLM, deep learning).
      - talldayo 2y ago
        Right; but you can't cross-compile everything. This is really common in AI libraries, especially multi-target projects like ONNX: https://onnx.ai/ https://onnx.ai/ The math probably adds up in Google's favor with the TPUs, even if they end up being less efficient and slower per-unit than Nvidia hardware. They don't need to pay for the margins, and they can run them 24/7 for their intended purpose. The previous-generation TPUs can't be reused or resold for other purposes though, and if/when AI blows over as a trend you probably can't easily start mining crypto or doing HPC calculations like an Nvidia cluster would.
philomath_mn 2y ago
If they are selling something that everyone wants and there aren't enough GPUs to around, how else do you suggest allocating those GPUs to the public except by increasing the price?
rodiger 2y ago
"Your margin is my opportunity" A ton of companies are dumping R&D into designing their own chips with seeming success. Nvidia will be in tough company quite soon, but for now demand greatly outstrips supply.
asadm 2y ago
AMD had a chance, they dropped the ball and still won't fix their ways (on devexp, drivers, etc). I think Nvidia deserves the success.
Ductapemaster 2y ago
So say they mark them down to some reasonable level — they are still supply constrained and now the line just gets longer. It doesn’t really fix anything. Pricing allows them to prioritize their customers, which seems reasonable.
- pie420 2y ago
  if the line gets long enough, it allows other companies to compete with Nvidia and creates a healthier, more competitive market, in the same way AMD has made intel a more honest, fairer, less price-gouging company. Competition is the entire thesis of american capitalism
  - talldayo 2y ago
    > Competition is the entire thesis of american capitalism Then why aren't people competing with Nvidia? Why is OpenCL on life support and unsupported on major operating systems? Why are we doing this song-and-dance routine refusing to adopt certain GPU APIs but embracing closed ones instead? I'd like to believe that a tipping point will be reached, but if not now then when? People have talked about upending Nvidia's GPGPU compute empire for years, but besides application-specific replacements and proofs-of-concept, we don't have a real CUDA-killer. Apple does not ship one, Google does not ship one, Microsoft does not ship one and AMD doesn't either. So... when? If we continue along the current path, I suspect Nvidia will continue to find markets where CUDA is demanded and OEMS will continue to chase them down with half-measure solutions. Unless OpenCL is revived or someone commits to a proprietary CUDA-like platform, I suspect we'll be spinning our wheels and digging ourselves deeper.
    - thorncorona 2y ago
      > Then why aren't people competing with Nvidia? Why is OpenCL on life support and unsupported on major operating systems? Why are we doing this song-and-dance routine refusing to adopt certain GPU APIs but embracing closed ones instead? Because Nvidia invested 20 years into its API platform, and this advantage is slowly getting realized.
dcgudeman 2y ago
Here is the definition of monopoly: "the exclusive possession or control of the supply of or trade in a commodity or service." There are other huge players: AMD and Intel. Additionally Google and Amazon are producing their own AI hardware. Now can you explain to me how Nvidia has a monopoly?
- Spivak 2y ago
  Because monopoly means "wields monopoly power and engages in anticompetitive behavior that harms consumers." Having competition or not, having large market share or not doesn't matter one bit. One of ways of wielding monopoly power is no longer being a price taker and being able to set prices at in practice to whatever you want.
  - dcgudeman 2y ago
    Your definition of monopoly contains a reference to "monopoly", so I am still not sure what your definition of monopoly is. It sounds like it is something like "big company that charges a price you think is high". Why do you think you know what the "right" price is. Isn't it a good thing the company delivering the most value is capturing the most profits? They will likely continue to deliver breakthroughs with the profits they are generating. Why would you want to kneecap a company generating so much value?
IncreasePosts 2y ago
If they sold them at 30% margins or whatever you think is reasonable, then scalpers would just pocket the 770%. Not sure how that is better for the ultimate purchasers or Nvidia.
talldatethrow 2y ago
Tech hardware companies are the only people where I'm not annoyed at their prices. Want a fence built for you? $3000 for 2 days work with 96 pieces of wood. Want a basic android phone with 4 gb of memory, 128 gb of storage, a camera, speakers, touch screen, wifi, cell tower connection, GPS, battery, charger and cable? $60 on sale at Walmart for a Moto G Power Stylus edition. (My current phone and that's what I paid a few months ago)
hi-v-rocknroll 2y ago
Monetize while the iron is hot. It's because Microsoft/OpenAI and Meta are throwing multiple $10B's at NVIDIA. This is unsustainable and NVIDIA's stock will contract over 3-6 years unless Microsoft and Meta can translate hype into essential, profitable products which isn't a sure thing™.

mrweasel 2y ago

Best description ever of what AI companies produce: "software systems that are trained on large amounts of data and can generate text". No further explanation of what Cohere might provide to it's customer of value, just the ability to generate text.

eamsen 2y ago
Yet, here we are, generating text. Some of which has value.
- andrei_says_ 2y ago
  But often it’s hard to know which.
DebtDeflation 2y ago
If we're going to waste electricity, I'd rather we waste it on AI than on mining Bitcoin. At least AI is providing some utility to its consumers and there's reason to believe it will provide even more in the future.
- kaliqt 2y ago
  Bitcoin provides massive utility to customers. The banking cartels + global government massive taxes are destroying livelihoods daily. The compute is "wasted" to pick winners and secure the system, but it's not actually wasted, it can be made more efficient by actually using the compute to do something, but even if that was not the case, it pays back its value 100x.
- ErikBjare 2y ago
  "waste it on AI" "at least AI is providing utility" Doesn't sound like a waste, does it? I'd go further and argue mining Bitcoin isn't either, but I digress.
itkovian_ 2y ago
I think there's a reasonable argument that cohere might not capture the value and it might all go to the big players. But I can't understand how anyone still has the view llms themselves are not valuable.

tuttyboy 2y ago

There are so many LLMs that they’re starting to feel cheap.

hi-v-rocknroll 2y ago
The shark will be completely jumped once McDonald's offers a Happy Meal prize that's a virtual pet powered by an LLM.
- djmips 2y ago
  That sounds cool.
  - powerbroker 2y ago
    Its even cooler if the pet has fleas that are organized as a cellular automata that plays the game of 'life' on the body of the pet.
    - djmips 2y ago
      another great idea!

metadat 2y ago

Also Oracle is in on this one: https://www.oracle.com/customers/cohere/ https://www.oracle.com/customers/cohere/

anothername12 2y ago

What's the best way to make money of all this LLM stuff before it implodes?