5 ms·
The next big thing that will drive demand for Nvidia chips is AI Search (https://openai.com/index/searchgpt-prototype/ https://openai.com/index/searchgpt-protot
by breadwinner 2y ago
The next big thing that will drive demand for Nvidia chips is AI Search (https://openai.com/index/searchgpt-prototype/ https://openai.com/index/searchgpt-prototype/). To avoid being obsoleted Google and Microsoft Bing have to spend big on Nvidia hardware, and when Nvidia releases newer chips with lower power consumption becomes available OpenAI, Google and Microsoft will be forced to lap it up.
- wmf 2y agoGoogle has TPUs and Microsoft is buying AMD GPUs so I wouldn't say they have to buy Nvidia.
- metadat 2y agoAMD GPUs are still mainly used only for inference. Anyone who wants to use CUDA continues to be forced to buy Nvidia hardware.
- dleink 2y agoHow far along are efforts to run CUDA on other hardware?
- qeternity 2y agoInference is the main cost involved in deploying at scale…
- bravetraveler 2y agoGPUs are only part of the stack, too. Be careful building hegemony. Don't forget the network.
- JonChesterfield 2y agoA few years ago everything had to be CUDA. Opencl and so forth were unacceptable. Now people say CUDA is better optimised than ROCm. Fewer claims that ROCm never works, more that it leaves performance on the table. You've just offered that people are using AMDGPU in production for inference. I think some are using it for training. That software moat is looking rather empty to me. And good riddance, the cuda programming model is nasty. The MI300A is gorgeous. Massive APU thing. Single block of fast memory that x64 threads and gpu kernels have atomic RMW access to. It is the evolution beyond the separate accelerator concept and the whole language patterns around "offloading" kernels is kind of nonsense on it. The gpu and x64 cores are peers. Viable alternatives to cuda are coming online at the same time as the hardware moves beyond it. It's a liability, not a moat.
- deleted 2y ago[deleted]
- djtango 2y agoGoogle has been sitting on AI search for a long time - I recall people saying it simply wasn't cost effective to do AI powered search because the way AI costs scale. Has something changed on this front or is it just an arms race?
- shaism 2y agoI am working in search (not Google). My take: Today, using AI for search-related problems is still not cost-effective for most use cases. That being said, the landscape is evolving quickly. First, in some areas, an individual search creates more value than in others. An individual consumer doing a Google Search is totally different from a lawyer searching for reference material. Areas where the individual search creates more value can already benefit from AI today. Second, LLMs become exponentially cheaper, driven by more cost-effective computing but also more cost-effective models. Look at the pricing of GPT4o-mini vs GPT4 (the original). The models are comparable in performance for many search-related problems, but the price has decreased by 200x in 1.5 years ($0.15 vs $30 per 1M token). If that price trend continues, more and more search use cases will benefit from AI.
- delfinom 2y agoSearch is going to be dead because the majority of results will soon be AI generated drivel boosted through SEO.
- JonChesterfield 2y agoSearch should survive. The information is still out there and there is value in finding it. Google is on thin ice though.
- nojvek 2y agoPerplexity is solid. I don’t use Google as my primary search engine anymore.