4 ms·

Is there somebody not researching faster llm performance?

stefan_ 2y ago
Does Intel qualify? They seem busy making GPUs for midrange insignificant gaming builds, while all their AI offerings are hopeless.
- deergomoo 2y ago
  As someone who owns an insignificant midrange gaming build and has been irritated at the inflation of GPU prices with crypto and now genAI, I’m glad someone is.
- init2null 2y ago
  Not everyone is willing to pay for 4K, 120 Hz, and quality raytracing. There is a market for less than premium experiences. I wanted some marginal CUDA for Blender, so I got a 3060. This experience is anything but insignificant to me.
  - deleted 2y ago
    [deleted]
- rowanG077 2y ago
  How is it insignificant. Intel GPUs are the most exciting thing that has happened in GPUs the last 5-10 years imo.
  - n144q 2y ago
    Only for a small group of people -- i.e. users look for low-mid end gaming. Most people would say H100 is easily more exciting than all released Intel GPUs combined.
    - rowanG077 2y ago
      Most people who know about GPUs don't know even know what an H100 is. It's a data center GPU who almost no one will ever even touch, see or hear for their entire life, working on something that currently has lower impact then gaming. I'm not going to buy an intel GPU. But just to lay some pressure on the evil overlord that is Nvidia is very exciting.
- n144q 2y ago
  Virtually nobody was buying Intel GPUs: https://www.tomshardware.com/pc-components/gpus/discrete-gpu-sales-increase-as-intels-share-drops-to-0 https://www.tomshardware.com/pc-components/gpus/discrete-gpu... Looks like their latest GPU is selling out. Hope they actually move the market a little bit and iterate on this.
- yjftsjthsd-h 2y ago
  They're still trying. I mean, not succeeding, but they're making some sort of effort. (On which note... If anyone's figured out how to run ollama/llamafile/... on any of the apparently numerous libraries that supposedly accelerate AI on Intel hardware please share any hints because supposedly I've got the hardware but I can't make sense of any of the docs...)
CharlesW 2y ago
> Is there somebody not researching faster llm performance? The surprising bit is less about what they're working on and more about the collaboration itself, including the mutual and coordinated co-marketing/PR. Apple and Nvidia haven't had a business relationship in over a decade, ever since Apple stopped using Nvidia GPUs in Macs. This sort of re-engagement and subsequent promotion isn't something that "just happens", in my experience. It's reasonably possible that this could portend additional future outcomes, such as official support for Nvidia-based eGPUs, Apple's licensing of/support for CUDA, Mac Pros with Nvidia GPUs, etc.
- ksec 2y ago
  That is exactly what I was going to say. Nvidia has been a big No No inside Apple after they strained there relationship with MacBook Pro recall with Nvidia GPU causing overheating. Apple's heat dissipation were never good in the first place which makes the matter worst. So this isn't just about Faster LLM performance. This could possibly opens up a whole new world for Mac ecosystem with CUDA.
  - musicale 2y ago
    > Apple's heat dissipation were never good in the first place Apple has shipped many successful fanless systems - most recently the MacBook Air. Apple's metal laptop cases (starting with the titanium PowerBook G4 in 2001) look nice and also help with heat dissipation - though the case itself can heat up. Thankfully shutdowns or burns due to overheating were (and are) rare. Recently ThinkPads (for example) seem to have more thermal issues, but YMMV.
SllX 2y ago
Probably not, but Apple and NVIDIA collaborating on anything is news. They’re two companies that both want to be the ones wearing the Big Boy pants in any relationship so when they did anything together in the past, it hasn’t worked out for long.
weebull 2y ago
I'm not.

Does Intel qualify? They seem busy making GPUs for midrange insignificant gaming builds, while all their AI offerings are hopeless.

deergomoo 2y ago
As someone who owns an insignificant midrange gaming build and has been irritated at the inflation of GPU prices with crypto and now genAI, I’m glad someone is.
init2null 2y ago
Not everyone is willing to pay for 4K, 120 Hz, and quality raytracing. There is a market for less than premium experiences. I wanted some marginal CUDA for Blender, so I got a 3060. This experience is anything but insignificant to me.
- deleted 2y ago
  [deleted]
rowanG077 2y ago
How is it insignificant. Intel GPUs are the most exciting thing that has happened in GPUs the last 5-10 years imo.
- n144q 2y ago
  Only for a small group of people -- i.e. users look for low-mid end gaming. Most people would say H100 is easily more exciting than all released Intel GPUs combined.
  - rowanG077 2y ago
    Most people who know about GPUs don't know even know what an H100 is. It's a data center GPU who almost no one will ever even touch, see or hear for their entire life, working on something that currently has lower impact then gaming. I'm not going to buy an intel GPU. But just to lay some pressure on the evil overlord that is Nvidia is very exciting.
n144q 2y ago
Virtually nobody was buying Intel GPUs: https://www.tomshardware.com/pc-components/gpus/discrete-gpu-sales-increase-as-intels-share-drops-to-0 https://www.tomshardware.com/pc-components/gpus/discrete-gpu... Looks like their latest GPU is selling out. Hope they actually move the market a little bit and iterate on this.
yjftsjthsd-h 2y ago
They're still trying. I mean, not succeeding, but they're making some sort of effort. (On which note... If anyone's figured out how to run ollama/llamafile/... on any of the apparently numerous libraries that supposedly accelerate AI on Intel hardware please share any hints because supposedly I've got the hardware but I can't make sense of any of the docs...)

CharlesW 2y ago

> Is there somebody not researching faster llm performance? The surprising bit is less about what they're working on and more about the collaboration itself, including the mutual and coordinated co-marketing/PR. Apple and Nvidia haven't had a business relationship in over a decade, ever since Apple stopped using Nvidia GPUs in Macs. This sort of re-engagement and subsequent promotion isn't something that "just happens", in my experience. It's reasonably possible that this could portend additional future outcomes, such as official support for Nvidia-based eGPUs, Apple's licensing of/support for CUDA, Mac Pros with Nvidia GPUs, etc.

ksec 2y ago
That is exactly what I was going to say. Nvidia has been a big No No inside Apple after they strained there relationship with MacBook Pro recall with Nvidia GPU causing overheating. Apple's heat dissipation were never good in the first place which makes the matter worst. So this isn't just about Faster LLM performance. This could possibly opens up a whole new world for Mac ecosystem with CUDA.
- musicale 2y ago
  > Apple's heat dissipation were never good in the first place Apple has shipped many successful fanless systems - most recently the MacBook Air. Apple's metal laptop cases (starting with the titanium PowerBook G4 in 2001) look nice and also help with heat dissipation - though the case itself can heat up. Thankfully shutdowns or burns due to overheating were (and are) rare. Recently ThinkPads (for example) seem to have more thermal issues, but YMMV.

SllX 2y ago

Probably not, but Apple and NVIDIA collaborating on anything is news. They’re two companies that both want to be the ones wearing the Big Boy pants in any relationship so when they did anything together in the past, it hasn’t worked out for long.

weebull 2y ago

I'm not.