5 ms·

Hmmm, 2-3 high end GPUs implies it's likely not very far off from mainstream. Maybe runnable on the average device within 10-20 years... perhaps even sooner if

Hmmm, 2-3 high end GPUs implies it's likely not very far off from mainstream. Maybe runnable on the average device within 10-20 years... perhaps even sooner if the model/software can be optimized?

lolspace 4y ago
20 years?
- adam_arthur 4y ago
  In 20 years I'll still be alive and enjoying myself. Does 20 years seem long to you?
  - krsdcbl 4y ago
    it seems a bit long for achieving just running your own AI models locally, given it seems to be largely a question of vram and that you already _could_ do it today with a handful of graphic cards. 20y ago we had the GeForce Ti4400, current graphics cards now come with 100x the vram and 50-60x the bandwidth.
  - 5e92cb50239222b 4y ago
    That's one hell of an assumption. Many of my Russian friends were absolutely certain they'll be alive and well for at least the next 20 years not that long ago.
    - adam_arthur 4y ago
      If I'm dead then being able to run an AI locally doesn't matter anyway
    - simne 4y ago
      Sure, they will be alive if not come to Ukraine.
dahdum 4y ago
> 2-3 high end GPUs implies it's likely not very far off from mainstream Looks like FLOP/s per $ are doubling every ~3 years for high end cards, and 10x in ~10 years. So probably not that far off for desktop users. https://www.lesswrong.com/posts/c6KFvQcZggQKZzxr9/trends-in-gpu-price-performance https://www.lesswrong.com/posts/c6KFvQcZggQKZzxr9/trends-in-...
- visarga 4y ago
  FLOPs remain unused when the bottleneck is bandwidth.
bemmu 4y ago
I’d be surprised if there weren’t any algo breakthroughs before that to make these several times faster (10x?). (such as are all of the weights really needed all of the time, or could you load different ones based on recent context?) If in 10 years on top of that you’d have that 10x faster hardware as well, you might be running GPT-3s as just a subcomponent of games/apps.
- ETH_start 4y ago
  ChatGPT prompt: What's the difference between a component and subcomponent? A component is a self-contained unit of functionality, often in the form of a class or object, that represents a piece of an overall system or application. A subcomponent, on the other hand, is a smaller, simpler component that is used as part of a larger component. A subcomponent is typically used to break down a larger component into smaller, more manageable pieces, and is often used to create reusable building blocks for larger components. In other words, a subcomponent is a component that is used by another component.

Hmmm, 2-3 high end GPUs implies it's likely not very far off from mainstream. Maybe runnable on the average device within 10-20 years... perhaps even sooner if the model/software can be optimized?

lolspace 4y ago

20 years?

adam_arthur 4y ago
In 20 years I'll still be alive and enjoying myself. Does 20 years seem long to you?
- krsdcbl 4y ago
  it seems a bit long for achieving just running your own AI models locally, given it seems to be largely a question of vram and that you already _could_ do it today with a handful of graphic cards. 20y ago we had the GeForce Ti4400, current graphics cards now come with 100x the vram and 50-60x the bandwidth.
- 5e92cb50239222b 4y ago
  That's one hell of an assumption. Many of my Russian friends were absolutely certain they'll be alive and well for at least the next 20 years not that long ago.
  - adam_arthur 4y ago
    If I'm dead then being able to run an AI locally doesn't matter anyway
  - simne 4y ago
    Sure, they will be alive if not come to Ukraine.

dahdum 4y ago

> 2-3 high end GPUs implies it's likely not very far off from mainstream Looks like FLOP/s per $ are doubling every ~3 years for high end cards, and 10x in ~10 years. So probably not that far off for desktop users. https://www.lesswrong.com/posts/c6KFvQcZggQKZzxr9/trends-in-gpu-price-performance https://www.lesswrong.com/posts/c6KFvQcZggQKZzxr9/trends-in-...

visarga 4y ago
FLOPs remain unused when the bottleneck is bandwidth.

bemmu 4y ago

I’d be surprised if there weren’t any algo breakthroughs before that to make these several times faster (10x?). (such as are all of the weights really needed all of the time, or could you load different ones based on recent context?) If in 10 years on top of that you’d have that 10x faster hardware as well, you might be running GPT-3s as just a subcomponent of games/apps.

ETH_start 4y ago
ChatGPT prompt: What's the difference between a component and subcomponent? A component is a self-contained unit of functionality, often in the form of a class or object, that represents a piece of an overall system or application. A subcomponent, on the other hand, is a smaller, simpler component that is used as part of a larger component. A subcomponent is typically used to break down a larger component into smaller, more manageable pieces, and is often used to create reusable building blocks for larger components. In other words, a subcomponent is a component that is used by another component.