3 ms·

If compute is gonna play the role of electricity in coming decades, then having a compute wall similar to Tesla powerwall is a necessity.

ta988 2y ago
Only if improvements in speed and energy savings slow down
- CooCooCaCha 2y ago
  And if models don't get any larger, which they will
CooCooCaCha 2y ago
Powerwall and electric car in the garage, compute wall in the closet, 3d printer and other building tools in the manufacturing room, hydroponics setup in the indoor farm room, and AI assistant to help manage it all. The home becomes a mixed living area and factory.
- crowcroft 2y ago
  The vision of this sounds so cool, but man, for a lot of use cases at the moment most 'smart home' stuff is still complicated and temperamental. How do we get from here to there, cause I want to get there so bad.
  - CooCooCaCha 2y ago
    Tech and economic development can help a bit, like the new Bamboo 3d printer makes 3d printing a lot more "idiot proof". However, I think we need AI beyond current LLMs to really take us there. I'm not saying LLMs can't get us there, we don't know, just beyond what we have. We need AI that we can trust with real tasks IRL.
rbanffy 2y ago
Powerwall makes sense because you can’t generate energy at any time and, therefore, you store it. Computers are not like that - you don’t “store” computations for when you need them - you either use capacity or you don’t. That makes it practical to centralise computing and only pay for what you use.
- jfoutz 2y ago
  I was going to make a pedantic argument/joke about memoization. It is kind of an interesting thought though. A big wall of SSD is a fabulous amount of storage. and maybe a clever read only architecture, would be cheaper than SSD. and a clever data structure for shared high order bits, maybe, maybe there is potential for some device to look up matrix multiply results, or close approximations that could be cheaply refined. Right now, I doubt it. But big static cache it is a kind of interesting idea to kick around Saturday afternoon.
  - rbanffy 2y ago
    > maybe there is potential for some device to look up matrix multiply results, or close approximations that could be cheaply refined. Shard that across the planet and you'd have a global cache for calculations. Or a lookup for every possible AI prompt and its results.
  - marcosdumay 2y ago
    > I was going to make a pedantic argument/joke about memoization. You are reading the GP the wrong way around. You store partial results exactly because you can't store computation. Computation is perishable¹, you either use it or lose it. And one way to use it is to create partial results you can save for later. 1 - Well, partially so. Hardware utilization is perishable, but computation also consumes inputs (mostly energy) that aren't. How much it perishable depends on the ratio of those two costs, and your mobile phone has a completely different outlook from a supercomputer.
- passion__desire 2y ago
  I didn't mean ComputeWall in the sense of storage of compute but in a sense of client server model where client (ComputeWall, maybe made of DGX-2) could continue to function independently in cases of natural calamity or other issues.

ta988 2y ago

Only if improvements in speed and energy savings slow down

CooCooCaCha 2y ago
And if models don't get any larger, which they will

CooCooCaCha 2y ago

Powerwall and electric car in the garage, compute wall in the closet, 3d printer and other building tools in the manufacturing room, hydroponics setup in the indoor farm room, and AI assistant to help manage it all. The home becomes a mixed living area and factory.

crowcroft 2y ago
The vision of this sounds so cool, but man, for a lot of use cases at the moment most 'smart home' stuff is still complicated and temperamental. How do we get from here to there, cause I want to get there so bad.
- CooCooCaCha 2y ago
  Tech and economic development can help a bit, like the new Bamboo 3d printer makes 3d printing a lot more "idiot proof". However, I think we need AI beyond current LLMs to really take us there. I'm not saying LLMs can't get us there, we don't know, just beyond what we have. We need AI that we can trust with real tasks IRL.

rbanffy 2y ago

Powerwall makes sense because you can’t generate energy at any time and, therefore, you store it. Computers are not like that - you don’t “store” computations for when you need them - you either use capacity or you don’t. That makes it practical to centralise computing and only pay for what you use.

jfoutz 2y ago
I was going to make a pedantic argument/joke about memoization. It is kind of an interesting thought though. A big wall of SSD is a fabulous amount of storage. and maybe a clever read only architecture, would be cheaper than SSD. and a clever data structure for shared high order bits, maybe, maybe there is potential for some device to look up matrix multiply results, or close approximations that could be cheaply refined. Right now, I doubt it. But big static cache it is a kind of interesting idea to kick around Saturday afternoon.
- rbanffy 2y ago
  > maybe there is potential for some device to look up matrix multiply results, or close approximations that could be cheaply refined. Shard that across the planet and you'd have a global cache for calculations. Or a lookup for every possible AI prompt and its results.
- marcosdumay 2y ago
  > I was going to make a pedantic argument/joke about memoization. You are reading the GP the wrong way around. You store partial results exactly because you can't store computation. Computation is perishable¹, you either use it or lose it. And one way to use it is to create partial results you can save for later. 1 - Well, partially so. Hardware utilization is perishable, but computation also consumes inputs (mostly energy) that aren't. How much it perishable depends on the ratio of those two costs, and your mobile phone has a completely different outlook from a supercomputer.
passion__desire 2y ago
I didn't mean ComputeWall in the sense of storage of compute but in a sense of client server model where client (ComputeWall, maybe made of DGX-2) could continue to function independently in cases of natural calamity or other issues.