3 ms·

If you're running a massive model for logic you're probably better off not putting it in the robot. And it'll be a long time before there's enough robots to ma

If you're running a massive model for logic you're probably better off not putting it in the robot. And it'll be a long time before there's enough robots to make up a significant share of usage. More basic movement control doesn't need loads of ram as far as I know.

b112 4mo ago
The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops. The larger the context window, the better with models. Having a few TB of RAM would be exceptionally helpful. All this just made me realise something however. Having your robot dormant and charging, is a bit of a waste. You could have robots dormant, but its compute in use to act as a compute node. If the distribution of robots is similar world-wide, we'd need a fraction of the datacenters we have now. Using such nodes for training purposes would be beyond advantageous. And the company which can slice up the work and having training done in batches would get the big bucks. And actually, with consumer facing products soon all laden with extra ram and gpu for local compute, that applies there too. Imagine leasing out idle time on your desktop or even laptop for cash. There may be a market here, especially with the cost of new datacenters. Any company able to securely package compute without risking data safety is going to make a mint. Anyone have any ideas?
- Dylan16807 4mo ago
  > The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops. I don't think you understood my post. The equivalent of self-driving is the movement control I was talking about. Self-driving cars don't have high level logic, except for route planning. Which often is offloaded to the cloud. An extra 30 milliseconds on understanding your speech is nothing. > Imagine leasing out idle time on your desktop or even laptop for cash. https://vast.ai/ https://vast.ai/
  - b112 4mo ago
    You're missing the point. The issue is "how is the thing used" not trying to make identical break points in differing tech. A self driving car cannot have object detection, collision avoidance, human detection offloaded to the cloud. Ever. At all. A network drop can't mean it smashes into things. Or stops unsafely. The same is true with an android. Imagine it turning on an frying pan, cooking dinner, and then going offline part way through. Or turning on a tap to wash something, and going offline while the sink overflows and destroys the house. There are myriad of such scenarios, but local compute is absolutely, 100% necessary. Anyone betting the farm on putting network controlled devices into homes for any serious task is going to lose their shirt. Local compute is an absolute requirement, and a few TB of RAM and local compute will be nothing over the scope of this discussion (a few years minimum, just to build and kick off all these new fabs). By the time these fabs are online, expect most smart phones to have 1TB of RAM and significant llm capable compute (gpu or other custom silicon). I would be astonished if flagship model phones in 2030 weren't sold with 1TB RAM. Note I'm saying flagship, there will be of course economy models as always. Certainly laptops will be sold in multi-TB RAM configs.
    - Dylan16807 4mo ago
      I'm not missing the point. I'm saying that some amount of local compute is necessary but the really big stuff can be remote. You don't need terabytes to turn things back off. Not that I want to trust "not setting the house on fire" and "not flooding the house" to this kind of model in the first place... > I would be astonished if flagship model phones in 2030 weren't sold with 1TB RAM. I'll be astonished if they have 50GB. Have you looked at RAM size/price trends? I'm not even talking about the last year, just the pattern before that. We're not in the 80s and 90s anymore. The most recent price lows were roughly $3.50/GB in 2013, $2.50/GB in 2016, and $1.50/GB in 2023. If we're lucky the cheapest stuff will hit $1/GB in a few years, and the kind that would actually fit on a phone motherboard would be significantly more than that. Samsung hit 16GB on their top model in 2020 and it's either been 16GB or 12GB ever since. Apple only went up to 12GB in the last year. Google offers 16GB. A couple niche offerings have 24GB. Why would these RAM numbers even double during the next four years?

b112 4mo ago

The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops. The larger the context window, the better with models. Having a few TB of RAM would be exceptionally helpful. All this just made me realise something however. Having your robot dormant and charging, is a bit of a waste. You could have robots dormant, but its compute in use to act as a compute node. If the distribution of robots is similar world-wide, we'd need a fraction of the datacenters we have now. Using such nodes for training purposes would be beyond advantageous. And the company which can slice up the work and having training done in batches would get the big bucks. And actually, with consumer facing products soon all laden with extra ram and gpu for local compute, that applies there too. Imagine leasing out idle time on your desktop or even laptop for cash. There may be a market here, especially with the cost of new datacenters. Any company able to securely package compute without risking data safety is going to make a mint. Anyone have any ideas?

Dylan16807 4mo ago
> The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops. I don't think you understood my post. The equivalent of self-driving is the movement control I was talking about. Self-driving cars don't have high level logic, except for route planning. Which often is offloaded to the cloud. An extra 30 milliseconds on understanding your speech is nothing. > Imagine leasing out idle time on your desktop or even laptop for cash. https://vast.ai/ https://vast.ai/
- b112 4mo ago
  You're missing the point. The issue is "how is the thing used" not trying to make identical break points in differing tech. A self driving car cannot have object detection, collision avoidance, human detection offloaded to the cloud. Ever. At all. A network drop can't mean it smashes into things. Or stops unsafely. The same is true with an android. Imagine it turning on an frying pan, cooking dinner, and then going offline part way through. Or turning on a tap to wash something, and going offline while the sink overflows and destroys the house. There are myriad of such scenarios, but local compute is absolutely, 100% necessary. Anyone betting the farm on putting network controlled devices into homes for any serious task is going to lose their shirt. Local compute is an absolute requirement, and a few TB of RAM and local compute will be nothing over the scope of this discussion (a few years minimum, just to build and kick off all these new fabs). By the time these fabs are online, expect most smart phones to have 1TB of RAM and significant llm capable compute (gpu or other custom silicon). I would be astonished if flagship model phones in 2030 weren't sold with 1TB RAM. Note I'm saying flagship, there will be of course economy models as always. Certainly laptops will be sold in multi-TB RAM configs.
  - Dylan16807 4mo ago
    I'm not missing the point. I'm saying that some amount of local compute is necessary but the really big stuff can be remote. You don't need terabytes to turn things back off. Not that I want to trust "not setting the house on fire" and "not flooding the house" to this kind of model in the first place... > I would be astonished if flagship model phones in 2030 weren't sold with 1TB RAM. I'll be astonished if they have 50GB. Have you looked at RAM size/price trends? I'm not even talking about the last year, just the pattern before that. We're not in the 80s and 90s anymore. The most recent price lows were roughly $3.50/GB in 2013, $2.50/GB in 2016, and $1.50/GB in 2023. If we're lucky the cheapest stuff will hit $1/GB in a few years, and the kind that would actually fit on a phone motherboard would be significantly more than that. Samsung hit 16GB on their top model in 2020 and it's either been 16GB or 12GB ever since. Apple only went up to 12GB in the last year. Google offers 16GB. A couple niche offerings have 24GB. Why would these RAM numbers even double during the next four years?