3 ms·

There isn't much incentive for a consumer to have 1TB of RAM in 2033 either. AI models that require a huge amount of memory will be done in the cloud. Will me

There isn't much incentive for a consumer to have 1TB of RAM in 2033 either. AI models that require a huge amount of memory will be done in the cloud. Will memory demand increase by 2033? Of course. Will demand reach 1TB of RAM for a consumer level computer/mobile device? No.

ketzo 4y ago
Yeah, but I think it might legitimately 5 or 10x, if running models on-device is as helpful as it sounds.
visarga 4y ago
> Will demand reach 1TB of RAM for a consumer level computer/mobile device? No. Speculative Sampling allows to run a smaller model locally and only check up with a larger model in the cloud from time to time, once every few tokens.
ikekkdcjkfke 4y ago
There is incentive to use RAM as planned obselence though. As can be seen when updating old devices, forcing apps to swap frequently in and out of disk
- _5uxp 4y ago
  I think nowadays Intel CPUs are more planned obselence that any other hardware, so behind.
dr_zoidberg 4y ago
Today the upper range of RAM in devices is in the hundred-GB. 10 years ago it was about 16GB or so, and making a (probably bad) interpolation I think it wouldn't be crazy to have the upper range in the TB mark in 10 years time. We could get there faster too, for specific use cases (compiling, rendering, etc). On the other hand, yes, everyday use (web, mail, video/media consumption) don't require today much more than 8 or 16GB of RAM. If we go a bit creative, a PC with Linux can run very smoothly on 4GB alone, and surely someone here can point to their one anecdote of a machine with 2 GB or even 1 GB sporting a nice desktop environment, or 128mb CLI-only machine. Edit: also memory has to improve its bandwidth and data transfer rates to keep up with faster processors, so it could also improve over 10 years time without much focus on storage capacity. Or maybe they focus on latency instead, or a mix of all three. Point is that it's not a single metric to improve.

ketzo 4y ago

Yeah, but I think it might legitimately 5 or 10x, if running models on-device is as helpful as it sounds.

visarga 4y ago

> Will demand reach 1TB of RAM for a consumer level computer/mobile device? No. Speculative Sampling allows to run a smaller model locally and only check up with a larger model in the cloud from time to time, once every few tokens.

ikekkdcjkfke 4y ago

There is incentive to use RAM as planned obselence though. As can be seen when updating old devices, forcing apps to swap frequently in and out of disk

_5uxp 4y ago
I think nowadays Intel CPUs are more planned obselence that any other hardware, so behind.

dr_zoidberg 4y ago

Today the upper range of RAM in devices is in the hundred-GB. 10 years ago it was about 16GB or so, and making a (probably bad) interpolation I think it wouldn't be crazy to have the upper range in the TB mark in 10 years time. We could get there faster too, for specific use cases (compiling, rendering, etc). On the other hand, yes, everyday use (web, mail, video/media consumption) don't require today much more than 8 or 16GB of RAM. If we go a bit creative, a PC with Linux can run very smoothly on 4GB alone, and surely someone here can point to their one anecdote of a machine with 2 GB or even 1 GB sporting a nice desktop environment, or 128mb CLI-only machine. Edit: also memory has to improve its bandwidth and data transfer rates to keep up with faster processors, so it could also improve over 10 years time without much focus on storage capacity. Or maybe they focus on latency instead, or a mix of all three. Point is that it's not a single metric to improve.