4 ms·

The interesting part of this "RAM crisis" is similar to other fields where a problem results multiple parties looking for alternative solutions. This yields fo

The interesting part of this "RAM crisis" is similar to other fields where a problem results multiple parties looking for alternative solutions. This yields for exciting ideas or workarounds that might result a post-crisis memory boom (hopefully) also for local machines. 1. Lowest, Apple is evaluating new Chinese manufacturer which means change of supply demand if indeed it has reasonable QA. (https://www.ft.com/content/f4ac5c92-03be-4499-b16a-017a7e9ee228 https://www.ft.com/content/f4ac5c92-03be-4499-b16a-017a7e9ee...) 2. Companies tries to workaround performance - suddenly single channel is 'ok' ? :) (https://www.gigabyte.com/press/news/2403 https://www.gigabyte.com/press/news/2403)

egorfine 3mo ago
> suddenly single channel is 'ok' Single channel RAM surely beats any disk-based swap.
- rock_artist 3mo ago
  of course, but I was under the impression the real shortage is RAM mostly.
  - varispeed 3mo ago
    If you don't have RAM you swap, so it is still better to at least ensure swap is a little bit faster.
tonyedgecombe 3mo ago
Necessity is the mother of invention.
dofm 3mo ago
Necessity is the mother of invention, after all. (One of the oldest abstract concepts in intellectual thought, I suspect.) There is a tight resource starvation/motivation loop — the demand put on RAM and SSD and GPUs by the largest frontier models is a direct motivation to make smaller LLMs. Like an evolutionary pressure making animals smaller and more food-efficient. These smaller models, once successful, are still likely to consume more RAM and SSD and GPUs than any other application short of high quality video processing itself (the smaller LLMs and higher end video processing seem to have about the same needs). But the resources would distribute through the market more traditionally, leading to less insane cycles. So it seems to me that the way out of the RAM/SSD price cycle crisis that manufacturers are in — where the price fluctuates between high and low due to supply constraints and then oversupply from new production capacity - is for them to fund research into smaller LLMs. They'll still sell essentially the same amount of product. Maybe more.
HumblyTossed 3mo ago
I would love it if we started designing software with hardware constraints in mind again.
- porksoda 3mo ago
  We do already, if it ooms at 32g I have to prompt again /s
- overfeed 3mo ago
  Consumer hardware costs are external. We are in the era of ignoring externalized costs, for example, Windows 11 hardware requirements, social media harm, and mainstreaming of gambling. Maybe the pendulum will swing back in the future, but today, line-going-up is the prime directive.
- thewhitetulip 3mo ago
  Since AI generated code is everywhere, most people don't read code anymore. Forget about hyper optimization

egorfine 3mo ago

> suddenly single channel is 'ok' Single channel RAM surely beats any disk-based swap.

rock_artist 3mo ago
of course, but I was under the impression the real shortage is RAM mostly.
- varispeed 3mo ago
  If you don't have RAM you swap, so it is still better to at least ensure swap is a little bit faster.

tonyedgecombe 3mo ago

Necessity is the mother of invention.

dofm 3mo ago

Necessity is the mother of invention, after all. (One of the oldest abstract concepts in intellectual thought, I suspect.) There is a tight resource starvation/motivation loop — the demand put on RAM and SSD and GPUs by the largest frontier models is a direct motivation to make smaller LLMs. Like an evolutionary pressure making animals smaller and more food-efficient. These smaller models, once successful, are still likely to consume more RAM and SSD and GPUs than any other application short of high quality video processing itself (the smaller LLMs and higher end video processing seem to have about the same needs). But the resources would distribute through the market more traditionally, leading to less insane cycles. So it seems to me that the way out of the RAM/SSD price cycle crisis that manufacturers are in — where the price fluctuates between high and low due to supply constraints and then oversupply from new production capacity - is for them to fund research into smaller LLMs. They'll still sell essentially the same amount of product. Maybe more.

HumblyTossed 3mo ago

I would love it if we started designing software with hardware constraints in mind again.

porksoda 3mo ago
We do already, if it ooms at 32g I have to prompt again /s
overfeed 3mo ago
Consumer hardware costs are external. We are in the era of ignoring externalized costs, for example, Windows 11 hardware requirements, social media harm, and mainstreaming of gambling. Maybe the pendulum will swing back in the future, but today, line-going-up is the prime directive.
thewhitetulip 3mo ago
Since AI generated code is everywhere, most people don't read code anymore. Forget about hyper optimization