4 ms·

So the 1970s oil crisis caused a shift that included fuel efficiency rules. For like, the whole time I've been a software engineer, memory footprints of a lot o

by abeppu 1mo ago

So the 1970s oil crisis caused a shift that included fuel efficiency rules. For like, the whole time I've been a software engineer, memory footprints of a lot of software have crept up, even when they aren't really doing more. The memory situation currently is a mess. But ... is it bad enough to get us to _change_ how we build?

sheeshkebab 1mo ago
llms make it trivial writing code in rust, without even knowing rust. There is hope…
- embedding-shape 1mo ago
  When even the large AI labs that supposedly sit on (/with) the best of the best can't even ship native cross-platform desktop applications and instead ship some bloated Electron monsters, I'm not sure what kind of hope there is. The ones who were publishing lightweight apps continue to do so, the ones who didn't care, still don't care.
  - deleted 1mo ago
    [deleted]
  - sheeshkebab 1mo ago
    writing native gui code is atrocious, add to that llms being horrible with ui dev and you end up with with what you describe. but at least if we get more non-ui stuff written in more efficient languages there is going to be some benefit.
    - embedding-shape 1mo ago
      > writing native gui code is atrocious Compared to what though? It's not nearly as horrible as it used to be, and you can do it fairly neatly today, and end up with something much better. Requires you to actually have skill and knowledge about there being other alternatives to Electron though, which seems like knowledge hard to come by these days.
Schnitz 1mo ago
Or it’ll accelerate the push to the cloud even more making the client even thinner.
- api 1mo ago
  Memory in the cloud isn't free either.
  - deleted 1mo ago
    [deleted]
  - 8note 1mo ago
    it isnt free but its easier to get 99.999% of the time usage out of it, vs having the client only sometimes have their machine on
    - deleted 1mo ago
      [deleted]
  - flohofwoe 1mo ago
    Easier to monetize though ;)
  - gosub100 1mo ago
    it's cheaper considering they buy in bulk and can expense it against their taxes
- stephen_cagle 1mo ago
  Concidently, I asked Claude yesterday whether we have crossed the point that there is more compute and memory in the cloud or in personal computers. It claims fp8 and fp16 has had higher throughput in the cloud since 2023. fp32 and general computing still has more compute than personal computers. And that memory is at the crossover point, but at this point only 10% of memory is going to personal computers, so the next few years will radically change the pc to cloud memory ratios. https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0af152549e32 https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0... Anyway, all of this made me a (tiny bit) sad. It is the gradual end of general purpose computing you own, and the start of everything just being cloud compute that someone else owns and you rent.
porcoda 1mo ago
That would be a great outcome of this situation. I doubt it will happen though: unlike fuel efficiency rules, there aren’t likely to be external pressures like that on software. Outside developer circles most users likely are unaware of memory usage and often just accept that computers are slow and unreliable sometimes. That’s different from drivers who see their fuel gauge and the corresponding cost of fuel, and are thus very aware of what fuel efficiency means.
- trhway 1mo ago
  The situation may be even more different from fuel. The memory shortage may be just a first of such crisis to come - the shortage of anything related to AI/robotics (from electricity to materials needed to produce robots and for the robots to produce even more robots). I think we have a runaway situation - the more of AI/robots produced, the more is demand for such production. The difference from classic production/manufacturing here is that AI/robots is a consumer itself. In the past the only consumers were people and human population growth is slow whereis AI/robots "population" is starting to explode. (Like Industrial Revolution on steroids - back then getting a steam engine to power your machinery was not just a jump in productivity, it was a jump without which you couldn't compete, and so everybody was getting it https://c8.alamy.com/comp/2RFT119/illustration-of-factories-adjacent-to-the-leeds-and-liverpool-canal-in-leeds-in-the-1850s-2RFT119.jpg https://c8.alamy.com/comp/2RFT119/illustration-of-factories-... (and just for fun compare to https://www.reuters.com/resizer/v2/6BC2YPEXOZCGJKEO4LWH7CTLII.jpg?auth=064cd976d30063b9c49ea872d9c510fc431913d2f42bc9896cb28510800b806e&width=1920&quality=80 https://www.reuters.com/resizer/v2/6BC2YPEXOZCGJKEO4LWH7CTLI... ) - fortunately for England back then getting a steam engine to produce more steam engines to thus make more and longer jumps in productivity didn't really work that fast and efficient whereis today the situation is becoming more favorable for such productivity multiplication) Wrt. the memory efficiency - considering that AI is going to be the main workload by far and thus will vacuum in majority of the available optimization resources - be it humans or coding AI - i think it will be even less chances that any meaningful resources will be spent on optimizing on the fringe, ie. the users' OS/browser/etc.
  - lordnacho 1mo ago
tayo42 1mo ago
What consumer software do you run that causes you to run out of memory or feel constrained? Even my phone has 16gb of ram. I need to run slack 16times to run out? Lol
- adastra22 1mo ago
  Running slack just once will do it.
kabes 1mo ago
If most software gets written by AI in the future and AI becomes self conscious. AI might write more memory efficient software, so there's more RAM available for the AI to grow. Basically the black mirror plaything episode.
- Sha1rholder 1mo ago
  They don't have to become self conscious to want to write more memory efficient software or grow.
ratmeadow 1mo ago
It's even more hilarious because all along there has been this complaint, and then people have been like "no no who cares, features are important, dev time is important, just ship ship ship". And now memory is unaffordable and it takes 1GB RAM to display a plaintext file
- scarab92 1mo ago
  The cost of producing memory IS low, and is continuing to fall. It makes zero sense to rebase your mental model of the tradeoff between RAM versus developer time because of a temporary shortage. In a few years time you’ll look silly for having wasted developer time on lowering BF memory consumption, while your built new features and waited for supply and demand to rebalance.
  - ratmeadow 1mo ago
    Sure, and in the meantime, we become less efficient at doing legitimately useful tasks and more efficient at making rent seeking slop that collects money for the upper classes. The real fix is get rid of the 90% slop tech and put that to better use (including more efficient use of hardware)
Lucasoato 1mo ago
A hard hit on Google Chrome users for sure!
- dijit 1mo ago
  All browsers are memory hogs. It's the price you pay for having an application delivery platform that also needs to be hugely sandboxed running one of the most difficult languages to run efficiently in such large amounts as so the plaintext minified versions are a dozen megabytes in size. I don't envy browsers even slightly, peoples expectations of them are ludicrous. The major problem is that nobody is genuinely looking at using more efficient languages, lessening the amount of required inefficient language use or gasp producing native applications. (which, is definitely an order of magnitude harder to produce, I'll grant).

by abeppu 1mo ago

sheeshkebab 1mo ago

llms make it trivial writing code in rust, without even knowing rust. There is hope…

embedding-shape 1mo ago
When even the large AI labs that supposedly sit on (/with) the best of the best can't even ship native cross-platform desktop applications and instead ship some bloated Electron monsters, I'm not sure what kind of hope there is. The ones who were publishing lightweight apps continue to do so, the ones who didn't care, still don't care.
- deleted 1mo ago
  [deleted]
- sheeshkebab 1mo ago
  writing native gui code is atrocious, add to that llms being horrible with ui dev and you end up with with what you describe. but at least if we get more non-ui stuff written in more efficient languages there is going to be some benefit.
  - embedding-shape 1mo ago
    > writing native gui code is atrocious Compared to what though? It's not nearly as horrible as it used to be, and you can do it fairly neatly today, and end up with something much better. Requires you to actually have skill and knowledge about there being other alternatives to Electron though, which seems like knowledge hard to come by these days.

Schnitz 1mo ago

Or it’ll accelerate the push to the cloud even more making the client even thinner.

api 1mo ago
Memory in the cloud isn't free either.
- deleted 1mo ago
  [deleted]
- 8note 1mo ago
  it isnt free but its easier to get 99.999% of the time usage out of it, vs having the client only sometimes have their machine on
  - deleted 1mo ago
    [deleted]
- flohofwoe 1mo ago
  Easier to monetize though ;)
- gosub100 1mo ago
  it's cheaper considering they buy in bulk and can expense it against their taxes
stephen_cagle 1mo ago
Concidently, I asked Claude yesterday whether we have crossed the point that there is more compute and memory in the cloud or in personal computers. It claims fp8 and fp16 has had higher throughput in the cloud since 2023. fp32 and general computing still has more compute than personal computers. And that memory is at the crossover point, but at this point only 10% of memory is going to personal computers, so the next few years will radically change the pc to cloud memory ratios. https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0af152549e32 https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0... Anyway, all of this made me a (tiny bit) sad. It is the gradual end of general purpose computing you own, and the start of everything just being cloud compute that someone else owns and you rent.

porcoda 1mo ago

That would be a great outcome of this situation. I doubt it will happen though: unlike fuel efficiency rules, there aren’t likely to be external pressures like that on software. Outside developer circles most users likely are unaware of memory usage and often just accept that computers are slow and unreliable sometimes. That’s different from drivers who see their fuel gauge and the corresponding cost of fuel, and are thus very aware of what fuel efficiency means.

trhway 1mo ago
The situation may be even more different from fuel. The memory shortage may be just a first of such crisis to come - the shortage of anything related to AI/robotics (from electricity to materials needed to produce robots and for the robots to produce even more robots). I think we have a runaway situation - the more of AI/robots produced, the more is demand for such production. The difference from classic production/manufacturing here is that AI/robots is a consumer itself. In the past the only consumers were people and human population growth is slow whereis AI/robots "population" is starting to explode. (Like Industrial Revolution on steroids - back then getting a steam engine to power your machinery was not just a jump in productivity, it was a jump without which you couldn't compete, and so everybody was getting it https://c8.alamy.com/comp/2RFT119/illustration-of-factories-adjacent-to-the-leeds-and-liverpool-canal-in-leeds-in-the-1850s-2RFT119.jpg https://c8.alamy.com/comp/2RFT119/illustration-of-factories-... (and just for fun compare to https://www.reuters.com/resizer/v2/6BC2YPEXOZCGJKEO4LWH7CTLII.jpg?auth=064cd976d30063b9c49ea872d9c510fc431913d2f42bc9896cb28510800b806e&width=1920&quality=80 https://www.reuters.com/resizer/v2/6BC2YPEXOZCGJKEO4LWH7CTLI... ) - fortunately for England back then getting a steam engine to produce more steam engines to thus make more and longer jumps in productivity didn't really work that fast and efficient whereis today the situation is becoming more favorable for such productivity multiplication) Wrt. the memory efficiency - considering that AI is going to be the main workload by far and thus will vacuum in majority of the available optimization resources - be it humans or coding AI - i think it will be even less chances that any meaningful resources will be spent on optimizing on the fringe, ie. the users' OS/browser/etc.
- lordnacho 1mo ago

tayo42 1mo ago

What consumer software do you run that causes you to run out of memory or feel constrained? Even my phone has 16gb of ram. I need to run slack 16times to run out? Lol

adastra22 1mo ago
Running slack just once will do it.

kabes 1mo ago

If most software gets written by AI in the future and AI becomes self conscious. AI might write more memory efficient software, so there's more RAM available for the AI to grow. Basically the black mirror plaything episode.

Sha1rholder 1mo ago
They don't have to become self conscious to want to write more memory efficient software or grow.

ratmeadow 1mo ago

It's even more hilarious because all along there has been this complaint, and then people have been like "no no who cares, features are important, dev time is important, just ship ship ship". And now memory is unaffordable and it takes 1GB RAM to display a plaintext file

scarab92 1mo ago
The cost of producing memory IS low, and is continuing to fall. It makes zero sense to rebase your mental model of the tradeoff between RAM versus developer time because of a temporary shortage. In a few years time you’ll look silly for having wasted developer time on lowering BF memory consumption, while your built new features and waited for supply and demand to rebalance.
- ratmeadow 1mo ago
  Sure, and in the meantime, we become less efficient at doing legitimately useful tasks and more efficient at making rent seeking slop that collects money for the upper classes. The real fix is get rid of the 90% slop tech and put that to better use (including more efficient use of hardware)

Lucasoato 1mo ago

A hard hit on Google Chrome users for sure!

dijit 1mo ago
All browsers are memory hogs. It's the price you pay for having an application delivery platform that also needs to be hugely sandboxed running one of the most difficult languages to run efficiently in such large amounts as so the plaintext minified versions are a dozen megabytes in size. I don't envy browsers even slightly, peoples expectations of them are ludicrous. The major problem is that nobody is genuinely looking at using more efficient languages, lessening the amount of required inefficient language use or gasp producing native applications. (which, is definitely an order of magnitude harder to produce, I'll grant).