3 ms·

Concidently, I asked Claude yesterday whether we have crossed the point that there is more compute and memory in the cloud or in personal computers. It claims

Concidently, I asked Claude yesterday whether we have crossed the point that there is more compute and memory in the cloud or in personal computers. It claims fp8 and fp16 has had higher throughput in the cloud since 2023. fp32 and general computing still has more compute than personal computers. And that memory is at the crossover point, but at this point only 10% of memory is going to personal computers, so the next few years will radically change the pc to cloud memory ratios. https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0af152549e32 https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0... Anyway, all of this made me a (tiny bit) sad. It is the gradual end of general purpose computing you own, and the start of everything just being cloud compute that someone else owns and you rent.

bigstrat2003 1mo ago
Nobody cares what Claude said. Why not tell us what you think?
api 1mo ago
I agree with the sentiment, but there is a devils' advocate argument: Most compute loads are bursty. Most PCs and phones sit idle almost all the time. It's an inefficient use of capital. In the cloud you can be idle and then suddenly command 1000X more power than any affordable PC could ever have... for 10 seconds... then be idle again. Those kinds of work loads aren't possible to realize on a PC without massively overbuying hardware. I'd like to see more research on secure private cloud computing. Homomorphic encryption has gotten better but is still very slow, but there's other approaches that can help quite a bit but don't require much of a performance hit. Look into secure multiparty computation, reversible anonymization, etc. I don't think personal computing will die, but I think it'll go back to being a hobbyist and pro thing like it was from its birth in the 1970s up until the middle 1990s. In the 1980s the average person did not own a computer, or if they did it was just to play some games. The average person was not loading BASIC programs off bootleg tapes and calling BBSes. BTW I've been thinking a lot lately about how AI could save personal computing by allowing us to create an off-ramp from the present rats nest of complexity to something saner and simpler. Just thoughts right now but it's interesting.
- wizzwizz4 1mo ago
  Generative AI is definitely creating political will for simpler, more purposely-designed computer systems, but it's certainly not going to directly create help them in any meaningful way. AI-generated codebases are the posterchild of rat's nests of complexity.
  - api 1mo ago
    I'd say the code I see coming out of leading models today is okay. Not great, but not terrible. It's also easy to get LLMs to code in more efficient languages, and you can also goad them via prompts and CLAUDE.md type stuff into prioritizing performance and simplicity. It's sort of like what I say about self driving cars. Are they better than good human drivers? Not yet. Are they better than bad, drunk, or texting drivers? Yes. LLMs are already better than bad coders and honestly most coders are bad coders.
    - wizzwizz4 1mo ago
      In a world where we no longer have the rat's nest of complexity, we pretty much no longer need to produce code. All the software we will ever need for the next century could fit on books in one, single-floor library, if we exclude art projects like video games. An operating system with full driver support for every I/O peripheral (if those devices use standard protocols, allowing for a https://xkcd.com/927/ https://xkcd.com/927/ constant of 5), and support for every standard document format (again, assuming 5 bitmap image formats, 5 vector image formats, 5 kinds of print-ready document, 5 kinds of hypertext document…), could easily fit in 2GiB, and that would be done: everything anyone could possibly need out of a computer, apart from art, mathematics and science, for… I don't even know how long. We're still using the Latin alphabet, more or less unchanged since the late 15th century. Working technology doesn't need to change. Being able to automatically generate large quantities of code that is worse than a good human programmer can produce is not helpful in this world you're envisioning.
      - mikestorrent 1mo ago
        You might be a fan of kelvin versioning: https://wiki.xxiivv.com/site/kelvin_versioning.html https://wiki.xxiivv.com/site/kelvin_versioning.html
mikestorrent 1mo ago
> It is the gradual end of general purpose computing you own, and the start of everything just being cloud compute that someone else owns and you rent. A few thoughts... (a) There've never been more general purpose computers than there are right now. Spec progress may stall for consumer products due to the price increases, but the demand for workstations is not going to go down that much; they're critical items to get work done. Many are hoping this means a return to memory efficiency; the 8GB Macbook Neo is a good sign that this is something Apple believes to some extent also. (b) We've arguably been doing this since the 90s. A website (especially in the days of the heavy backend and more passive client) is cloud compute that someone else owns. You rent it by paying them in the form of looking at ads... (c) There are a lot of companies and people working on making sure you have every chance to run your AI locally. It's surprisingly good now and keeps getting better. At some point we'll see the same oscillation that we always do from thin client to fat client as the latency advantage of working at the edge starts to win out. How long until I have a laptop with a Talaas chip in it running a basic local model at 16000 tok/sec that is just good enough to use A2A to reach a more powerful agent running elsewhere if I need it?
nl 1mo ago
It's interesting to note that the total compute forecast to be deployed in 2026-2027 is more than the amount that has been deployed during human history.

bigstrat2003 1mo ago

Nobody cares what Claude said. Why not tell us what you think?

api 1mo ago

I agree with the sentiment, but there is a devils' advocate argument: Most compute loads are bursty. Most PCs and phones sit idle almost all the time. It's an inefficient use of capital. In the cloud you can be idle and then suddenly command 1000X more power than any affordable PC could ever have... for 10 seconds... then be idle again. Those kinds of work loads aren't possible to realize on a PC without massively overbuying hardware. I'd like to see more research on secure private cloud computing. Homomorphic encryption has gotten better but is still very slow, but there's other approaches that can help quite a bit but don't require much of a performance hit. Look into secure multiparty computation, reversible anonymization, etc. I don't think personal computing will die, but I think it'll go back to being a hobbyist and pro thing like it was from its birth in the 1970s up until the middle 1990s. In the 1980s the average person did not own a computer, or if they did it was just to play some games. The average person was not loading BASIC programs off bootleg tapes and calling BBSes. BTW I've been thinking a lot lately about how AI could save personal computing by allowing us to create an off-ramp from the present rats nest of complexity to something saner and simpler. Just thoughts right now but it's interesting.

wizzwizz4 1mo ago
Generative AI is definitely creating political will for simpler, more purposely-designed computer systems, but it's certainly not going to directly create help them in any meaningful way. AI-generated codebases are the posterchild of rat's nests of complexity.
- api 1mo ago
  I'd say the code I see coming out of leading models today is okay. Not great, but not terrible. It's also easy to get LLMs to code in more efficient languages, and you can also goad them via prompts and CLAUDE.md type stuff into prioritizing performance and simplicity. It's sort of like what I say about self driving cars. Are they better than good human drivers? Not yet. Are they better than bad, drunk, or texting drivers? Yes. LLMs are already better than bad coders and honestly most coders are bad coders.
  - wizzwizz4 1mo ago
    In a world where we no longer have the rat's nest of complexity, we pretty much no longer need to produce code. All the software we will ever need for the next century could fit on books in one, single-floor library, if we exclude art projects like video games. An operating system with full driver support for every I/O peripheral (if those devices use standard protocols, allowing for a https://xkcd.com/927/ https://xkcd.com/927/ constant of 5), and support for every standard document format (again, assuming 5 bitmap image formats, 5 vector image formats, 5 kinds of print-ready document, 5 kinds of hypertext document…), could easily fit in 2GiB, and that would be done: everything anyone could possibly need out of a computer, apart from art, mathematics and science, for… I don't even know how long. We're still using the Latin alphabet, more or less unchanged since the late 15th century. Working technology doesn't need to change. Being able to automatically generate large quantities of code that is worse than a good human programmer can produce is not helpful in this world you're envisioning.
    - mikestorrent 1mo ago
      You might be a fan of kelvin versioning: https://wiki.xxiivv.com/site/kelvin_versioning.html https://wiki.xxiivv.com/site/kelvin_versioning.html

mikestorrent 1mo ago

> It is the gradual end of general purpose computing you own, and the start of everything just being cloud compute that someone else owns and you rent. A few thoughts... (a) There've never been more general purpose computers than there are right now. Spec progress may stall for consumer products due to the price increases, but the demand for workstations is not going to go down that much; they're critical items to get work done. Many are hoping this means a return to memory efficiency; the 8GB Macbook Neo is a good sign that this is something Apple believes to some extent also. (b) We've arguably been doing this since the 90s. A website (especially in the days of the heavy backend and more passive client) is cloud compute that someone else owns. You rent it by paying them in the form of looking at ads... (c) There are a lot of companies and people working on making sure you have every chance to run your AI locally. It's surprisingly good now and keeps getting better. At some point we'll see the same oscillation that we always do from thin client to fat client as the latency advantage of working at the edge starts to win out. How long until I have a laptop with a Talaas chip in it running a basic local model at 16000 tok/sec that is just good enough to use A2A to reach a more powerful agent running elsewhere if I need it?

nl 1mo ago

It's interesting to note that the total compute forecast to be deployed in 2026-2027 is more than the amount that has been deployed during human history.