4 ms·

> If your business is selling tokens, it'd be extremely lucrative for you if the whole society relies on tokens to perform basic operations. That's where we're

by Gormo 2mo ago

> If your business is selling tokens, it'd be extremely lucrative for you if the whole society relies on tokens to perform basic operations. That's where we're heading to. I'm sure mainframe time-share providers in the '60s and '70s were salivating at the possibility of computers mediating most business tasks, too, completely unaware of the microcomputer revolution that was about to happen.

ljf 2mo ago
Very much this - we are firmly at the centralised stage - when I have truly local AI running on my devices, my cost will be the energy they use - if I use open source models. I follow a guy on instagram who is doing this today, with old mobile phones and Raspberry Pi - for now I'll stick to my free Perplexity account, but looking forward to being self-sufficient one day.
- Gormo 2mo ago
  > but looking forward to being self-sufficient one day. I don't know how much experimentation you're doing with local AI, but that day may be sooner than you think. The ecosystem is evolving extremely rapidly.
  - ljf 2mo ago
    I'm only on the edges to be honest and watching others - I'm not as technical as I once was, and I don't have the time I'd like to invest in this right now - too many competing hobbies. Looking forward to the day this becomes very very easy for the likes of me.
fny 2mo ago
But we do live in this era, no? What is cloud compute other than a time share?
- Gormo 2mo ago
  Someone in the early '80s could have also said "we do live in this era" in response to someone pointing at an Apple II or original IBM PC and seeing it as something that would ultimately upend the mainframe market entirely. In fact, we're already further along than that in terms of local AI. I'm currently able to get usable results at 8-10 tokens/sec using open-weight models on my laptop's integrated GPU, running on battery power. A $4,000 DGX Spark (less than what an IBM PC cost at launch in inflation-adjusted dollars) can get 3-5 times the inferencing performance with models 3-5x larger.

by Gormo 2mo ago

ljf 2mo ago

Very much this - we are firmly at the centralised stage - when I have truly local AI running on my devices, my cost will be the energy they use - if I use open source models. I follow a guy on instagram who is doing this today, with old mobile phones and Raspberry Pi - for now I'll stick to my free Perplexity account, but looking forward to being self-sufficient one day.

Gormo 2mo ago
> but looking forward to being self-sufficient one day. I don't know how much experimentation you're doing with local AI, but that day may be sooner than you think. The ecosystem is evolving extremely rapidly.
- ljf 2mo ago
  I'm only on the edges to be honest and watching others - I'm not as technical as I once was, and I don't have the time I'd like to invest in this right now - too many competing hobbies. Looking forward to the day this becomes very very easy for the likes of me.

fny 2mo ago

But we do live in this era, no? What is cloud compute other than a time share?

Gormo 2mo ago
Someone in the early '80s could have also said "we do live in this era" in response to someone pointing at an Apple II or original IBM PC and seeing it as something that would ultimately upend the mainframe market entirely. In fact, we're already further along than that in terms of local AI. I'm currently able to get usable results at 8-10 tokens/sec using open-weight models on my laptop's integrated GPU, running on battery power. A $4,000 DGX Spark (less than what an IBM PC cost at launch in inflation-adjusted dollars) can get 3-5 times the inferencing performance with models 3-5x larger.