3 ms·
Indeed. Local models becoming available and halfway decent don't obviate the laws of scale. And because there's no ceiling to what scaling more will buy you in
by bitwize 4mo ago
Indeed. Local models becoming available and halfway decent don't obviate the laws of scale. And because there's no ceiling to what scaling more will buy you in terms of capability, there's no reason not to scale more, there's no incentive for billionaires not to grab all the fab capacity they can.
Enjoy paying $1000 or more for a little 4 GiB cloud terminal that connects you to all your online accounts where all your actual work gets done. This is the future.
- colechristensen 4mo ago>there's no ceiling to what scaling more will buy you in terms of capability This is highly doubtful. Rule of thumb: everything people think is exponential is actually an S curve.
- bitwize 4mo agoThere's a limit that won't be breached without a fundamental breakthrough in physics of computation, but we're not there yet by a long shot. You can train bigger models, faster, and infer with them faster and more precisely, by throwing more compute at the problem for the foreseeable.
- rbanffy 4mo agoAt some point, and I can already see it, they’ll be better than us at writing code. We are still in the loop to coerce them into architecting well, but that’s nothing magical. What’s frontier now is prosumer in a couple years and commonplace in a couple more.
- nok22kon 4mo agobetter rule: exponentials are overlapping S-curves