3 ms·

The costs won't always be that high, due to Moore's law and other advances. Anybody can train a GPT-2 right now for a couple hundred bucks. https://www.tomshard

The costs won't always be that high, due to Moore's law and other advances. Anybody can train a GPT-2 right now for a couple hundred bucks. https://www.tomshardware.com/tech-industry/artificial-intelligence/former-tesla-ai-director-reproduces-gpt-2-in-24-hours-for-only-672 https://www.tomshardware.com/tech-industry/artificial-intell... Also model performance follows a power law. There are strongly diminishing results, where after some point throwing twice as much money at the problem only gains you 1% accuracy. OpenAI knows this, so they're all in on regulatory capture.

gigatexal 2y ago
It’s always going to be like this. Take Apple. Plucky garage started anti-establishment startup now a giant gate keeper of apps and stores using any means necessary to keep that 30% tax/tariff/whatever you call it
- scarface_74 2y ago
  Jobs wanted lock down computing as far as the first Mac in 1984.
bmelton 2y ago
AI seems to have validated Wirth's Law[1] more than it has Moore's, and they are running in direct competition [1] - "Software is getting slower more rapidly than hardware is becoming faster"
JohnMakin 2y ago
It’s odd you’re invoking moore’s law here, which actually isn’t a law, and is well understood and accepted over at least the last decade across business and academia that it is slowing and is not limitless. As per what “other” advances you’re referring to, I can’t speculate - maybe quantum computing? We are quite a long way away from that being a commercial reality.
- Herring 2y ago
  It's mainly hardware upgrades (new TSMC nodes), training data quantity/quality improvements, and model architecture (and broader software) changes. Here's a fun project I've had my eye on recently: https://github.com/KellerJordan/modded-nanogpt/ https://github.com/KellerJordan/modded-nanogpt/
dutchbookmaker 2y ago
Right now it seems more and more like we will all just end up using Chinese AI. All this regulation is doing is forcing Chinese companies to figure out how to do more with less. Who cares if Chinese AGI is a year behind OpenAI/Anthropic. At some level, progress will stall and then OpenAI/Anthropic will be on the clock before being undercut out of business on price. They are probably just positioning now for when that happens and they have to become some kind of regulated utility for US government AI use. Just another example of having our head up our ass as Americans. Seems like what we are doing will almost guarantee most the world outside the US is using Chinese AI in the future.
- Herring 2y ago
  I really really freaking wish Americans would stop worrying about who has the best $STUFF, and start worrying about whether their own kids are fed/clothed/educated. China is too big. This was going to happen eventually. Empires come and go, that's just a fact of life. The question was whether you'd fall back gracefully like Western Europe, now ranking at the top of "World's Happiest Countries", or whether you'll become Russia 2.0 with the biggest guns, richest oligarchs, and the worst quality of life. Eisenhower 1953 said "Every gun that is made, every warship launched, every rocket fired signifies, in the final sense, a theft from those who hunger and are not fed, those who are cold and are not clothed." Nobody listened. Americans think life is a Marvel movie, and they are Iron Man. Turns out conflict is not a long-term winning strategy.
- NemoNobody 2y ago
  How could anyone trust a Chinese AI?

gigatexal 2y ago

It’s always going to be like this. Take Apple. Plucky garage started anti-establishment startup now a giant gate keeper of apps and stores using any means necessary to keep that 30% tax/tariff/whatever you call it

scarface_74 2y ago
Jobs wanted lock down computing as far as the first Mac in 1984.

bmelton 2y ago

AI seems to have validated Wirth's Law[1] more than it has Moore's, and they are running in direct competition [1] - "Software is getting slower more rapidly than hardware is becoming faster"

JohnMakin 2y ago

It’s odd you’re invoking moore’s law here, which actually isn’t a law, and is well understood and accepted over at least the last decade across business and academia that it is slowing and is not limitless. As per what “other” advances you’re referring to, I can’t speculate - maybe quantum computing? We are quite a long way away from that being a commercial reality.

Herring 2y ago
It's mainly hardware upgrades (new TSMC nodes), training data quantity/quality improvements, and model architecture (and broader software) changes. Here's a fun project I've had my eye on recently: https://github.com/KellerJordan/modded-nanogpt/ https://github.com/KellerJordan/modded-nanogpt/

dutchbookmaker 2y ago

Right now it seems more and more like we will all just end up using Chinese AI. All this regulation is doing is forcing Chinese companies to figure out how to do more with less. Who cares if Chinese AGI is a year behind OpenAI/Anthropic. At some level, progress will stall and then OpenAI/Anthropic will be on the clock before being undercut out of business on price. They are probably just positioning now for when that happens and they have to become some kind of regulated utility for US government AI use. Just another example of having our head up our ass as Americans. Seems like what we are doing will almost guarantee most the world outside the US is using Chinese AI in the future.

Herring 2y ago
I really really freaking wish Americans would stop worrying about who has the best $STUFF, and start worrying about whether their own kids are fed/clothed/educated. China is too big. This was going to happen eventually. Empires come and go, that's just a fact of life. The question was whether you'd fall back gracefully like Western Europe, now ranking at the top of "World's Happiest Countries", or whether you'll become Russia 2.0 with the biggest guns, richest oligarchs, and the worst quality of life. Eisenhower 1953 said "Every gun that is made, every warship launched, every rocket fired signifies, in the final sense, a theft from those who hunger and are not fed, those who are cold and are not clothed." Nobody listened. Americans think life is a Marvel movie, and they are Iron Man. Turns out conflict is not a long-term winning strategy.
NemoNobody 2y ago
How could anyone trust a Chinese AI?