3 ms·
Electricity becoming a commodity doesn't necessarily make nuclear reactors become commodities. And as we can see, the publication of open-weight models doesn't
by typ 2mo ago
Electricity becoming a commodity doesn't necessarily make nuclear reactors become commodities. And as we can see, the publication of open-weight models doesn't really diffuse the know-how of training SoAT models as we expected. Even the major cloud providers are still unable to build competitive ones from scratch on their own. That's very different than the traditional FOSS ecosystem, where everyone can copy, learn, and evolve on one's own (like, Bitcoin -> altcoins).
On token pricing, I think it's very much bottlenecked by hardware (the aggregate of compute) rather than the number of competing models. Assuming that the ceiling of the token price is determined by the economic value a unit of compute can provide, then the less efficient ones would be priced out of the compute allocation. It's not consumers bidding up a limited number of different models, but more like tokens of different models bidding up the limited computing resource. Less-intelligent tokens (which are generated by weaker models) are crowded out by smarter tokens from the limited compute. My prediction is that we'll see a meaningful downward pressure on token prices only when the new batches of next-generation hardware get mass-deployed.
- podocarp 2mo agoI don't think the big companies not making Sota models being an indicator that open source failed. If all semiconductor patents expired tomorrow it would still be prohibitively difficult to manufacture your own CPU at home or at work.
- typ 2mo agoMe neither. But open weights are not open source in the first place. It wouldn't be surprising that the raw intelligence of an LLM model has little to do with the architecture (which is the part most published). The secret sauce, like how/what data gets curated and cleaned, has rarely been talked about. So, even if the labs upload their weights, competitors are very difficult, if not impossible, to replicate the technology independently.
- childintime 2mo ago> Electricity becoming a commodity doesn't necessarily make nuclear reactors become commodities. Bad comparison? Nuclear reactors become obsolete. The only reason to build a nuclear reactor is as a government vanity project, just to get the bribes you want, or weapons grade material. Dirty tech.
- LunaSea 2mo agoAnd you know the small benefit of having stable, large scale and low CO2 emission electricity production.
- childintime 2mo agoYou have put on rosy glasses. Solar begins saving CO2 immediately, while nuclear reactors begin emitting CO2 before they are ready. After a 10-15 year investment in concrete that results in around 4x the CO2 emissions.. they are obsolete by the time they start producing. And additionally they produce still nuclear waste, nobody wants to have them nearby, and they are a strategically high risk in multiple ways (military, financial, environmental). They are vanity projects.
- LunaSea 1mo agoVanity projects that make France produce electricity with 25x less CO2 / kWh than Germany.