7 ms·
Yoshua Bengio is a very respected scientist with well deserved reputation, but this discussion is upsetting… “academia now trains much smaller models… 10^26 FLO
by pama 2y ago
Yoshua Bengio is a very respected scientist with well deserved reputation, but this discussion is upsetting… “academia now trains much smaller models… 10^26 FLOPs is 10 to the 26 floating point operations per second.. yes.. how big is that compared to GPT-4? It is much bigger than all the existing ones…” (flops has a different meaning: there is no per second in the law; one single H100 from last year performs 1e15 FLOPs per second; llama3.1 was close to the 1e26 limit this year, and the total training FLOPS of other models are not published; research could change once compute is even cheaper but state laws move at glacial speeds…).
It is disheartening so see the damage capacity in the hands of a couple of paranoic people who perhaps read the wrong scifi and had lots of power to influence others. If California passes this law, in a few years the world economy will be very different.
- EasyMark 2y agoI think companies will simply quit doing business in California. They are killing the golden goose with all these regulations, just like they poison their own real estate markets allowing NIMBYs to dictate regulations that keep housing prices high and allow petty criminals to run rampant while cities like San Francisco continue to diminish and suffer
- evgen 2y ago> I think companies will simply quit doing business in California Sure they will. Just like how the auto manufacturers stopped trying to sell in CA after stricter emissions regulations and just like how companies stopped trying to serve European visitors after GDPR came into force. California is a $4T GDP, placing it just behind Japan and ahead of India in global rankings. I am sure they are happy to tell non-complaint companies to piss off and not let the door hit them on the way out. This is the canary in the coal mine for cowboy AI companies, once EU regulators get into the game their days of playing fast and loose with this tech will be over.
- pama 2y agoUnfortunately, it increasingly feels like the cowboy is the lobbyist in this picture and the victims of their scam are the regulators and people with semi religious beliefs. The specific limits in the law are possible to reach with this year’s hardware and enough money to buy a large gpu cluster and let it train for months. In five years the current limit in the law might be accessible by academia using government clusters and in ten years it may be accessible to patient home users, especially if some of the specialized hardwares that are now in the works find ways to commercialize and have mass appeal. Limiting the amount of compute by law is such an odd concept when we are in a phase of exponential growth.
- echoangle 2y agoThe law isn’t permanent, if you notice that it doesn’t make sense in 5 years, you can still change it.
- pama 2y agoDont underestimate the glacial kinetics of existing laws. I dont recall many examples of legal regulation that took lots of lobbying and years of planning to pass, to then be changed within years. Sometimes non-permanent laws, such as executive orders, change when executives change, but laws are typically only created and stay until they are either deemed unconstitutional or completely out of sync with reality, and even then only if there is nothing more important or new to write laws about. The more common end of bad laws is for executives to decide to not enforce them, eg outdated federal or state laws about drugs, but it takes a lot of momentum and time to undo a bad law, much more time than to write it in the first place. Just like with bad code in large codebases, new code is easier to write and brings more glory than carefully eliminating or amending poor code.