3 ms·

This is a good one for everything related to SB1047 https://pca.st/episode/44b41e28-5772-41c4-bcd7-5d7aa48d5120 https://pca.st/episode/44b41e28-5772-41c4-bcd7-5

This is a good one for everything related to SB1047 https://pca.st/episode/44b41e28-5772-41c4-bcd7-5d7aa48d5120 https://pca.st/episode/44b41e28-5772-41c4-bcd7-5d7aa48d5120

pama 2y ago
Yoshua Bengio is a very respected scientist with well deserved reputation, but this discussion is upsetting… “academia now trains much smaller models… 10^26 FLOPs is 10 to the 26 floating point operations per second.. yes.. how big is that compared to GPT-4? It is much bigger than all the existing ones…” (flops has a different meaning: there is no per second in the law; one single H100 from last year performs 1e15 FLOPs per second; llama3.1 was close to the 1e26 limit this year, and the total training FLOPS of other models are not published; research could change once compute is even cheaper but state laws move at glacial speeds…). It is disheartening so see the damage capacity in the hands of a couple of paranoic people who perhaps read the wrong scifi and had lots of power to influence others. If California passes this law, in a few years the world economy will be very different.
- EasyMark 2y ago
  I think companies will simply quit doing business in California. They are killing the golden goose with all these regulations, just like they poison their own real estate markets allowing NIMBYs to dictate regulations that keep housing prices high and allow petty criminals to run rampant while cities like San Francisco continue to diminish and suffer
  - evgen 2y ago
    > I think companies will simply quit doing business in California Sure they will. Just like how the auto manufacturers stopped trying to sell in CA after stricter emissions regulations and just like how companies stopped trying to serve European visitors after GDPR came into force. California is a $4T GDP, placing it just behind Japan and ahead of India in global rankings. I am sure they are happy to tell non-complaint companies to piss off and not let the door hit them on the way out. This is the canary in the coal mine for cowboy AI companies, once EU regulators get into the game their days of playing fast and loose with this tech will be over.
    - pama 2y ago
      Unfortunately, it increasingly feels like the cowboy is the lobbyist in this picture and the victims of their scam are the regulators and people with semi religious beliefs. The specific limits in the law are possible to reach with this year’s hardware and enough money to buy a large gpu cluster and let it train for months. In five years the current limit in the law might be accessible by academia using government clusters and in ten years it may be accessible to patient home users, especially if some of the specialized hardwares that are now in the works find ways to commercialize and have mass appeal. Limiting the amount of compute by law is such an odd concept when we are in a phase of exponential growth.
      - echoangle 2y ago
        The law isn’t permanent, if you notice that it doesn’t make sense in 5 years, you can still change it.
        pama 2y ago
        Dont underestimate the glacial kinetics of existing laws. I dont recall many examples of legal regulation that took lots of lobbying and years of planning to pass, to then be changed within years. Sometimes non-permanent laws, such as executive orders, change when executives change, but laws are typically only created and stay until they are either deemed unconstitutional or completely out of sync with reality, and even then only if there is nothing more important or new to write laws about. The more common end of bad laws is for executives to decide to not enforce them, eg outdated federal or state laws about drugs, but it takes a lot of momentum and time to undo a bad law, much more time than to write it in the first place. Just like with bad code in large codebases, new code is easier to write and brings more glory than carefully eliminating or amending poor code.
trissi1996 2y ago
IMHO it's not, it just parrots the same old arguments for "safety", arguing against straw-men and framing the other side as having wrong assumptions about AI safety/being unfair/etc, all while not going into the principled counter-arguments and their own assumptions at all. Here are some counter-points: Regulation: - Very little effort is made to evaluate risk of over-regulating, regulatory capture and counterproductive wrong regulation - The downside of under-regulating is vastly overemphasized, most arguments boil down to "we have to act FAST now or x BAD thing might happen" - The risk of over/wrongly regulating is vastly under emphasized with the same FUD reasons. - according to one of the many straw-men argument in the pod I'm a libertarian against any and all regulation because I criticize possible regulatory capture, I would enthusiastically support regulation that foundation models have to be: -- given freely to public researchers/academics for in-depth independent safety-research -- open weighted after a while (e.g. after ~ a year, safety concerns should be mostly ruled out and new generations are out so ROI is already likely there. [e.g. there's NO safety reason at all for ClosedAI to not release gpt-3.5, llama3 is better already]) Proposed FLOP cut-off of SB 1047: - according to the pod, the cut off is much more advanced than anything currently released. - The 10^26 FLOP cutoff is way to low, llama-405b is ~4×10^25 FLOPs - 405B is maybe 20% smarter than 70B, while taking over an order of magnitude more FLOPS to train, the cutoff itself is very likely not much smarter than the current SOTA. - IMO none of the current SOTA models are very dangerous, but kill switch regulation is. Kill-Switches: - SB 1047 is (non-explicitly) calling for kill-switches over the cut-off due to liability of the model creators and market dynamics - Any kill-switch regulation means a complete dead-end to any advanced open-weights AI. This means that huge corporations and governments will control any and all advanced AI-development. This is top-down control of the maths you are allowed to run on your computer IMO that is Orwellian as fuck. China: - mentioning china is FUD 101, it's basically AI's "think of the children" - If they think they can stop china from building their own advanced LLMs, they're delusional. This regulation might even help them to get there faster. They don't even need to steal, there maybe a year or two behind the SOTA and catching up fast. I just don't get how so many people on a site with "hacker" in the name want to make it impossible to hack on these things for anyone not employed by the big corporate AI research labs.

This is a good one for everything related to SB1047 https://pca.st/episode/44b41e28-5772-41c4-bcd7-5d7aa48d5120 https://pca.st/episode/44b41e28-5772-41c4-bcd7-5d7aa48d5120

pama 2y ago

Yoshua Bengio is a very respected scientist with well deserved reputation, but this discussion is upsetting… “academia now trains much smaller models… 10^26 FLOPs is 10 to the 26 floating point operations per second.. yes.. how big is that compared to GPT-4? It is much bigger than all the existing ones…” (flops has a different meaning: there is no per second in the law; one single H100 from last year performs 1e15 FLOPs per second; llama3.1 was close to the 1e26 limit this year, and the total training FLOPS of other models are not published; research could change once compute is even cheaper but state laws move at glacial speeds…). It is disheartening so see the damage capacity in the hands of a couple of paranoic people who perhaps read the wrong scifi and had lots of power to influence others. If California passes this law, in a few years the world economy will be very different.

EasyMark 2y ago
I think companies will simply quit doing business in California. They are killing the golden goose with all these regulations, just like they poison their own real estate markets allowing NIMBYs to dictate regulations that keep housing prices high and allow petty criminals to run rampant while cities like San Francisco continue to diminish and suffer
- evgen 2y ago
  > I think companies will simply quit doing business in California Sure they will. Just like how the auto manufacturers stopped trying to sell in CA after stricter emissions regulations and just like how companies stopped trying to serve European visitors after GDPR came into force. California is a $4T GDP, placing it just behind Japan and ahead of India in global rankings. I am sure they are happy to tell non-complaint companies to piss off and not let the door hit them on the way out. This is the canary in the coal mine for cowboy AI companies, once EU regulators get into the game their days of playing fast and loose with this tech will be over.
  - pama 2y ago
    Unfortunately, it increasingly feels like the cowboy is the lobbyist in this picture and the victims of their scam are the regulators and people with semi religious beliefs. The specific limits in the law are possible to reach with this year’s hardware and enough money to buy a large gpu cluster and let it train for months. In five years the current limit in the law might be accessible by academia using government clusters and in ten years it may be accessible to patient home users, especially if some of the specialized hardwares that are now in the works find ways to commercialize and have mass appeal. Limiting the amount of compute by law is such an odd concept when we are in a phase of exponential growth.
    - echoangle 2y ago
      The law isn’t permanent, if you notice that it doesn’t make sense in 5 years, you can still change it.
      - pama 2y ago
        Dont underestimate the glacial kinetics of existing laws. I dont recall many examples of legal regulation that took lots of lobbying and years of planning to pass, to then be changed within years. Sometimes non-permanent laws, such as executive orders, change when executives change, but laws are typically only created and stay until they are either deemed unconstitutional or completely out of sync with reality, and even then only if there is nothing more important or new to write laws about. The more common end of bad laws is for executives to decide to not enforce them, eg outdated federal or state laws about drugs, but it takes a lot of momentum and time to undo a bad law, much more time than to write it in the first place. Just like with bad code in large codebases, new code is easier to write and brings more glory than carefully eliminating or amending poor code.

trissi1996 2y ago

IMHO it's not, it just parrots the same old arguments for "safety", arguing against straw-men and framing the other side as having wrong assumptions about AI safety/being unfair/etc, all while not going into the principled counter-arguments and their own assumptions at all. Here are some counter-points: Regulation: - Very little effort is made to evaluate risk of over-regulating, regulatory capture and counterproductive wrong regulation - The downside of under-regulating is vastly overemphasized, most arguments boil down to "we have to act FAST now or x BAD thing might happen" - The risk of over/wrongly regulating is vastly under emphasized with the same FUD reasons. - according to one of the many straw-men argument in the pod I'm a libertarian against any and all regulation because I criticize possible regulatory capture, I would enthusiastically support regulation that foundation models have to be: -- given freely to public researchers/academics for in-depth independent safety-research -- open weighted after a while (e.g. after ~ a year, safety concerns should be mostly ruled out and new generations are out so ROI is already likely there. [e.g. there's NO safety reason at all for ClosedAI to not release gpt-3.5, llama3 is better already]) Proposed FLOP cut-off of SB 1047: - according to the pod, the cut off is much more advanced than anything currently released. - The 10^26 FLOP cutoff is way to low, llama-405b is ~4×10^25 FLOPs - 405B is maybe 20% smarter than 70B, while taking over an order of magnitude more FLOPS to train, the cutoff itself is very likely not much smarter than the current SOTA. - IMO none of the current SOTA models are very dangerous, but kill switch regulation is. Kill-Switches: - SB 1047 is (non-explicitly) calling for kill-switches over the cut-off due to liability of the model creators and market dynamics - Any kill-switch regulation means a complete dead-end to any advanced open-weights AI. This means that huge corporations and governments will control any and all advanced AI-development. This is top-down control of the maths you are allowed to run on your computer IMO that is Orwellian as fuck. China: - mentioning china is FUD 101, it's basically AI's "think of the children" - If they think they can stop china from building their own advanced LLMs, they're delusional. This regulation might even help them to get there faster. They don't even need to steal, there maybe a year or two behind the SOTA and catching up fast. I just don't get how so many people on a site with "hacker" in the name want to make it impossible to hack on these things for anyone not employed by the big corporate AI research labs.