5 ms·

The law seems to only care about models that take at least 10^26 floating point ops or models that achieve comparable benchmark performance. This is a truly ab

The law seems to only care about models that take at least 10^26 floating point ops or models that achieve comparable benchmark performance. This is a truly absurd number! I’m ok with organizations with that sort of compute capacity being subject to regulatory oversight and reporting and liability.

causal 2y ago
Important point, but I don't really have a reference for that number. Do we know what GPT-4 or other models would be in comparison?
- srackey 2y ago
  No, it’s not a good point. It could be 10^100000000 flops and it wouldn’t fucking matter. There’s no evidence that would do anything at all. “AGI” may not even be possible with those flops. You’re talking about what would require unprecedented control over computing to ease fears over a scary robot fanfic. None of the “safety” concerns are real. A GPT4 open source would do nothing but hurt Sam Altman’s bottom line - YOU ARE BEING DUPED.
changoplatanero 2y ago
With algorithmic improvements and new hardware, achieving comparable performance to 10^26 flops may not be so absurd anymore.
srackey 2y ago
If you’re not being serious, I appreciate your sarcasm. If you are, this may be one of the worst things I have ever read. It’s fucking math people. Math. It will not create some scary golem. It will create marginally better chatbots. You are arguing for totalitarian control over computing to line Microsoft’s pockets. Shame. Shame shame shame shame shame.
stale2002 2y ago
Its not an absurd number, depending on how it is calculated. I've seen that number thrown around, and by some calculations it would already apply to open source models like LLama. No, we don't need to ban or regulate LLama or any existing open source models. If someone wants to be worried about GPT-6, fine. But there is no need to regulate the stuff thats already out there.
Ukv 2y ago
Open-source models runnable on consumer hardware now achieve comparable benchmark performance to the "absurd" models from a couple of years ago.
brigadier132 2y ago
> least 10^26 floating point ops It's dumb to create laws based on arbitrary technological limits. Computing power is still increasing exponentially, yesterday's supercomputer is tomorrows gaming gpu.
ninjagoo 2y ago
This is not very far off from Llama3 training. 400 TFLOPS per GPU [1] and 6.4M GPU hours [2] puts Llama3 70B at 9.2*10^24 (so 10^25) floating point ops. [1] https://ai.meta.com/blog/meta-llama-3/ https://ai.meta.com/blog/meta-llama-3/ [2] https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct
deleted 2y ago
[deleted]
jahewson 2y ago
"I think there is a world market for maybe five computers." Thomas Watson, president of IBM, 1943

causal 2y ago

Important point, but I don't really have a reference for that number. Do we know what GPT-4 or other models would be in comparison?

srackey 2y ago
No, it’s not a good point. It could be 10^100000000 flops and it wouldn’t fucking matter. There’s no evidence that would do anything at all. “AGI” may not even be possible with those flops. You’re talking about what would require unprecedented control over computing to ease fears over a scary robot fanfic. None of the “safety” concerns are real. A GPT4 open source would do nothing but hurt Sam Altman’s bottom line - YOU ARE BEING DUPED.

changoplatanero 2y ago

With algorithmic improvements and new hardware, achieving comparable performance to 10^26 flops may not be so absurd anymore.

srackey 2y ago

If you’re not being serious, I appreciate your sarcasm. If you are, this may be one of the worst things I have ever read. It’s fucking math people. Math. It will not create some scary golem. It will create marginally better chatbots. You are arguing for totalitarian control over computing to line Microsoft’s pockets. Shame. Shame shame shame shame shame.

stale2002 2y ago

Its not an absurd number, depending on how it is calculated. I've seen that number thrown around, and by some calculations it would already apply to open source models like LLama. No, we don't need to ban or regulate LLama or any existing open source models. If someone wants to be worried about GPT-6, fine. But there is no need to regulate the stuff thats already out there.

Ukv 2y ago

Open-source models runnable on consumer hardware now achieve comparable benchmark performance to the "absurd" models from a couple of years ago.

brigadier132 2y ago

> least 10^26 floating point ops It's dumb to create laws based on arbitrary technological limits. Computing power is still increasing exponentially, yesterday's supercomputer is tomorrows gaming gpu.

ninjagoo 2y ago

This is not very far off from Llama3 training. 400 TFLOPS per GPU [1] and 6.4M GPU hours [2] puts Llama3 70B at 9.2*10^24 (so 10^25) floating point ops. [1] https://ai.meta.com/blog/meta-llama-3/ https://ai.meta.com/blog/meta-llama-3/ [2] https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct

deleted 2y ago

[deleted]

jahewson 2y ago

"I think there is a world market for maybe five computers." Thomas Watson, president of IBM, 1943