4 ms·
In the definitions of which models it applies to: > An artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or
by vasco 2y ago
In the definitions of which models it applies to:
> An artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or floating-point operations, the cost of which exceeds one hundred million dollars ($100,000,000) when calculated using the average market prices of cloud compute at the start of training as reasonably assessed by the developer
So unless you casually think you're going to throw 100 million dollars at a model training run, you're safe to innovate.
The bill itself seems reasonable to me and I'm in the camp of superalignment not really mattering because you don't know who the AI is going to align with, and there's many people with bad intent. The outcome of a model deciding to be evil or an evil person using a super-aligned model is pretty much the same except for 1 extra prompt.
- yeahwhatever10 2y agoHow exactly are they going to measure the compute number?
- striking 2y ago> when calculated using the average market prices of cloud compute at the start of training as reasonably assessed by the developer
- cscurmudgeon 2y ago1. The point is once a bill like this goes through, it is easy to lower the numbers. 2. And in the future, these operations will be much cheaper. 3. New or better architectures can be more powerful with lesser compute.
- striking 2y agoIf the operations are getting cheaper over time, doesn't that leave more room for AI to get better as time goes on?
- cscurmudgeon 2y agoYes, making the bill, as it is now, useless over time. And in the present, it adds unnecessary constraints for California companies while companies outside CA/US don't face such issues.
- deleted 2y ago[deleted]
- timschmidt 2y agoThe same amount of compute will cost ~$1500 after 16 doublings. In 24 years or less.
- Symmetry 2y agoIn a previous version of the bill it was either that many flops or that much money. Now it's both.
- chc4 2y agoWas it very shocking when you woke up today and found yourself transported from the year 2000, when Moore's Law still held?
- gleenn 2y agoYour comment is a bit snarky and I understand what you're saying but my understanding was that everyone thought Moore's Law was over when the computers stopped getting 2x faster every 18 months but that's not what Moore said. He said the transistor count would double amd it just so happened that that meant computers got a lot faster too because they benefited from all the xtra transistors. But GPUs do still get faster with more transistors because of the embarrassingly parallel nature of many matrix operations. So doubling the transistor count could still effectively significantly improve the speed and reduce time of AI training which is dominated by matrix math. Am I wrong? Did the transistor count doubling also slow down?
- timschmidt 2y agoThere's always someone out there claiming Moore's law to be dead. It never is. Chiplets and chip stacking in particular have enabled big increases in transistor count without associated downsides like decreased yield. So long as there is innovation and competition in the semiconductor industry, density / $, which you correctly point out was Moore's original metric, will continue to increase. The potential for increasing performance and decreasing cost of LLM inference is even higher than Moore's law suggests, as the algorithms improve and are pushed closer to ideal hardware to run them.