The Author's entire section "What is a Covered Model Here?" kind of sums up one of the main issues with the bill, even if they say themselves how clear the bill

The Author's entire section "What is a Covered Model Here?" kind of sums up one of the main issues with the bill, even if they say themselves how clear the bill is. ''' (1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024, or a model that could reasonably be expected to have similar performance on benchmarks commonly used to quantify the performance of state-of-the-art foundation models, as determined by industry best practices and relevant standard setting organizations. (2) The artificial intelligence model has capability below the relevant threshold on a specific benchmark but is of otherwise similar general capability. ''' They then go on to say: >Under this definition, if no one was actively gaming benchmarks, at most three existing models would plausibly qualify for this definition: GPT-4, Gemini Ultra and Claude. I am not even sure about Claude. Well how did they draw this conclusion? They decided that other models that do well are simply gaming benchmarks, and therefore can ignore the provision? >Um, no, because the open model weights models do not remotely reach the performance level of OpenAI? To me the author is simply incredibly biased and not well versed in other models. Is chatgpt4 better? Yes. Is Mistral8 comparable? I'd easily say yes. Using it just once would lead most to a similar conclusion, it's only after throwing word problems and tricks where you can really see the differences. And these kind of laws open to interpretation are always an issue because a responsible company has to err on the side of caution. If your model is often helpful, but only 80% as often as chatgpt, is that comparable? If someone released a new benchmark tomorrow, and your model actually beats out chatgpt, does your model all of a sudden apply?