3 ms·
Cost per FLOP continues to drop on an exponential trend (and what bit flops do we mean?). Leaving aside more effective training methodologies and how that muddi
by Vetch 2y ago
Cost per FLOP continues to drop on an exponential trend (and what bit flops do we mean?). Leaving aside more effective training methodologies and how that muddies everything by allowing superior to GPT4 perf using less training flops, it also means one of the thresholds soon will not make sense.
With the other threshold, it creates a disincentive for models like llama-405B+, in effect enshrining an even wider gap between open and closed.
- pas 2y agoWhy? Llama is not generated by some guy in a shed. And even if it were, if said guy has such amount of compute, then it's time to use some of it to describe the model's safety profile. If it makes sense for Meta to release models, it would have made sense even with the requirement. (After all the whole point of the proposed regulation is to get some better sense of those closed models.)
- llamaimperative 2y agoAlso the bill was amended NOT to extend liability to derivative models that the training company doesn’t have effective control over.
- llamaimperative 2y agoBoth thresholds have a system to be adjusted.