3 ms·
Access to copyrighted materials might make for slightly better-trained models the way that access to more powerful GPUs does. But I don't think it will accelera
by mitthrowaway2 1y ago
Access to copyrighted materials might make for slightly better-trained models the way that access to more powerful GPUs does. But I don't think it will accelerate foundational advances in the underlying technology. If anything, maybe having to compete under tight constraints means AI companies will have to innovate more, rather than merely push scale.
- int_19h 1y agoThe problem is that regardless of any innovations, scale still matters. If you figure out the technique to, say, make a model that is significantly better given N parameters - where N is just large enough to be the perfect fit for the amount of training data that you have access to - then someone else with access to more data will use the same technique to make a model with >N parameters, and it will be better than yours.
- mr_toad 1y agoBut AI is mostly scale and only a little bit innovation. It’s undergraduate maths and a whole lot of computing power and data. Not being able to train on data on the internet would be a significant handicap.