3 ms·

interesting article, I have to give that a try! :D One ting is that while getting the value of running pretrained model weights like OPT-175B, there are also a

interesting article, I have to give that a try! :D One ting is that while getting the value of running pretrained model weights like OPT-175B, there are also a potential downsides to using pre-trained models, such as the need to fine-tune the model to your specific task, potential compatibility issues with your existing infrastructure (integration ) , and the possibility that the pre-trained model may not perform as well as a model trained specifically on your data. Ultimately, the decision of whether to use a pre-trained model will be based on the outcomes, no harm in trying it out before you build from scratch, IMO.

ilaksh 4y ago
But OpenAI's latest models (and a few others that are basically comparable) make that an obsolescent viewpoint since they are so general and capable and can adjust to a given context on the fly. So now what makes sense in my opinion is to keep going in that direction of generality. Take advantage of their API and otherwise work on open source efforts to reproduce the performance of those models or come up with new techniques that can get the same capabilities with less incredible resource needs.