3 ms·
hugging face model https://huggingface.co/pbelcak/UltraFastBERT-1x11-long https://huggingface.co/pbelcak/UltraFastBERT-1x11-long
by vorticalbox 3y ago
hugging face model
https://huggingface.co/pbelcak/UltraFastBERT-1x11-long https://huggingface.co/pbelcak/UltraFastBERT-1x11-long
- lawlessone 3y ago> This model is provided only as sanity check for research purposes, it is untested and unfit for deployment. I guess this means it isn't pretrained yet? Is it still just random weights?
- mjn 3y ago"Unfit for deployment" or "not intended for deployment" is semi-standard wording for research models that are just raw language models with none of the safety/bias/offensiveness filtering that is usually desired for product applications. For example, if you deploy it as a customer-service chatbot, it might tell your customers to kill themselves, or call them racial slurs. It doesn't mean that there's anything technically wrong with the language model per se as a model of language, just that there has been no effort made to ensure it's fit to be deployed as-is for any given generative-AI use case, and the model authors would prefer you didn't do that.
- gpm 3y agoIs it even fine tuned for question answering?
- ilaksh 3y agoIs it possible to use this with something like Llama 2?
- sanxiyn 3y agoThis requires retraining from scratch, so no, you can't use Llama 2 pretrained weight. As far as I can tell you can take Llama 2 modelling code, training infrastructure, training data and apply proposed modification (they provide PyTorch nn.Module which should be drop in replacement of nn.Linear) and run the training if you have enough compute and it should work. Doesn't mean it would work, there are always lots of practical problems, but it should work in principle.