3 ms·

How does LLaMA handle fast fine-tuning? Are they using transformer adapters for it?

• loufe 4y ago
  It's already been adapted for hugging face transformers[1]. Apparently that should unlock its full potential. Oobabooga integrated the change into text-generation-webui[2] meaning we can already access a large chunk of its potential (from what I understand). [1] https://github.com/huggingface/transformers/pull/21955 https://github.com/huggingface/transformers/pull/21955 [2] https://github.com/oobabooga/text-generation-webui/commit/90206204aa41a73fe754e580ae1913dc2b01a384 https://github.com/oobabooga/text-generation-webui/commit/90...
  • bitL 4y ago
    That's absolutely fantastic! Thanks for the links!