5 ms·
It is literally takes 20 min on my PC to prepare multilanguage corpus and train BPE+sentencepiece tokenizer with fairseq. You have all the documentation in ther
by pk-protect-ai 3y ago
It is literally takes 20 min on my PC to prepare multilanguage corpus and train BPE+sentencepiece tokenizer with fairseq. You have all the documentation in there. If you do not know how to use these tools, does not mean they are not there. You literally do not need to edit anything.
- monocasa 3y agoSweet. I've also been able to reverse engineer stripped binaries and reimplement them; I talked at a major conference last year about that actually. That doesn't make this release anymore source.