4 ms·
In 2022, most people using NLP use transformers from huggingface. The tokenizer used is written in Rust and used transparently from Python.
by wdroz 4y ago
In 2022, most people using NLP use transformers from huggingface. The tokenizer used is written in Rust and used transparently from Python.