5 ms·
It is a tokenizer artifact most likely (https://github.com/huggingface/transformers/issues/4786 https://github.com/huggingface/transformers/issues/4786). So the
by frwickst 6mo ago
It is a tokenizer artifact most likely (https://github.com/huggingface/transformers/issues/4786 https://github.com/huggingface/transformers/issues/4786). So the output is not properly decoded in this case, it should just be a space.