3 ms·
Haven't read the paper, but they are probably using something like sentencepiece with sub-word splitting and then charge by the number of resulting token. http
by belval 5y ago
Haven't read the paper, but they are probably using something like sentencepiece with sub-word splitting and then charge by the number of resulting token.
https://github.com/google/sentencepiece https://github.com/google/sentencepiece