3 ms·

Great! Just did a quick test and got a 6-7x speedup on tokenization.

• clmnt 7y ago
  Mind sharing what tests your ran & with which setup? Thanks!