4 ms·
Fast Bing Tokenizer for BERT
- sergeio76 7y agoState of the art performance for BERT :-)
- jiguang 7y ago+1
- chuah 7y agoSuper fast BERT tokenizer. It will save you 100x time in preprocessing text.
- sergeio76 7y agoMainly it will help improve latency at inference time. For preprocessing training one can always add more threads, but not at inference time.