4 ms·
Are GPT models even bottlenecked by input tokenization? What real world speedup does this actually translate into?
by Llamamoe 4y ago
Are GPT models even bottlenecked by input tokenization? What real world speedup does this actually translate into?
- Jensson 4y agoCould be used for their wrapper API where they check and alter your input and the models output. This way they could quickly detect "bad words" and so on and return their standard platitudes about how the model can't answer. Or maybe they do a quick pass on websites and filter out websites with bad words quickly, just leaving a small fraction for the model to train on. Parsing to get the standardized tokens quickly would make that easier.