3 ms·
In this case there's nothing that's variable, though, and the competition is able to pull it off precisely. Indeed, they themselves were able to do it before!
by _flux 2y ago
In this case there's nothing that's variable, though, and the competition is able to pull it off precisely. Indeed, they themselves were able to do it before!
- kevindamm 2y agoPerhaps they have a new tokenization method that's non-deterministic? If there are parallel lookaheads, not necessarily an rng, race conditions could make for variable cost. Or an expansion of certain terms into multiple token outputs, but the selection of which terms are expanded is based on a dynamic pool of Named Entities being recognized. Or maybe they just want to hide their process.. there is some secret sauce there even if it's deterministic, and so much depends on getting a good initial embedding, I've seen a tokenizer make or break an otherwise great model. I am merely hypothesizing, it may not be nondeterministic but I'm not going to assume it's not.