4 ms·
Or there’s a trade off between more complex schemas and logit bias going off the rails since there’s probably little to no backtracking.
by civilitty 3y ago
Or there’s a trade off between more complex schemas and logit bias going off the rails since there’s probably little to no backtracking.
- newhouseb 3y agoGood point. Backtracking is certainly possible but it is probably tricky to parallelize at scale if you're trying to coalesce and slam through a bunch of concurrent (unrelated) requests with minimal pre-emption.