2 ms·
If you mean storing the whole tokenised input in memory before processing it again, then I'd say never, and go as far as calling it hugely wasteful and ineffici
by userbinator 4y ago
If you mean storing the whole tokenised input in memory before processing it again, then I'd say never, and go as far as calling it hugely wasteful and inefficient.
The other comments here about catching all errors is valid, but you don't need to store any of the previous output to do so. The tokeniser can provide a token at a time as necessary, and thus also provide errors in the same way.