2 ms·
> This is something impossible in other languages, being able to program your own tokenizer for some foreign language from within cognition, and have future cod
by lifthrasiir 3y ago
> This is something impossible in other languages, being able to program your own tokenizer for some foreign language from within cognition, and have future code be tokenized exactly like how you want it to be. [Emphases in original]
Doesn't that sound like TeX \catcode? In fact, I think you can bootstrap Cognition from TeX as well.