7 ms·
And "tokens" are literally "any string of non whitespace characters." One of it's biggest downfalls imo is that the toeknizer is too simple. I'm making GitHub.
by vitiral 5y ago
And "tokens" are literally "any string of non whitespace characters."
One of it's biggest downfalls imo is that the toeknizer is too simple. I'm making GitHub.com/civboot/fngi as a "better than forth".
- codr7 5y agoOh, I've been down that rabbit hole for a long, long time :) It makes a nice foundation for syntactic experiments. https://github.com/codr7/gfoo https://github.com/codr7/gfoo https://github.com/codr7/forthy2 https://github.com/codr7/forthy2