3 ms·

Check out this tokenizer for it as well: https://github.com/Xe/x/blob/master/web/tokiponatokens/toki_pona.go https://github.com/Xe/x/blob/master/web/tokiponatok

• xena 7y ago
  I used this for recompiling a subset of toki pona to a subset of prolog: https://github.com/Xe/x/tree/master/cmd/la-baujmi https://github.com/Xe/x/tree/master/cmd/la-baujmi. This is something I wish I had more time/effort to develop out more.
  • elliotec 7y ago
    Your blog has been a really exciting corner of the internet for me the past couple months. Thank you, big fan, keep doin what you're doin.
    • xena 7y ago
      Thanks! I'm working on a blogpost analyzing the speed gains I got by porting my webassembly environment to Rust. So far it's about 50x as fast on average, but most of the gain is from using an ahead-of-time compiler instead of an interpreter.