4 ms·

Is this just a reinvented humanhash?

• pranshuchittora 5mo ago
  Kinda similar, but this is token efficient. Each word is ~1 BPE token
• asdfsa32 5mo ago
  Everything is old is new.
• anuramat 5mo ago
  was wondering myself, just tried comparing to petnames crate -- gets you about 2 tokens per word on average not that anyone should ever care; typos in random-looking ids are very real but already covered by human readable ids besides, this is for a specific tokenizer