4 ms·

I have seen the term "trigram" twice today on HN. I am familiar with the term only for something like on the South Korean flag [1]. Interesting to see this bein

I have seen the term "trigram" twice today on HN. I am familiar with the term only for something like on the South Korean flag [1]. Interesting to see this being used for a special case of n-grams in language processing. [1]: https://en.wikipedia.org/wiki/Bagua https://en.wikipedia.org/wiki/Bagua

tgv 4y ago
It's not really special. It used to be quite common, as well as bigrams (n=2). In the Roman alphabet, 3 characters give a fair trade-off between precision and recall, making them suitable to things like spelling error correction. Everything old will be new again.
- akomtu 4y ago
  Afaik, GPT works with trigrams, and calls them tokens.
- msm_ 4y ago
  They also give you a good performance. I'm a maintainer of a specialised open-source 3gram database, and there's a lot of performance tricks you can do with 3grams. There are around 2*24 different 3grams (less if you only care about printable characters). This is small enough to pack into a single array on a modern computer. For example the file format I use has a header that reserves 8 bytes of data [1] for every possible 3gram - that's just 128MB of overhead per index, not a lot of disk storage. For 4grams, that's 32GB, much less acceptable even now. And 2grams are just not very useful for large datasets. [1] for the curious - the header is basically uint64_t file_offsets_to_3gram_data[2**24].
dekhn 4y ago
Much of google search ranking is based on (well, was ten years ago) on bigram and trigram statistics. Short of a real n-gram model this is remarkably effective but has some obvious failure modes.
Rimintil 4y ago
If you want a challenge, try solving the Eye Messages puzzle from Noita, which is a bunch of trigrams! https://noita.wiki.gg/wiki/Eye_Messages https://noita.wiki.gg/wiki/Eye_Messages

4 ms·

I have seen the term "trigram" twice today on HN. I am familiar with the term only for something like on the South Korean flag [1]. Interesting to see this bein

by zelphirkalt 4y ago

tgv 4y ago
It's not really special. It used to be quite common, as well as bigrams (n=2). In the Roman alphabet, 3 characters give a fair trade-off between precision and recall, making them suitable to things like spelling error correction. Everything old will be new again.
- akomtu 4y ago
  Afaik, GPT works with trigrams, and calls them tokens.
- msm_ 4y ago
  They also give you a good performance. I'm a maintainer of a specialised open-source 3gram database, and there's a lot of performance tricks you can do with 3grams. There are around 2*24 different 3grams (less if you only care about printable characters). This is small enough to pack into a single array on a modern computer. For example the file format I use has a header that reserves 8 bytes of data [1] for every possible 3gram - that's just 128MB of overhead per index, not a lot of disk storage. For 4grams, that's 32GB, much less acceptable even now. And 2grams are just not very useful for large datasets. [1] for the curious - the header is basically uint64_t file_offsets_to_3gram_data[2**24].
dekhn 4y ago
Much of google search ranking is based on (well, was ten years ago) on bigram and trigram statistics. Short of a real n-gram model this is remarkably effective but has some obvious failure modes.
Rimintil 4y ago
If you want a challenge, try solving the Eye Messages puzzle from Noita, which is a bunch of trigrams! https://noita.wiki.gg/wiki/Eye_Messages https://noita.wiki.gg/wiki/Eye_Messages