the lucene standard tokenizer doesn't offer support for tokenizing emojis: it strips them out. I'm not aware of any lucene tokenizer that does support emojis to

the lucene standard tokenizer doesn't offer support for tokenizing emojis: it strips them out. I'm not aware of any lucene tokenizer that does support emojis tokenization. This isn't a huge deal, as lucene is open source and tokenizers can be written, but I'm just highlighting it as a shortcoming that's illustrative of the time and place lucene evolved from.