3 ms·

thanks this is a good reference, this was the type of term I was looking for. I can implement this off of a paper. it's a bit tougher than regular string match

by zemo 6y ago

thanks this is a good reference, this was the type of term I was looking for. I can implement this off of a paper. it's a bit tougher than regular string matching because of homoglyph attacks (i.e., people trying to subvert the matching by writing f_ck or f0ck or fȗck or whatever) but I can probably do a pretty straightforward tokenization at ingest time to clean that up.

082349872349872 6y ago
If your users don't ever discuss places such as Scunthorpe, England or Fucking, Austria, the tokenisation may be effective.
- zemo 6y ago
  it's a multiplayer video game so assume users will attempt to subvert it by any means possible.
  - new_guy 6y ago
    I want to stick my long-necked Giraffe up your fluffy white bunny.
    - zemo 6y ago
      yeah, not trying to do sentiment analysis, that's out of scope lol