5 ms·
Very good problem description and nice list of countermeasures! However, the following countermeasure made be wonder: > Manually retype excerpts to avoid invi
by vog 9y ago
Very good problem description and nice list of countermeasures!
However, the following countermeasure made be wonder:
> Manually retype excerpts to avoid invisible characters and homoglyphs.
Isn't this something you can automate? We should create linters for plain text (rather than code)? For example, depending on the language, reduce the text to a certain set of characters. Every character not in this whitelist is either replaced, or causes an error message the user (journalist) needs to deal with (i.e. remove it, or replace it with an innocent alternative, perhaps even proposing this replacement back to the linter project).
Of course, there are multiple ways for linting, which might become a fingerprint on it own. But then, if there are only 3 or 4 of such linter styles actually used (ideally, standardize on exactly one linting style), you can only tell which linter was used by the journalist, without any information about their source.
- adtac 9y agoConvert to image and run it through an OCR? Needlessly complicated, but it could possibly mimic the retyping part.
- claudiulodro 9y agoWhy not just a regex that matches on ASCII characters and removes the rest?
- adtac 9y agoBecause you still probably want accented characters and other unicode elements, I think.
- walshemj 9y agowhy outwith non latin languages accents and oddities like the sharp S dont actually change the meaning.
- sharpercoder 9y agoUsing RegEx is generally frowned upon; RegEx is a bad language to write anything in apart from prototypes. Furthermore, this will not work for most text as even US English text contains special characters. Think paraphrasing other languages, names and imported words.
- GordonS 9y agoFrowned upon by who? 'Bad' in what way? Regular expressions are powerful, and are used in plenty of production software.
- sharpercoder 9y ago* RegEx opens your application up for DoS attacks * RegEx is not very readable * RegEx can be (very) slow * It's not trivial to write RegEx code that achieves your goal in a high-quality way. Often quircks and edge-cases are missed. I'm not saying that you should never use them, but oftentimes a (much) better alternative of achieving your goals is present. See https://blog.codinghorror.com/regular-expressions-now-you-have-two-problems/ https://blog.codinghorror.com/regular-expressions-now-you-ha...
- geofft 9y agoAs the article you link argues, use of regular expressions when they are inappropriate is bad. This particular case - finding and replacing certain characters with other characters - is pretty well-suited to the problem, and is probably more readable than a bunch of open code to do the same thing. (I'm not sure what you mean by DoS attacks - are you referring to the exponential case of backtracking? If so, don't use a regex engine with that problem, and don't use lookbehind/lookahead assertions, which aren't needed to solve this problem.)
- sharpercoder 9y ago> This particular case - finding and replacing certain characters with other characters - is pretty well-suited to the problem, and is probably more readable than a bunch of open code to do the same thing. No, it is not a good solution to the problem; you ignore my earlier comments. English or latin text is not comprised of the sole ASCII characterset; it contains characters outside this set (quoting other languages, names, imported words for example).
- flogic 9y agoI think the general problem with any automated solution is that there is so much room to game them. For instance, I could selectively replace a few visible ASCII characters with non-ASCII look a likes. Then, the investigators just need to see which characters are missing. Even with the OCR option you could selectively add typos.
- dreamfactored 9y agoHow about back and forth through a translation app or two?
- hyperpape 9y agoHow is that different from his option 5? I suspect he put it low on the list because it could be a cat and mouse game of trying to anticipate all the potential information leaks.
- 3pt14159 9y agoHi, I'm the author. I was kinda on the fence, but I was considering my target: Journalists. A journalist may not notice simple differences like an extra space here or there when reading, but probably wouldn't retype a double space. I agree that this should be automated in some way, but it's a bit of an arms race.
- nine_k 9y agoWhat kind of arms do you envision for normalizing text down to Latin-1, or even ASCII, while normalizing any whitespace to single space characters? There is a known variant when every subscriber of a confidential text receives a slightly different copy with the same meaning. But it's much harder to implement, and does not scale.
- laumars 9y agoNot that much harder to implement. You could automate that too with some basic text substitution. Eg replacing various instances of "and" with ampasand or plus sign. You could also vary joined words like "without" / "with out", and alternative between the types of quotation marks used (as there are several in unicode). And this is without breaking into more intelligent heuristics where you swap out synonyms ("more intelligent" because you'd need to be careful not to alter passages that need to be kept verbatim, like quoted text or where a synonym might alter the context of the sentence.but with a little care I think that is achievable as well)
- blackkettle 9y agoI'm curious as to why you think a linter tool for sanitizing the text is less effective than the other measures you describe. As long as the target problem is zero-width characters, this seems like it should be more effective than all but #1. I would agree if you include the synonym or other lexical fingerprints.
- 3pt14159 9y agoYou might be interested in reading my update: https://www.zachaysan.com/writing/2018-01-01-fingerprinting-update https://www.zachaysan.com/writing/2018-01-01-fingerprinting-... Humans make fixes that are hard to codify in programming. I know it isn't perfect, but with my audience in mind (journalists) I thought it was probably safer than something automatic.
- Nomentatus 9y agoCutting and pasting using Ctrl-Shift-V in Libre Office does the trick. (Then select unformatted.) You still have to manually eliminate the spaces.