10 ms·
Show HN: I made a Chrome extension to reveal zero-width characters
- gervase 9y agoIt would be a nice enhancement to do so automatically (rather than on-click), but at least for me, this serves the purpose.
- kawsper 9y agoYes I agree, or maybe just update the app-icon if it detects presence of zero-width characters, and then you can click to show them.
- thrownaway954 9y agonot only that, but display a banner warning at the top of the page that tells the user the the page contains zero-width characters.
- kerkeslager 9y agoI agree with other users who said that it would be nice if this happened automatically instead of on click--I think after the initial novelty wore off, I'd probably forget to click it. It might be cleaner to just strip the non-printing characters and display a number of characters stripped, similar to how ad blockers display the number of ads blocked.
- chpmrc 9y agoI like the adblock analogy, might work on it, thanks.
- Torn 9y agoPlease do, and publish as a real extension - I'd definitely use it
- needToSay 9y agoBut that would be a performance killer or what?
- kerkeslager 9y agoI'd rather my browser did the right thing slowly than the wrong thing fast.
- jannes 9y agoHow would you analyse content that's asynchronously loaded and inserted into the DOM? I think DOM mutation observers would be a huge hit on performance.
- kerkeslager 9y agoI'd rather my browser did the right thing slowly than the wrong thing fast.
- Aditya_Garg 9y agoI think this extension has a very specific use case, particularly that you want to leak confidential information and you don't want it to trace back to you. As others have mentioned, I could see this being used regularly by someone who plays Eve
- Someone1234 9y agoIt might make more sense as a clipboard filter. If you're an English speaker that doesn't often handle languages which gain from zero width characters you could just have a listener scan your clipboard for zero-width characters, silently strip them, and then re-populate the clipboard. Although I've been thinking about building a clipboard filter for Windows a lot lately, I'm getting tired of copying text to the address bar or notepad to strip text formatting information.
- tomerico 9y agoCtrl+shift+v pastes without formatting
- enzanki_ars 9y agoNot on all apps though. Office apps do not respect that shortcut convention and require clicking on the "Paste Without Formatting" button.
- MereInterest 9y agoIn Office, you can hit Ctrl after pasting, then "t" to select paste without formatting, which is nicer than switching from keyboard to mouse. The obnoxious thing is that, if the source text did not actually have formatting, then there is no menu that opens by Ctrl, and "t" is instead inserted verbatim. Wrecks havoc with my muscle memory, that I need to keep track of where I copied text from when pasting.
- enzanki_ars 9y agoAs much as that would be a helpful shortcut to remember, I know I won't. As a result, I just changed the default paste settings in Office.
- irrational 9y agoThat just removes formatting. zero-width characters are not formatting so they wouldn't be removed. They are as real as the letter A - you wouldn't want ctrl+shift+v to remove all A's from your text, would you?
- IloveHN84 9y agoWhat about Firefox?
- inetknght 9y agoI, too, disprefer Chrome.
- overcast 9y agoYou made that word up!
- patrickaljord 9y agoAll words are made up so that's ok.
- jaclaz 9y ago>All words are made up so that's ok. Not really, JFYI ;): http://www.marriedtothesea.com/030807/dictionary.gif http://www.marriedtothesea.com/030807/dictionary.gif
- chanz 9y agoYou can use this addon: https://addons.mozilla.org/de/firefox/addon/foxreplace/?src=search https://addons.mozilla.org/de/firefox/addon/foxreplace/?src=... Apply an autoload rule with these settings: Name: zero length replacement HTML: Output only URL: <entering no url matches all sites> Replace: (\uFEFF|\u200B|\u200C) Type: Regular expression With: <span style="background-color:#F00 !important">😳</span>
- ric2b 9y agoThank you!
- lainga 9y agoOP, does this mess up ZWJ emoji (like any of the skin-tone-modified emoji)?
- chpmrc 9y agoSorry I'm afraid I don't understand the question.
- jedanbik 9y agoMeaning, does it mess this up? https://emojipedia.org/zero-width-joiner/ https://emojipedia.org/zero-width-joiner/
- lainga 9y agoIt's a pretty obscure edge case. You could retain ZWJs when they were between two Unicode points with the Symbol or Emoji Symbol classes, but on further reflection, it would probably be more honest to just strip the ZWJs everywhere and decompose the emojis. EDIT: also what jedanbik said
- ascorbic 9y agoHardly obscure. ZWJ emoji sequences are really common now.
- ubernostrum 9y agoIt's how a lot of newer emoji work. Suppose, for example, that you want the emoji "Man Facepalming Medium Skin Tone". There is no single Unicode code point for that. Instead, it uses five code points: U+1F926 FACE PALM U+1F3FD EMOJI MODIFIER FITZPATRICK TYPE-4 U+200D ZERO WIDTH JOINER U+2642 MALE SIGN U+FE0F VARIATION SELECTOR-16 The first two compose to make the "facepalm" with desired skin tone. The last two make "man". The zero-width joiner composes the first two and last two together, so that the whole sequence renders as a single emoji. Here's a list of emoji which use the ZWJ this way: https://emojipedia.org/emoji-zwj-sequences/ https://emojipedia.org/emoji-zwj-sequences/
- gormz 9y agolol you didn't even give anyone a chance. I saw someone asking about this in the comments from the article this morning.
- chpmrc 9y agoI was really intrigued by this zero-width characters thing haha
- EamonnMR 9y agoI'll be sure to add a link to http://zerowidth.space http://zerowidth.space
- chpmrc 9y agoThank you! I appreciate it.
- laumars 9y agoIt feels like you're trying to make a point on that site but I cannot fathom out what that point was. Are you saying zero width spaces are bad? Are you educating people that they exist? It's not all too clear what your point is. Also I disagree with the following: > If you're working on a website there's a good chance that someone will want to copy/paste it, automate using it, etc. Zero-width spaces will create no end of hassle to those people. I could work around that with literally just 1 line of code: newString = replace(originalString, "", " ") But these days many HTML parsers will tidy up the output for you so depending on the frameworks you're using, you might not even need to use the above line of code.
- EamonnMR 9y ago> Zero-width spaces will create no end of hassle to those people. It's up to you to decide what to do with that information. > newString = replace(originalString, "", " ") If I could have registered "zerowidth.character" I would have. You've got to handle different characters and such. I should say that the site is mainly a joke because we had to use a particular tool which had a bunch of zero width spaces in it, and it got on our nerves.
- laumars 9y ago> If I could have registered "zerowidth.character" I would have. You've got to handle different characters and such. Ahhh I didn't get that from the site. I thought it was focused on one specific type of zero width character > I should say that the site is mainly a joke because we had to use a particular tool which had a bunch of zero width spaces in it, and it got on our nerves. That's fair enough. :)
- TheAceOfHearts 9y agoA bit tangential, here's a few unix utils you can use to inspect text: `vis`: display non-printable characters in a visual format `cat -e`: Display non-printing characters and display a dollar sign (`$') at the end of each line. `hexdump -c`: Display the input offset in hexadecimal, followed by sixteen space-separated, three column, space-filled, characters of input data per line. `od -a`: Output named characters.
- pavelbr 9y agoIs `hexdump` not `xxd`?
- lainga 9y ago`hexdump` is part of util-linux, `xxd` is not. They do the same thing though, up to formatting differences.
- dotancohen 9y agoAnd on Debian it's called `hd`.
- gkya 9y agoxxd is part of vim IIRC.
- ballenf 9y agoIt's starting to seem like the universe has some fundamental order for things that we can escape temporarily, but that are inescapable in the long run. 1. Photo and video evidence was a game-changer for establishing facts and chronologies. As we've seen, it's becoming harder and harder to distinguish fake photos and videos from real. It's not a stretch to predict a time when we'll only be able to make probabilistic statements about the veracity of photos or videos. (E.g., "60% liklihood of being undoctored.") This is probably already true about photos, although the expense of making a perfect fake is still pretty high, in terms of expertise. I'd argue the "natural state" is one where word-of-mouth and first-hand accounts are the most authoritative evidence we can have (other than physical evidence like DNA left behind). And even physical evidence left behind can't tell us what the person did while there or how an event transpired. 2. Tracking communications. In the digital age, we've some people have come to assume that all digital text is untrackable and anonymous. Your "11001110" is the same as mine. Historically, it was pretty difficult to transcribe information without leaving traces of the origin of that info. These zero-width characters plus all the other text fingerprinting methods, and ubiquitous tracking in communication logs make it nearly impossible, again, to communicate with others without leaving a trail. And then there's the writing style analysis which makes it tough to write anything without leaving telltale fingerprints. So, I'm proposing that we are returning to the "natural state" of things. Probably overstating things a bit, but still an interesting thought to consider.
- dctoedt 9y ago> ... the "natural state" is one where word-of-mouth and first-hand accounts are the most authoritative evidence we can have (other than physical evidence like DNA left behind) First-hand accounts aren't always reliable. https://en.wikipedia.org/wiki/Eyewitness_testimony https://en.wikipedia.org/wiki/Eyewitness_testimony
- andrewmcwatters 9y agoI think they're proposing more that these accounts are authentic, not necessarily reliable. It's common sense that eyewitness testimony isn't 100% reliable.
- boramalper 9y agoThe project is clearly motivated by Be careful what you copy: Invisibly inserting usernames into text[0] (posted 12 hours ago on HN), and many people here might think zero-width characters (or rather, any esoteric Unicode characters) are the only (or the prominent) way to watermark texts, which is wrong. The whole topic -watermarking- is worth at the very least a lengthy blog post, and maybe even several articles, so keep in mind that this comment is just a very brief introduction to different methods (of watermarking)[x]: (from trivial to more complex ones) ---- 1. Using Invisible Characters The linked[0] article basically talks about a very basic version of it, which should be enough to get the idea. Of course, more sophisticated techniques will use more than just two characters, and will take the position of each invisible character into consideration while encoding & decoding watermarks, ensure it's uniformly distributed throughout the paragraphs etc. Can be defeated by simply removing invisible characters. ---- 2. Using Unicode Characters That Look Alike The same working mechanism as IDN homograph attack[1]. Can be defeated by simply removing "out-of-place" letters/characters, after determining the language of a given text/paragraph/sentence etc. ---- 3. Using Unicode Equivalence > Code point sequences that are defined as canonically equivalent are assumed to have the same appearance and meaning when printed or displayed. For example, the code point U+006E (the Latin lowercase "n") followed by U+0303 (the combining tilde "◌̃") is defined by Unicode to be canonically equivalent to the single code point U+00F1 (the lowercase letter "ñ" of the Spanish alphabet). https://en.wikipedia.org/wiki/Unicode_equivalence https://en.wikipedia.org/wiki/Unicode_equivalence Can be defeated by simply normalising the text, and line endings! ---- (Now it's getting harder!) 4. Changing the Layout of Documents You can change (the rendering of): (a) the margins (b) the ligatures (c) the space between (i) specific characters [kerning] (ii) consequent words/lines/paragraphs to embed fingerprints. This is especially dangerous as documents are often leaked by taking screenshots or photocopies, which is secure against Unicode attacks, but not of these. Can be defeated by copying the plaintext, and pasting it to a text editor, and applying steps 1, 2, 3. Also, bear in mind that if you can still -unintentionally- leak information: (a) when you use a document editor (LibreOffice Writer, Microsoft Word) as the layout engines might act differently depending on your software version, platform, file format etc. (b) paper size (A4, US Letter, ...) ---- 5. Substituting with Synonyms If characters can be replaced by their equivalents, why not replace words or sentences even? Words can be substituted by their synonyms based on an algorithm that can create fingerprints accordingly, and I presume even sentences can be rephrased with the latest advancements in AI/ML. Also, some of the substitutions will be unintentional: a scribble on a piece of leaked document can also leak information about its leaker (different spellings in American and British English, ways of writing date & time, decimal separators etc). Can be defeated by paraphrasing. ---- The list can probably extended even further but this was all I could remember on the top of my head. =) [0]: https://news.ycombinator.com/item?id=16749422 https://news.ycombinator.com/item?id=16749422 [1]: https://en.wikipedia.org/wiki/IDN_homograph_attack https://en.wikipedia.org/wiki/IDN_homograph_attack [x]: Not that I'm working in a related field, but I researched watermarking & fingerprinting techniques for a similar but much more extensive project for journalists/whistle-blowers to detect fingerprinting/watermarking in documents. Let me know if you are interested and we can collaborate!
- srebalaji 9y agofirefox ?
- chanz 9y agoYou can use this addon: https://addons.mozilla.org/de/firefox/addon/foxreplace/?src=search https://addons.mozilla.org/de/firefox/addon/foxreplace/?src=... Apply an autoload rule with these settings: Name: zero length replacement HTML: Output only URL: <entering no url matches all sites> Replace: (\uFEFF|\u200B|\u200C) Type: Regular expression With: <span style="background-color:#F00 !important">😳</span>
- kbd 9y agoIf it's a button to click anyway, couldn't this just be a bookmarklet?
- chpmrc 9y agoYep, but it was also a way to learn how to build and publish a Chrome extension.
- username223 9y agoDoes anyone remember how many years of pain were caused by different line endings -- "\n" vs. "\r" vs. "\r\n"? Unicode is 10x that, and the pain is just beginning.
- yuchi 9y agoDid you mean ^10?
- username223 9y agoProbably, since it's an omnishambles on a scale I won't pretend to understand. Eight-fingered and two-thumbed humans can't type it, but they've managed to make it a joke (see zalgo text), and it only gets more ridiculous with time (see how flags are done, or emoji skin-tone modifiers). It's only a matter of time before Metafont will be easier to read and write than the hieroglyphics we're supposed to call "text."
- jacquesc 9y agoAnything like this for VSCode? Just adding option-space accidentally in a ruby file (after an `end`) will cause ruby to explode and not tell you anything. Randomly deleting code until it runs was my only fix.
- iyrkki_odyss 9y agoThere is an extension called Highlight Bad Chars. I installed it after reading this
- jacquesc 9y agoBrilliant, thanks!
- kragen 9y agoHave you thought about revealing things like non-breaking space, thin space, hair space, and so on? I think at least Fecebutt replaces non-breaking spaces with regular spaces in comments, perhaps precisely to frustrate such steganography. (Ironically, last I recall, they do preserve the difference between double and single spaces, e.g. after periods, which is even more invisible in HTML.) I second the recommendations of vis, cat -e (or cat -vte), and od -a. I didn't know about hexdump; I always used xxd.
- sam1902 9y agoI made two python script to encode and decode string into and from zero width characters: Encoder: https://pastebin.com/DGansW69 https://pastebin.com/DGansW69 Decoder: https://pastebin.com/ZVdvjnZc https://pastebin.com/ZVdvjnZc I know it's pretty basic stuff but it does the job. The encoder outputs both in stdout and into a file so you can copy/past it more easily (from Sublime Text for example)
- alex256148 9y agohttp://acccurrent.info/news/anti-trump-judge-shows-up-drunk-misses-214-days-of-work-watch-what-she-says-when-called-out/ http://acccurrent.info/news/anti-trump-judge-shows-up-drunk-...
- tenryuu 9y agoSo does it show the ZWJ within emoji combinations?
- SuperGoodJared 9y agoI wonder if some sort of clipboard tool to detect and warn of sneaky looking things might be even more convenient.
- flashman 9y agoTime to read the web exclusively in ASCII with UTF-8 characters inserted using their codepoint ~WHITE SMILING FACE~ ~THUMBS UP SIGN~
- beautifulfreak 9y agoI wonder if this could be used on Tor to get a user ID.
- mrswilliam 9y agoincase you all want to increase your credit score, spy on your partner facebook,whatsapp ,instagram to find out if they are cheating , go ahead and contact reliablehacker 2017 AT g mail DOT com they are awesome now i can get my loan , tell them i refered you so i can get a discount on my next job with them ...thank me later, they can do almost about anyhing