16 ms·
Don't use text pixelation to redact sensitive information
- FR10 5y agoIts curious that in tv/film people's identity is censored by blur/pixelation/black bars on eyes and, documents are always censored by black bars. I dont recall seeing pixelation used for text, then why do we mostly go for pixelation in the first place when wanting to omit info on screen.
- computerfriend 5y agoSimilarly, never use it to redact a QR code. https://www.freecodecamp.org/news/lets-enhance-how-we-found-rogerkver-s-1000-wallet-obfuscated-private-key-8514e74a5433 https://www.freecodecamp.org/news/lets-enhance-how-we-found-...
- Moru 5y agoThis wasn't much because it was blurred but because they showed a few images unblurred of parts of the QR code and the plain text code. The rest was brute forced and reconstructed from the QR code format.
- kderbyma 5y agoMay I add...Rasterise your documents after redaction if using digital tools to manually redact. I have seen many a doc with history still included because they were using illustrator or something similar
- anonymouse008 5y agoRespect. I do one layer more — black box, print out, then scan.
- bonyt 5y agoI’ve seen even this fail once, although it was unusual circumstances. Black box on what turned out to be slightly off-black text using a PDF highlighting tool. Not the ideal way to do it. When printed and scanned you could just barely read it if you had a monitor with a bad viewing angle or adjusted the contrast. A paralegal caught it before it went out.
- uxamanda 5y agoYou can also screenshot as an interim step, though not quite as secure as this. :-)
- dang 5y agoUrl changed from https://portswigger.net/daily-swig/new-tool-can-uncover-redacted-pixelated-text-to-reveal-sensitive-data https://portswigger.net/daily-swig/new-tool-can-uncover-reda..., which points to this.
- mlissner 5y agoMy organization built a similar tool that can find bad redactions caused when people just use a black rectangle on top of text in PDFs: https://free.law/projects/x-ray https://free.law/projects/x-ray Very fun project. Lots of problems out there.
- steelstraw 5y agoInteresting. They have the right idea (using a black rectangle) but in the wrong program. I can see how this could trip up non-technical people.
- pkaye 5y agoSeems like even the size of the rectangle would give hints on what it is redacting. For example "Yes" has a bigger box vs "No".
- Mountain_Skies 5y agoYes, it's been that way long before PDFs. Simply knowing the potential words, often names, that could appear in a document, gives those with the redacted documents a chance at determining what has been hidden based on size. This might be part of the reason why when declassifying documents, the redactions end up being more of a sentence than is needed. The extra buffer of hidden words gives some additional protection to what needs to be redacted.
- praptak 5y agoA malicious "redacting" algorithm submitted to the underhanded C contest used a similar idea, just on lower level. PNG allows ASCII numbers, so flipping all digits to 0 creates a pixel which is graphically "masked" but leaks information about the original pixel: "000" means the value was larger than 99.
- anderskaseorg 5y agoThat’s PPM format, not PNG. http://underhanded-c.org/_page_id_17.html http://underhanded-c.org/_page_id_17.html http://netpbm.sourceforge.net/doc/ppm.html http://netpbm.sourceforge.net/doc/ppm.html
- guidovranken 5y agoThe bottom line is that when you need to redact text, use black bars covering the whole text. Never use anything else. That actually may not be enough if you're applying the black bar to compressed image data like JPEG because compression artifacts surrounding the black bar can be leaking information about the covert data.
- dpark 5y agoMight be interesting to test how plausible that is. How likely is it that a human doesn’t see the artifacts but they are leading enough info to reconstruct the underlying data.
- colanderman 5y agoI think it's more likely the JPEG blocks which straddle the box edges simply don't cover any meaningful part of the text. 12 pt font at 96 DPI is 16 pixels tall, meaning 50% of the vertical height of a line simply wouldn't fall into the blocks straddling the edges of a line-height box. You'd get ascenders and descenders but not much else. Tops of numbers or all-caps I think is best case. Though, web images now are being served in higher resolutions (200+ DPI) for "retina" displays, and scanned images are generally 300 DPI, in which case you'd be lucky even to get ascenders and descenders. I'd be curious to give it a try though. If Facebook memes are any indication, many humans are totally oblivious to near-unreadable levels of artifacting.
- nerdponx 5y agoSave the edited version to PNG first, and then back to JPEG?
- lmm 5y agoNot necessarily good enough. In principle you need to either get the raw original, or black out every macroblock that ever contained any sensitive information.
- colanderman 5y ago
- falsenapkin 5y agoI want random-text-overlay-pixelation (or some lookalike) as a feature instead of plain-text-pixelation. Giant black boxes everywhere is quite the distraction, whereas blur/pixelation is not, so just make the underlying data random.
- redisman 5y agoThat sounds extremely confusing
- falsenapkin 5y agoIdeally the user wouldn't have to do anything. Whatever your editing tool is would have some "text blur" mode which functions and looks like normal blur but the end result is blurred garbage.
- andai 5y agoI think as far as culture goes it would be simpler to promote the idea that black bars are the only way to do it, than that you need to make sure you're using a special software that knows how to securely pixelate text. It is a cool idea though, and text-shaped pixelation is much more satisfying than random noise. (And if someone's cheeky enough to decode it, they will be very disappointed ;)
- falsenapkin 5y agoYou're definitely right.. but I still want my overcomplicated blur ;)
- redisman 5y agoOh I thought you meant it would replace black bars with random not blurred text lol
- klipt 5y ago> Giant black boxes everywhere is quite the distraction Opaque gray boxes then?
- dinkleberg 5y agoI wonder if a better (albeit slightly more involved) process to make blurred out sensitive inputs for shared content would be to: 1) add a block over the text, but use the same color as the form in put 2) add similar looking, but fake text on top of the block 3) apply the blur/pixelation. This way you keep the nicer aesthetics but you get the benefits of using the black box approach. Probably overkill for many cases, but when I create any sort of content I like to keep it looking fresh.
- 8note 5y agoFor somebody reviewing the content, how do they know that you applied this to decide that it shouldn't be up because it doxes somebody?
- dheera 5y agoI wrote about exactly this in 2007: https://it.slashdot.org/story/07/01/07/1352242/blurring-images-not-so-secure https://it.slashdot.org/story/07/01/07/1352242/blurring-imag... which was republished in 2014 in Gizmodo: https://gizmodo.com/why-you-should-never-use-pixelation-to-hide-sensitive-t-1610206920 https://gizmodo.com/why-you-should-never-use-pixelation-to-h... See also related HN comments: https://news.ycombinator.com/item?id=8078747 https://news.ycombinator.com/item?id=8078747
- J253 5y agoTo prevent some of the issues with blurring or compression leaking sensitive info, one simple workaround I’ve used is to just put black boxes over the text as usual in any program and then take a screenshot and save it out from there. No accidental history or compression leaking.
- Gigachad 5y agoTbh I think there needs to be some dedicated tool for censoring images. Even with this method there is a small chance that it isn’t 100% opaque. I have seen several images where you can pull them in to gimp and adjust the color levels until even the slightest color difference becomes blown out and you can read the censored text. It’s just too easy to mess this stuff up.
- addaon 5y agoOne thing that still leaks is the exact spacing between words adjacent to the black boxes. This is a particular concern one only one or two words are blacked out -- for example a name. With a known font (with known character widths and kerning tables) and a known text engine applying those kerning rules, there's often very few combinations of letters that give the exact pixel (or subpixel) width blank space.
- bobsmooth 5y agoSo blurring can be reversed and black bars can be cracked, is there any way to securely redact a document?
- fortran77 5y agoI always change the text and _then_ pixelate it. So if someone goes through the exercise of reversing the pixelization, they'll get the wrong text.
- Tempest1981 5y ago"Ah hah -- It was Ms. Lorem Ipsum!"
- est 5y agoOr just randomize text pixelation to troll people.
- eatbitseveryday 5y agoConvert PDF to jpg or png then use a black box and save. Gets burned into the image.
- aaronsimpson 5y agoEspecially in the age of AI/ML, doing something like this today is just BEGGING to get algorithm’d into oblivion.
- octoberfranklin 5y ago... unless you do it using totally random pixels, derived from a RNG rather than from material being redacted.
- baq 5y agopixelate random characters to give a realistic look
- Proven 5y ago
- Tempest1981 5y agoI wonder how much harder it is to un-redact Kanji characters?
- akg_67 5y agoWhat is a good safer way to redact digital documents and pictures. I typically cut the area I want redacted instead of overlaying with solid color.
- srg0 5y agoI usually convert PDF to raster and draw over. White or light gray box overlay will save ink/toner for everyone and is just as opaque as a black box. I still do pixelize with huge pixel size when information is not that important. I think it better conveys that there was something there, and redacted areas look more organic.
- Markoff 5y agoPixelation, just use less squares, I pixelate to one line of huge squares. I would not be so sure about cutting the area, don't forget to print it (to PDF) instead saving it.
- rurcliped 5y agoRedaction can be needed for a document, but redaction can also be needed for a video (car's license plate, credit card number, etc.). As far as I know, there's no video editing software that recommends black bars and, for example, the official Adobe documentation at https://helpx.adobe.com/premiere-pro/using/masking-tracking.html https://helpx.adobe.com/premiere-pro/using/masking-tracking.... seems to recommend their Masking and Mask Tracking features. Black bars may look very ugly in a video. Still, are video editing products recommending a process that has a high risk of leaking sensitive data? There might be reasons that attacking redaction in a video is harder than attacking redaction in a PDF. However, maybe it's actually easier in some cases, e.g., with several similar frames, the attack could take advantage of averaging across frames. Unfortunately I don't see anything at https://hackerone.com/adobe https://hackerone.com/adobe that could get someone a bug bounty for researching this.
- jiehong 5y agoIn a video, a bar with a color matching the rest of the object would be less noticeable; it would just appear as if it was blank.
- democra 5y agoWouldn't radomized pixelation without using the actual textual data in the background be enough? A tool where you can choose the background color and the text color. The pixalation tool then overlays the blur effect with random characters.
- can16358p 5y agoThat should work, given than the result doesn't have any actual input from the original data (except width of the text) and completely opaque over the actual text.
- rikschennink 5y agoI imagine that would be the same as a black fill, just in a different color with noise. I've used the original image data as a fill, but scaled down, so you get a mosaic effect. Then I randomise the tiles in that mosaic and then I blur. The result is that it seems like the original data was redacted but in reality the original data has been scrambled to such an extent that it can no longer be retrieved.
- gurchik 5y agoTangentially related but I immediately recognized the person in the video in the article as "AltF4" who has done a lot of work in the Super Smash Bros. Melee scene. He has worked on slippi[1], created a Python API library for the game[2], and coded a bot[3] to demonstrate the library which professional players find very difficult to beat based on its inhuman reaction times (as you'd expect). I've always wondered what some of these Melee players do as a real job and I guess in this case I found out by accident (in hindsight, the name of the company is in his Github bio, which I had never checked before). 1. https://slippi.gg/about https://slippi.gg/about 2. https://github.com/altf4/libmelee https://github.com/altf4/libmelee 3. https://www.youtube.com/watch?v=Gb3v6uPuKic https://www.youtube.com/watch?v=Gb3v6uPuKic
- alexb_ 5y agoAltF4, Dan Salvato, Fizzi, and UnclePunch are all absolute legends. All of the things they've done to improve Melee are staggering. Amazing work by all of them.
- pc86 5y agoFor anyone else wondering, this is the person AltF4, no relation to AT0TA or Blackglasses of TheAltF4Stream on Twitch (that I know of, anyway).
- praash 5y ago> [The challenge image] has a curious bit of coloring in it. What gives? Shouldn’t it just be black and white since the text is black? > I’m actually not 100% sure why this happens (and sometimes doesn’t), but it’s an artifact of the rasterization process when text is rendered to screen. This is a brilliant technique called subpixel rendering.[0] A "typical" computer screen's pixels are split into three columns of red, green and blue, instead of lighting up as a solid square. Using color fringing at non-pixel-aligned edges of characters can effectively triple the perceived horizontal resolution. Wrong assumptions about the pixel layout will show up very badly, as shown in [1]. HiDPI displays (also mobile, not just 4K) get no real benefit from this and usually have a more complex subpixel layout anyway. I remember seeing old iPads usually showing awful fringes as subpixel rendering was enabled without considering the actual orientation of the display. [0] https://en.m.wikipedia.org/wiki/Subpixel_rendering https://en.m.wikipedia.org/wiki/Subpixel_rendering [1] http://www.lagom.nl/lcd-test/subpixel.php http://www.lagom.nl/lcd-test/subpixel.php
- praptak 5y agoI remember that Windows had a calibration tool which let you tune the system params for subpixel rendering to look good on your screen. It also switched it off globally when the "magnifying glass" tool was active.
- gambiting 5y agoIt still does - for fonts. It's called "ClearType Text Tuner" and it's part of Windows. It allows you to select one of several different subpixel rendering types to pick one that looks best on your screen.
- 542458 5y agoIrritatingly, some windows applications implement their own subpixel rendering. As somebody on a 1440p BGR panel, those applications become extremely obvious to me as the aliasing is exactly backwards to what it should be. One irritating example is the web version of MS Word, as it doesn’t use the browser font rendering and instead implements it’s own (in canvas I think?) - giving all text eye-straining fringes on my display. Google Docs has the good sense to stick to black-and-white anti-aliasing at least.
- GSGBen 5y agoDoes anyone know if significant Greenshot blur is irreversible, or know of any cases where it was reversed (and which level)?
- riidom 5y agoBlurring is a lot easier to un-redact, and for much longer time, than pixelation.
- rikschennink 5y agoI'm the developer behind Redact.Photo [0], instead of blurring or pixelating the image, I've added an additional step to improve data security. See "how it works" section on the page. 1. Image is scaled down to create a mosaic effect 2. Pixels are moved around a random offset of x and y axis 3. Blurring is applied Adding black bars over information will always be better but this does result in more smooth redaction that I think cannot be undone because of the randomisation step. [0] https://redact.photo https://redact.photo
- bagels 5y agoI tried it out. It still leaves a lot of the original information in the redact. Is it enough to reverse? I can't say it definitely is, but I would not be willing to risk something important with it.
- rikschennink 5y agoI'm not sure what you tested with, but it might "seem" that the original information is there? Only the original color data remains, but the detail is gone and each pixel position behind the redacted area is randomised/mixed. This randomisation step also overwrites pixel information so there is data loss as well. Because it's then blurred it looks like the info is blurred only. But if you'd be able to reverse the blur you'd end up with pixel noise. I find it hard to believe that that noise could be reverted.
- bobsmooth 5y agoQuick and it appears secure! I would add a blur around the edge of the boxes so there isn't a sharp seam.
- rikschennink 5y agoThanks for trying it out! Excellent idea, will see if i can implement that :)
- can16358p 5y agoLovely article, well explained. Quick note: when giving examples of variable width and monospace fonts, the both look variable width. No monospace font is displayed (mobile Safari iOS 15.3.1)
- samwhiteUK 5y agoSame on Firefox on Linux
- daniel_iversen 5y agoWouldn't an algorithm just like this one also be able to work on really blurry (i.e. far away zommed in) photos too? Especially with a huge corpus of trained content?
- privacyonsec 5y agoavg coder: "let's use machine learning !" bishopfox: yangi !
- pabs3 5y agoWhen redacting individual words, you probably don't want to use the same size black box as the rendered word, since someone might be able to figure out what word it was based on the length of the box and the document font. Also, consider removing metadata from digital files using mat2: https://0xacab.org/jvoisin/mat2 https://0xacab.org/jvoisin/mat2
- tus666 5y agoMaybe replace the text with lorem ipsum before pixelation? It will look roughly the same without leaking anything useful.
- werber 5y agoText pixelization is only useful in highly visible projects where you want people to be engaged to the point they figure out what the text is. Viral marketing, film, etc. this is a great tool, with very few valid applications.
- jakub_g 5y agoSee also: https://matzjb.se/2015/07/26/deconstructing-swirl-face/ https://matzjb.se/2015/07/26/deconstructing-swirl-face/
- transitivebs 5y agoReplace the text first and then use text pixelation :)
- usrbinbash 5y agoPixelation is one of these hilarious instances of where a needlessly complicated approach is used to do something that can be done much quicker, easier, and, as it turns out, way more secure. Wanna redact text? Put a solid bar over it. There, done. No takesies-backsies from that one.
- kawsper 5y ago> Put a solid bar over it. There, done. No tacksies-backsies from that one. And this is not so good advice if you're using software that supports layers and your solid bar is in another layer than what you're trying to hide. Wanna redact text? Delete it.
- usrbinbash 5y ago> Delete it. Redaction and Deletion are not equivalent. There are many instances where a visible redaction is required rather than a deletion.
- zauguin 5y agoSure, but you should still delete it. You can always add a bar at the old position afterwards. Then it's still a visible redaction, but you don't end up having a hidden copy of the unredacted text.
- signal11 5y ago> if you're using software that supports layers ...then really you should know how to flatten? Not understanding the tools you use for sensitive tasks like hiding/redacting data is a recipe for disaster.
- jrm4 5y agoSeems like "flatten" ought to be a trivially easy mechanism to implement in software somewhere, and maybe even as a standalone application for the less familiar. Basically just a nicely trimmed screenshot, no?
- elcomet 5y agoThe only good way to redact: Print, cut with scissors, scan.
- ajnin 5y agoAnother related piece of advice is that you should not use blur to hide sensitive information because blur is actually a mathematically reversible operation ! It's quite surprising at first but the information is still there, it has just been spread out. If you know the shape of the kernel and have enough resolution then you can unblur the image. You can even apply this process (called deconvolution) to physical blurs like imperfections in telescope optics. In practice however bit resolution is often too low to have good results. You should also be careful with video, since it contains many frames it might be possible to reverse rougher pixelations or blurs than would be possible with a still image. There exist various superresolution algorithm that can extract high-resolution images from multiple low-resolution video frames.
- jansan 5y agoI honestly doubt that deconvolution can fully restore an image after blurring. Do you have any links that supports your claim?
- Dan_- 5y agoSearch for 'swirl face'
- jansan 5y agoA blur filter is very different from a distortion algorithm (like the swirl filter). For a distortion algorithm you just have to put the pixels back into their original position.
- makeworld 5y agoMy understanding is that Gaussian blur is technically lossless, all the data is still there in the decimal points. In practice, pixel values are truncated to 8-bit integers rather than arbitrary precision decimals, but because the blur is lossless to begin with you can sometimes deblur and retrieve enough information for whatever you were trying to do.
- 5y ago
- jansan 5y agoAnyone remembers the Canadian paedophile who used a spiral effect to hide his face in photos? Simply reversing the spiral effect got the police a very clear image of his face and led to his arrest. https://www.theguardian.com/world/2008/aug/15/thailand.internationalcrime https://www.theguardian.com/world/2008/aug/15/thailand.inter...
- crispyambulance 5y agoNever say "never". It's perfectly fine to use pixelation in the vast majority of scenarios where you are just trying to NOT draw attention to details that aren't part of your message. If someone gets nerd-sniped enough to actually sit down and spend a few hours deconvolving someone's name or email address, that's not the end of the world unless it's a very exceptional piece of information. Use your judgement. In any case, manually blacking out rectangles to hide text looks ugly.
- TillE 5y agoThis is bizarre advice. If you care enough to redact information in an image which may linger on the internet forever, you probably care enough to want to do it properly. It's often going to be sensitive personal information like an address. I mean if you don't care, why bother at all?
- d3vmax 5y agoAdd in a gauss blur as well.
- DonHopkins 5y agoWhen I implemented the pixelation censorship effect in The Sims 1, I actually injected some random noise every frame, so it made the pixels shimmer, even when time was paused. That helped make it less obvious that it wasn't actually censoring penises, boobs, vaginas, and assholes, because the Sims were actually more like smooth Barbie dolls or GI-Joes with no actual naughty bits to censor, and the players knowing that would have embarrassed the poor Sims. The pixelized naughty bits censorship effect was more intended to cover up the humiliating fact that The Sims were not anatomically correct, for the benefit of The Sims own feelings and modesty, by implying that they were "fully functional" and had something to hide, not to prevent actual players from being shocked and offended and having heart attacks by being exposed to racy obscene visuals, because their actual junk that was censored was quite G-rated. (Or rather caste-rated.) But when we later developed The Sims Online based on the original The Sims 1 code, its use of pseudo random numbers initially caused the parallel simulations that were running in lockstep on the client and headless server to diverge (causing terribly subtle hard-to-track-down bugs), because the headless server wasn't rendering the randomized pixelization effect but the client was, so we had to fix the client to use a separate user interface pseudo random number generator that didn't have any effect on the simulation's deterministic pseudo random number generator. [4/6] The Sims 1 Beta clip ♦ "Dana takes a shower, Michael seeks relief" ♦ March 1999: https://www.youtube.com/watch?v=ma5SYacJ7pQ https://www.youtube.com/watch?v=ma5SYacJ7pQ (You can see the shimmering while Michael holds still while taking a dump. This is an early pre-release so he doesn't actually take his pants off, so he's really just sitting down on the toilet and pooping his pants. Thank God that's censored! I think we may have actually shipped with that "bug", since there was no separate texture or mesh for the pants to swap out, and they could only be fully nude or fully clothed, so that bug was too hard to fix, closed as "works as designed", and they just had to crap in their pants.) Will Wright on Sex at The Sims & Expansion Packs: https://www.youtube.com/watch?v=DVtduPX5e-8 https://www.youtube.com/watch?v=DVtduPX5e-8 The other nasty bug involving pixelization that we did manage to fix before shipping, but that I unfortunately didn't save any video of, involved the maid NPC, who was originally programmed by a really brilliant summer intern, but had a few quirks: A Sim would need to go potty, and walk into the bathroom, pixelate their body, and sit down on the toilet, then proceed to have a nice leisurely bowel movement in their trousers. In the process, the toilet would suddenly become dirty and clogged, which attracted the maid into the bathroom (this was before "privacy" was implemented). She would then stroll over to toilet, whip out a plunger from "hammerspace" [1], and thrust it into the toilet between the pooping Sim's legs, and proceed to move it up and down vigorously by its wooden handle. The "Unnecessary Censorship" [2] strongly implied that the maid was performing a manual act of digital sex work. That little bug required quite a lot of SimAntics [3] programming to fix! [1] Hammerspace: https://tvtropes.org/pmwiki/pmwiki.php/Main/Hammerspace https://tvtropes.org/pmwiki/pmwiki.php/Main/Hammerspace [2] Unnecessary Censorship: https://www.youtube.com/watch?v=6axflEqZbWU https://www.youtube.com/watch?v=6axflEqZbWU [3] SimAntics: https://news.ycombinator.com/item?id=22987435 https://news.ycombinator.com/item?id=22987435 and https://simstek.fandom.com/wiki/SimAntics https://simstek.fandom.com/wiki/SimAntics
- bryanrasmussen 5y agoyeah, sure, be careful what you give up even if you're the queen of england https://imgur.com/gallery/BGSwwqA https://imgur.com/gallery/BGSwwqA
- cityofdelusion 5y agoI don't know how universal this experience is (or why it happens), but if I step back a bit from the screen and squint/unsquint my eyes, my brain can unblur most of the pixelation effect.