11 ms·
Nice. Here's a similar personal story with a PSA that sometimes blurring is NOT sufficient. A friend of mine posted on Instagram a picture of a U.S. visa (or s
by kccqzy 6y ago
Nice. Here's a similar personal story with a PSA that sometimes blurring is NOT sufficient.
A friend of mine posted on Instagram a picture of a U.S. visa (or something similar; it was probably five years ago) to announce her trip to the U.S., and she took care to blur out sensitive information such as her passport number. But a Gaussian blur is easy to reverse and I successfully unblurred it and told her my discovery. I didn't use any specialized software; it was just Mathematica with its built-in ImageDeconvolve function with guessed parameters for the Gaussian kernel.
I personally recommend blacking out (add a black rectangle) instead of blurring, and if it is a PDF, convert to an image afterwards because too many PDF editors use non-destructive operations to add a new object instead of changing what's underneath.
- function_seven 6y agoYour advice is good, and I agree that you didn't use specialized software to reverse the blur, but this > I didn't use any specialized software; it was just Mathematica with its built-in ImageDeconvolve function with guessed parameters for the Gaussian kernel. is one of the most HN comments I've come across recently :)
- mroche 6y ago> is one of the most HN comments I've come across recently :) That gave me a laugh. I don't have any experience with Mathematica, but everytime I see it mentioned (usually on HN) I'm amazed at the sheer breadth the system is capable of. The amount of use cases and possibilities blows my mind.
- sellyme 6y agoThe top solution on this Code Golf question is possibly the most comical example of Mathematica's scope that I've ever seen: https://codegolf.stackexchange.com/questions/71631/upgoat-or-downgoat https://codegolf.stackexchange.com/questions/71631/upgoat-or...
- jtchang 6y agoThat answer is absurd yet awe inspiring what Mathematica can do.
- pmiller2 6y agoYeah, TIL Mathematica knows what a goat is, and can recognize one on sight.
- squibbles 6y agoThat statement really intrigued me. Since I like goats, I had to know how to do this. Use ImageInstanceQ[image, object], where image is the image and object is "caprine animal". [0] [1] [0] https://reference.wolfram.com/language/ref/ImageInstanceQ.html https://reference.wolfram.com/language/ref/ImageInstanceQ.ht... [1] https://codegolf.stackexchange.com/questions/71631/upgoat-or-downgoat/71680#71680 https://codegolf.stackexchange.com/questions/71631/upgoat-or...
- kiddico 6y agoThat's a very unique nerd-snipe you just experienced. https://xkcd.com/356/ https://xkcd.com/356/
- nullsense 6y agoTIL Mathematica is the GOAT.
- deleted 6y ago[deleted]
- brobdingnagians 6y agoThe other answers are also very clever and interesting. There are quite a few ways to determine whether the goat is up or down, and some are very simple.
- 6y ago
- batsigner 6y agoYet another example of someone mistaking the quality of a single person for the quality of a platform
- zwayhowder 6y agoReminds me of the Simpson's 3D episode. Professor Frink's >"Well, it should be obvious to even the most dimwitted individual, who holds an advanced degree in hyperbolic topology..."
- 1f60c 6y agoThat reminds me of this legendary comment: https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224
- function_seven 6y agoHa, I knew what that comment was before I clicked. (“Is it that rsync/ftp comment? Yup.”) ((EDIT: but it was curlftpfs, not rsync))
- toyg 6y agoThe HN equivalent of "I put on my robe and wizard hat".
- eee_honda 6y agogod, YES! i needed this reference in my life today :)
- juliansimioni 6y agoI never realized how low that comment's ID was until now. We've all said a lot since then :)
- billforsternz 6y agoMe too. The canonical HN comment, forever.
- ignoramous 6y agoYou'd love this follow-up Drew Houston and BrandonM thread shortly after Dropbox's IPO: https://news.ycombinator.com/item?id=16660140 https://news.ycombinator.com/item?id=16660140
- function_seven 6y agoThanks for that. I hadn't seen it when it was new. Now show me the thread where Steve Jobs gave a shoutout to CmdrTaco :) (https://slashdot.org/story/01/10/23/1816257 https://slashdot.org/story/01/10/23/1816257)
- gus_massa 6y agoIf it is in the installable version now, it will be in Wolfram Alpha in 5 years if you can guess the right command, and in 10 year Wolfram Alpha will just automatically select the blurred part and make a fake unblurred versions of the jpg.
- astrec 6y agoWhatever knocks this exchange off the top spot will be really special: https://news.ycombinator.com/item?id=35079 https://news.ycombinator.com/item?id=35079
- Namidairo 6y ago> and if it is a PDF, convert to an image afterwards because too many PDF editors use non-destructive operations to add a new object instead of changing what's underneath. You'd be surprised at how many times this happens on Government documents with redaction. :S
- banana_giraffe 6y agoEven when the black box is done right, sometimes there are quasi side-channel leaks of the size. The box covering a name for instances may be discoverable if there are only a few names possible, and it's a small box, meaning it's the shortest name.
- Someone1234 6y agoThat's why some even departments now have policies of printing and re-scanning retracted documents. It is dumb, but yet pretty hard to get wrong. Both MS Word and PDF have leaked redacted/removed information in the past. Wasting paper given the severity of some of these leaks is minimal cost.
- powersnail 6y agoWhy don't they convert the PDF to image and convert back? This approach seems to be a lot more efficient, and less prone to other type of human errors (e.g. missing page). Is there still an attack vector?
- mhh__ 6y agoIt's a bit like point and speak checklists on aircraft - it takes a certain amount of energy to do so you can't skip it without doing it deliberately
- Proziam 6y agoThe Japanese train system utilizes similar concepts IIRC. When I first read about this I was astonished about how effective it was [0](up to 85% error reductions)! [0] https://www.atlasobscura.com/articles/pointing-and-calling-japan-trains https://www.atlasobscura.com/articles/pointing-and-calling-j... [1] https://news.ycombinator.com/item?id=18952193 https://news.ycombinator.com/item?id=18952193
- dylan604 6y agoI once had to provide my employer copies of court documents proving something or other in order to qualify for the benefits plan I was attempting to enroll. The part of the document that contained the info they required also contained other information I did not want them to have, and I was more than irked at having to do this in the first place. I used Photoshop to draw a 99% black box as the redaction, but then using a 100% black font color typed in a nasty little message. Nobody was ever going to see it, but just knowing that if they did it would be a shock. I qualified for the package.
- dheera 6y agoYup. I wrote a blog post about this a long time ago in 2007, and it was republished in Gizmodo in 2014: https://gizmodo.com/why-you-should-never-use-pixelation-to-hide-sensitive-t-1610206920 https://gizmodo.com/why-you-should-never-use-pixelation-to-h... You can dictionary attack pixelated photos. With Gaussian kernels, besides deconvolution you can sometimes also dictionary attack them if you have the original font and if the kernel is properly normalized kernel (i.e. most gaussian blurs). Although I haven't tried, I think there may even be neural network based techniques that can perform even more effectively than a dictionary attack. Separately, if the image editing tools added sufficient random noise to their mosaic filters they might be able to thwart most of these attacks, or at least make them significantly harder.
- jacquesm 6y agoInteresting, thank you for the link. I had a hunch this should be possible but I wasn't aware that it was already proven. I used a similar trick on image recognition: turn images into a single 32 bit word by heavy pixelation and then look up a matching description. It's interesting how often that will work once you feed it with enough data. After all, that gives you 4 billion inputs mapped onto 4 billion descriptions, and plenty of those will contain the Eiffel tower with various cloudy backgrounds apparently recognized perfectly. It's a total cheat but it is funny how close that can get you to something that might be actually useful.
- Jugurtha 6y agoI wonder if you could use adaptive optimal kernels, AOK[0]? I had used this for work on multiphase flow recognition from an electrical capacitance tomography, ECT, as a proxy for void fraction. We wanted to tinker with time-frequency representations. [0]: https://pdfs.semanticscholar.org/20c2/b82eef0809df80a402f12589e78c6a58ed65.pdf https://pdfs.semanticscholar.org/20c2/b82eef0809df80a402f125...
- jacquesm 6y ago> electrical capacitance tomography Mind blown. Wow, that is very impressive.
- TwoBit 6y agoA pedophile ringleader was once caught by reversing a graphical swirl he used to try to hide his face in a picture.
- DanBC 6y agoYes. Wikipedia has an article about him here: https://en.wikipedia.org/wiki/Christopher_Paul_Neil https://en.wikipedia.org/wiki/Christopher_Paul_Neil
- ponker 6y agoNuts. He sexually abused multiple children and distributed pictures of this, but spent less than five years in jail and is now out.
- lysp 6y ago> I personally recommend blacking out (add a black rectangle) instead of blurring, and if it is a PDF, convert to an image afterwards because too many PDF editors use non-destructive operations to add a new object instead of changing what's underneath. We had a similar issue in Australia as well. Politicians phone bills are published on the government website in summary form. Someone in 2017 decided to blank out their phone numbers by changing the phone number text colour to white (same as background). End result - hundreds of politicians and former prime ministers had their phone numbers leaked. https://www.abc.net.au/news/2017-03-20/phone-numbers-of-federal-mps-former-prime-ministers-published/8370418 https://www.abc.net.au/news/2017-03-20/phone-numbers-of-fede...
- mickotron 6y agoI used to work in IT for a state based police force in Australia. Traffic reports can be requested by those involved in traffic accidents, which includes parties to the accident and their details. People used to be able to get the personal information of police officers if they were involved, intentionally or not, in a traffic accident with a police car. They would request for the traffic accident report, and that included the personal information (including home address) of the police officers in the car. I was in QA and I tested the change when it was fixed. It now includes the address of Police HQ when a police officer is involved in a traffic incident.
- nikanj 6y agoMy all-time favourite recommendation is "print, cut out the sensitive parts with an exacto knife, rescan". Firstly because it's a nice mix of analog and digital, and secondly because it's short enough to fit in a tweet - yet extremely secure.
- cgriswald 6y ago...shred cut out parts, burn remains, mix with water, encase in cement, explode, divide rubble into four parts, disperse one part each in Lake Superior, Pacific Ocean, Atlantic Ocean, and the Great Salt Lake; assume an alias, move to Alaska...
- michaelt 6y ago"Information to be withheld should be black highlighted using a tool such as the word highlighter tool like this ⬛⬛⬛⬛⬛ and then printed off. This print out should then be scanned in and saved as a PDF." Ministry of Defence redaction policy, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/302766/20130904-CN60-redaction.pdf https://assets.publishing.service.gov.uk/government/uploads/...
- ponker 6y agoThis is how military redactions have been done forever. If a soldier writes home to his family and includes classified details (“I watched the sun rise over Mt Vesuvius yesterday but today we are moving west”) the censors just cut out the text with a knife.
- irrational 6y agoWouldn’t that mean they were marching into the gulf of Naples?
- nkrisc 6y agoObviously they wouldn't want the enemy to know their troops are amphibious.
- undebuggable 6y ago> I personally recommend blacking out (add a black rectangle) instead of blurring Real life document workflows can be really tricky. What if one is required to print or photocopy the obscured document? Devastating for printer's toner or cartridge lifetime... In some cases opaque grayish rectangle does the job.
- deleted 6y ago[deleted]
- bayindirh 6y agoI generally edit the sensitive part out and match it to the background of the document looks much cleaner IMHO. However, I agree that it requires some quick hand in image manipulation software.
- pessimizer 6y ago> Devastating for printer's toner or cartridge lifetime Which could result in thousands of dollars of loss over decades. Is that really a significant concern? Charge the client for it.
- userbinator 6y agoWhite (with a black border) is fine too. Black is popular, but the goal is to make it an image with no residual data.
- bjornorn 6y agoDid the blog author actually un-blur the booking reference though? He states he tried to un-blur the barcode, was unsuccessful and then realized the booking reference was right there in the picture. Nothing about un-blurring it.
- howlgarnish 6y agoThe original image was not blurred, he simply read off the plaintext booking reference. (After first trying and failing to scan the also unblurred bar code.)
- greenmana 6y agoIf using for example Word you can conveniently just change the background text color to black. /s
- Sysosmaster 6y agoThe thing to remember here is that the only way to hide (real world) data in an image is to reduce the amount of data in the picture... a blur or swirl leaves most if not all data just in the picture (although distorted) Any filter that removes data (such as pixelate or blacking out / whiting out) can be used to safely hide this data... Just remember to also strip out any unwanted meta data (Exif-data) and do not use layers but a 'flattened' version of the picture.
- kortex 6y agoPixelation is also attackable. Generate input (e.g. GAN) and apply pixelation until it converges. Probably won't be super accurate but enough to probably ID someone. Black/delete (and flatten/rebroadcast) is the only way.
- freeone3000 6y agoI'd worry about hallucinations when applying a GAN to a pixellated image. You'll get out a face, but who's to say that it's the correct face? Lots of people look similar.
- thrwyoilarticle 6y ago>a Gaussian blur is easy to reverse That's the most surprising thing I've read today. I assumed it was destructive.
- jacquesm 6y agoIt's lossy, but not destructive, and a 'sharpen' operation is technically the same as blur but in reverse. So you won't end up pixel-perfect after doing an 'unblur' but you will be able to make out more than you could before.
- eru 6y agoIf you know anything about the probability distribution of likely inputs, it's even easier to reverse with minimal loss. Eg knowing that the input was black text on white background or a natural image (instead of eg white noise) helps a lot.
- zerd 6y agoAlso if you have multiple pixelated/blurry images that helps you can reconstruct it more easily, e.g. if different newspapers print pixelated picture of the "suspect" you can reconstruct it pretty accurately. Machine learning can also do a surprising good job of it, especially if you know what the target is (e.g. a face) https://www.vox.com/future-perfect/2019/9/4/20848008/ai-machine-learning-enhance-button https://www.vox.com/future-perfect/2019/9/4/20848008/ai-mach... Sample code: https://gist.github.com/JonathanFly/80b669a72bf624d17b56a1cfec742588#file-progressivefacesuperresolutiondemo-ipynb https://gist.github.com/JonathanFly/80b669a72bf624d17b56a1cf...
- eru 6y ago> Machine learning can also do a surprising good job of it, especially if you know what the target is (e.g. a face) Yes. Though that's just a corollary of doing better when you know something about the probability distribution of inputs. (But a very useful and practical corollary. My formulation didn't give any hint how you might make use of that knowledge of the distribution.)
- ErikAugust 6y agoSure. I would go a step further - just don’t post any photos of these sorts of documents ever. The risk and reward ratio is too skewed.
- irrational 6y agoThat is my argument against using any social media in a nutshell - the risk and reward ratio is too skewed.
- plorg 6y ago"I personally recommend blacking out (add a black rectangle) instead of blurring, and if it is a PDF, convert to an image afterwards because too many PDF editors use non-destructive operations to add a new object instead of changing what's underneath." I have this at work, with engineering drawings. With mobile equipment often were not dealing with engineering companies per se, and they won't or don't know how to get us CAD models of their equipment. And we often don't have the equipment on have at the time we need to make drawings. But if you have a PDF with vector drawings, often a manual, and one or two good dimensions you can make a reasonably accurate model. AutoCAD even makes this easy with the PDFIMPORT function. More often than I would expect, there's a whole other drawing view either covered by a white box or off-page. Once it looked like it had been drawn over with a white paintbrush tool, and if course the path of that too was also visible.
- bentcorner 6y ago> I personally recommend blacking out (add a black rectangle) instead of blurring I've seen people use image editors on mobile and they'll "scribble" out sensitive information, but one of the problems is that if you pick the wrong pen it'll blend your strokes so it's not 100% opacity (but on a casual glance it's close enough). You can zoom in and change the contrast of a photo that has been redacted this way and recover information.
- saagarjha 6y agoIt's unfortunate because that's the "thicker" brush so people tend to choose it first…
- sjs382 6y agoSometimes a black bar or even cropping isn't sufficient. You still have to trust the editing software. There was a scandal around 2003 when a TV host took a topless photo, cropped it and shared the cropped photo online. Unfortunately, the software (Photoshop—I think CS3) she used to crop the photo stored the original photo as metadata if you didn't change the original filename. The original (uncropped) photo could be seen in the "Open File" preview dialog when opening the cropped version.
- qwertox 6y agoBlacking out is the correct thing to do. Not cutting it so that it becomes transparent since this may still preserve the color component of the RGBA-pixels, even if it is invisible and blended with a black background.
- j_walter 6y agoI found many years ago that my pay statements suffered from the last item you mentioned. My personal info had a black box over things like the SSN...but if I just moved the window around the black box followed slower than the document so everything was visible. ADP never acknowledged the problem when I brought it to their attention, but they did eventually fix it.
- deleted 6y ago[deleted]
- stjohnswarts 6y agoWhy not use a randomized blur so people who like to do such things can waste time trying to figure it out when it's actually nothing but random numbers and has none of the original info?