20 ms·
Users reporting artifacts appearing in old images stored in Google Photos
- thriftwy 4y agoLooks like fractal encoding going bad.
- talkinghead 4y agobit rot
- eis 4y agoThat can be ruled out. Bit rot does not appear suddenly in a widespread manner and does not cause very similar visual effects in a variety of images. Also the original images are still available, just new modified versions were shown. To me it looks like a new compression algo gone wrong.
- deleted 4y ago[deleted]
- trustingtrust 4y agoA bad bit wouldn’t even be noticeable in an image. Changing one bit would make literally no difference. Artifacts look like something the compression algorithm has messed up. Also since the images are actually intact according to Google this is likely just the compression algorithm used to browse the images. Update: my limited knowledge on this is proof I’m wrong.
- can16358p 4y agoA bad bit might have no visible effect at all (e.g. Affecting part of EXIF metadata, or a totally black pixel turning again very close to total black) or can corrupt the whole image (flipping a bit in the file header or cosine quantization table). It totally depends.
- mahathu 4y agoAs a total layman when it comes to storage: Don't they use forward correcting codes to make sure a single bad bit can reliably be detected and flipped without affecting the data at all? Or are these used only for data transmission?
- d110af5ccf 4y agoThe vast majority of file formats do not do such things. Such a concern is (typically) out of scope for that layer. Even most data transmission components don't use forward correcting codes. They typically just have error detection.
- rainbowzootsuit 4y agoI've encountered older digital photos where a small corruption affects all of the remaining parts of the image as it is presumably recorded in some linear fashion as shown in this Wikipedia entry's "Visual example." https://en.wikipedia.org/wiki/Data_degradation https://en.wikipedia.org/wiki/Data_degradation
- Spacemolte 4y agoCorrection: "bit rot as depicted in a movie"
- deleted 4y ago[deleted]
- Forge36 4y agoLast post suggests Google has already started fixing the issue. Another suggests the original isn't lost and is available via takeout
- WirelessGigabit 4y agoEven the original isn’t in takeout. They separate the photo from the exif data.
- 2OEH8eoCRo0 4y agoThese artifacts aren't present when I download the original image. This gave me quite the scare though.
- jfoster 4y agoI had noticed one of the images in my Google Photos account seemingly not showing up. It was very noticeable because it was one of the "People" images in my Photos account. It's a bit concerning that a photo storage service seemingly can't maintain the integrity of their data. Makes me wonder about what other data issues there are, lurking in the depths of Google Drive...
- insightcheck 4y agoI personally found a corrupted file stored in online storage only, though it wasn't with Google, but with Microsoft's OneDrive. I tried to open a Word document that I hadn't viewed for several years, but got an error that the file was damaged. It looks like this happens sometimes, after searching for the error online. Luckily I was able to recover an earlier version of the file (via version history), but it was alarming that it happened in the first place (also I'm not sure if important edits were made in the latest corrupted version). It's unfortunate how many cloud services default to storing on the cloud only instead of also keeping a local copy, and don't even provide the option to opt out. Even if you choose the "keep offline" option for several services, for unknown reasons, this doesn't seem reliable (Maybe I'm 'using it wrong'? But in practice, I've found myself having to download files that I was sure I set to keep offline.)
- londons_explore 4y agoGoogle is pretty proud of their record of not losing user data. As far as I know, they have never permanently lost even one byte of consumer data. The same isn't true for GCP - there they lost some writes to customers persistent disks - but only ~50 megabytes worldwide, which is still rather good when you consider they store millions of terabytes.
- ocdtrekkie 4y agoIt really depends what you consider lost. Thousands of users (at least) have been permanently and eternally cut off from their Google data after some AI system decided to axe someone's access to their own account. The loss is significantly over a byte. Or 50 MB.
- scarface74 4y agoFor context, I know this is a display issue and not a storage issue and you can get your originals out via Takeout. That being said my photos are the only digital files I really care about not losing. My photos and videos from my phone get backed up to iCloud, Google Photos, OneDrive and Amazon’s photo storage that comes with Prime. My videos get backed up to all of the above except Amazon’s storage.
- ghaff 4y agoYou should back them up locally as well even though that has its own failure points. I consider anything other than a dedicated local and/or cloud file backup system a nice supplementary place to store photos but not primary.
- scarface74 4y agoAt one point I had them synced to my Windows computer via iTunes and that was backed up to Backblaze. But, I don’t have a personal computer anymore and soon won’t have a personal residence. My wife and I are going to be digital nomads traveling across the country for a few years.
- count 4y ago1TB thumb drives are tiny and can hold a TON of pictures.
- Cyph0n 4y agoSame here. I lost some photos years back and decided to never make that same mistake. My backup flow is: phone, iCloud, local ZFS array served by PhotoPrism (synced using PhotoSync), Backblaze.
- natch 4y agoThanks for sharing. I would add a par2 in there somewhere. Not that my system is as good as yours yet.
- nonameking2026 4y agoLooks like someone is doing some unauthorized time travelling.
- bilsbie 4y agoSeems like the back to the future theory of time travel. Photos start fading.
- zitsarethecure 4y agoOnce a year I select about 100 of my favourite pictures from my phone and have them printed and bound to a book. I suspect this is likely to be the only way my descendants will be able to see them.
- Forge36 4y agoI've started picking my favorites and printing on metal/canvas. My home backup is going to be it's own problem after my death.
- hef19898 4y agoThat's why I, and every other enthusiast photographer I know, keep my archive locally, meaning on my laptops hardrive and an external one as backup. You cannot trust any cloud provider to properly maintain your data, and external storage is becoming cheaper every year.
- newZWhoDis 4y ago> You cannot trust any cloud provider to properly maintain your data, and external storage is becoming cheaper every year. Exactly, and setting up a NAS has never been easier.
- reaperducer 4y agoFor 99% of people, a NAS is overkill. Plug-and-play USB drives get the job done at a fraction of the complexity.
- newZWhoDis 4y agoI love having a NAS that replaces Time Machine, Dropbox, iCloud, Netflix, Spotify, Disney plus, Hulu, runs home automation, has my security system on it, etc etc. People use so many cloud services and having it all private and self-hosted is absolutely amazing. It’s really not that hard to set up either, and there’s little to maintenance outside replacing a failed drive.
- varispeed 4y agoPhotos probably worn off from all that AI training /s
- sitkack 4y agoMy first thought was that Google as recompressed everyone's images to save space and corrupted them in the the process. Hopefully everyone gets their images restored.
- cm2187 4y agoI would have rather thought it would be some sort of image enhancement algo gone wrong.
- IshKebab 4y agoYeah it's definitely some overflowing brightness enhancement thing. You can see that in the before/after photo that someone posted.
- vanderZwan 4y agoI was thinking something similar: it has a distinct posterization look that I tend to get when I mess around with images and reduce the bit-width of my input a little too much and my integers overflow. But those tend to result in "zebra stripes" like patterns, so you're probably right that the issue is more one of not using saturating integers while amplifying a value above the bit-width of its container.
- deleted 4y ago[deleted]
- sitkack 4y agoYou can boost contrast to make low bitrate images look better, so it could be a combination of both.
- ReactiveJelly 4y agoIn the thread they say that the original files are available in Google Takeout, so it'll probably get fixed eventually.
- Shorel 4y ago
- woliveirajr 4y agoOld problem. If you have only one copy, you have none...
- amelius 4y agoThis is an indication that Google underestimates the value of data stored by users. E.g. if this was about storing monetary data, this would never have happened.
- anotheraccount9 4y agoDo you have any suggestions for a secure/private option to store photos? I'm not sure what would be best. (Dang, I really want to move away from Goo now)
- slazaro 4y agoYour own devices, with a proper backup solution (which you should already have with or without your photos in it). Nobody can mess with your data then. Personally, I use Syncthing to sync between devices, including a NAS as a target, which regularly backs up to Backblaze.
- lostlogin 4y agoThis sounds a bit like what I do. I r-sync two different libraries to NAS then upload them to Backblaze B2. Post r-sync success result to a muted Slack channel and failure to an unmuted alert channel. rsync -av /Source /Destination rsync -h
- victorbstan 4y agoiCloud
- celsoazevedo 4y agoiCloud may be secure, but private? No end-to-end encryption and Apple scans the user's content. It's no better than Google Photos in that regard.
- deepersprout 4y agoI'm using Nextcloud. Works great.
- TazeTSchnitzel 4y agoCould it be some sort of clipping/banding issue when decompressing? The way it follows curves would be unusual for photos corruption.
- mwambua 4y agoYikes, this is kind of scary. Google Photos is the only place to find some of my older photos. What’s the easiest way to download my entire library for backup on a physical drive?
- cj 4y agoGoogle Photos used to have the option to sync all photos into Google Drive but that feature was removed. I’m guessing the raw data is still available via Google Takeout
- iforgotpassword 4y agoTakeout
- willcipriano 4y agoI do takeout once a year for such an eventually. https://takeout.google.com/ https://takeout.google.com/
- mwambua 4y agoAh, awesome! Thanks!
- benhansenslc 4y agoI have been having real problems trying to do Google takeout with my Google Photos. I keep getting an error emailed to me saying "However, we were unable to create a copy of all your files. You can still download your files until September 25, 2022, but please be aware that there will be data missing." Apparently I am not the only one[1][2]. Other people have a mix of errors[3]. It is not the reliable data extraction I was expecting for something as valuable as my family photos. Google is losing a lot of trust from me based on this. [1]: https://news.ycombinator.com/item?id=25591630 https://news.ycombinator.com/item?id=25591630 [2]: https://sysc.org/what-is-google-takeout/ https://sysc.org/what-is-google-takeout/ [3]: https://www.reddit.com/r/googlephotos/comments/khited/google_takeout_google_photos_only_needed_to/ https://www.reddit.com/r/googlephotos/comments/khited/google...
- 4y ago
- llaolleh 4y agoNot your server not your photos?
- agumonkey 4y agoImpressive compression glitch. Bitrot evolved.
- gbasin 4y agoNothing a little Stable Diffusion can't fix. Soon, digital memories will be just like real ones — slightly different every time we look back, but with no way of knowing what's changed
- brador 4y ago> with no way of knowing what's changed You can just hash them locally or use AI to identify modifications.
- johnnymorgan 4y agoExactly..what is proof of existence for $400 Alex
- ReactiveJelly 4y agoThe promise of the cloud was that you don't need local storage, because Someone Else will remember things for you, and you just ask them what you remember. This is also the problem of the cloud.
- rsync 4y ago"The promise of the cloud was that you don't need local storage, because Someone Else will remember things for you ..." Agreed. And that promise has been fulfilled - although it costs more than nothing and you need to do your homework. You know these providers are proven bad actors. You know their business model(s) are antagonistic to your goals. You also know there are stable, time-tested alternatives. What did you think was going to happen here ?
- sgregnt 4y agoI would say at least partially aligned with your goals otherwise why would you use them in the first place?
- Godel_unicode 4y ago
- lostlogin 4y agoAt least one user on that thread is reporting it’s fixed. It’ll be interesting to see if all do.
- 12ian34 4y agoI've been long considering switching my Google Photos library fully over to ente [0], of course alongside my existing monthly backups to local HDDs and an annual backup to a cloud server via borgbackup. With every passing week, every Google fuck up like this and with ente's great pace of development, I draw myself nearer and nearer to the precipice of full switchover. I will soon be having my last Google Takeout. [0]: https://ente.io/ https://ente.io/
- Scene_Cast2 4y agoI remember trying ente, but the mobile app had a notoriously low amount of features and low FPS (felt like 30fps on my modern, 120Hz phone - stuck out like a sore thumb). Has there been any progress on that front?
- TingPing 4y agoI think flutter is pretty bad, some animations feel off, but core functionality like scrolling through hundreds of pictures functions just fine. I switched to the service completely.
- Apocryphon 4y agoDoes Google Photos or ente actually use Flutter? edit: RTFA, guess ente does - https://github.com/ente-io https://github.com/ente-io
- vishnumohandas 4y agoHey! One of the founders of ente here. We have shipped a bunch of performance improvements and features[1] over the last few months. But we are far from where we want to be. If you happen to revisit and find something lacking, please write to vishnu[at]ente.io, we'd love to make ente work better for you. [1]: https://roadmap.ente.io/top-released/ https://roadmap.ente.io/top-released/
- Hendrikto 4y ago
- epinephrinios 4y agoThat's why I use Dropbox. They added a lot of new Photos features.
- aceazzameen 4y agoI use Dropbox and Google Photos. Photos for sharing and Dropbox + local backups of the Dropbox folder for a more permanent storage solution.
- nharada 4y agoSeems unlikely Dropbox would be immune from some sort of issue like this. Probably best to have your photos in storage somewhere else (ideally both locally and off-site).
- nathanaldensr 4y agoYou know how the mantra in the crypto community is "not your wallet, not your coins?" This applies to this situation. Once you trust Google with your bits, don't be surprised when they modify them without telling you.
- jonnydubowsky 4y agoUs this the beginning of some Back To The Future moment in this timeline?
- philips 4y agoI keep intending to launch a service that downloads cloud data and sends it to you on an SD card. I hope to fix up my prototype and launch it next year. https://www.clonecamel.com/ https://www.clonecamel.com/ If anyone knows of a quality source of bulk SD cards or flash drives in high capacity please let me know.
- ravel-bar-foo 4y agoThe susceptibility of the SD card to data corruption is a big showstopper. The only digital files I've had corrupted were stored on my phone's external SD cards (after 3 to 7 years). When I looked into it, it seemed to be an issue related to lack of radiation hardening. So if you do make this service, please include some error detection/error correction metadata.
- philips 4y agoThat is good feedback The idea would be to subscribe to have this data shipped once or more per year so hopefully some of the data would survive. Any advice on error correction metadata and tools?
- squeaky-clean 4y agoOnly tangentially related but I just checked the Google Photos iOS app to see if any of mine are corrupted, and it won't even allow me to view my cloud photos without giving it full permission to all current photos on my device?! Isn't this against the app store policy? You can't deny me access to one feature just because I refuse to grant permissions to a different feature. But because it's Google I'm sure they can break whatever rules they want as long as it doesn't involve trying to avoid Apple's 30% fees.
- sixothree 4y agoAnd it will spend its spare time analyzing every single photo in your library.
- smoldesu 4y agoGoogle doesn't scan photos that don't exist on their servers.
- soulofmischief 4y ago... currently. Trusting a dubious central authority up until you get burned is a bad heuristic.
- smoldesu 4y agoI agree, which is why I don't pay for cloud storage.
- b112 4y agoEspecially when Google has said "we won't do thing X", then been caught already doing it, or just silently starting to do it. Just location tracking. The lies. Over, and over, and over, and over again the lies.
- maxbond 4y agoInstitutions of that nature are incapable of following through on nearly any promise made to the public at large; every promise is a lie, whether they break it or not, because they can't be said to have the intention to uphold it. I see no reason not to believe that Page and Brin sincerely desired Google not to be evil; but it's the institution decides to do evil, not any individual, so they did not have (or could not maintain, as the organization scaled) the power to guarantee this. No one does.
- bumblebritches5 4y ago
- viksit 4y agopsa: noticed today that going to the original photo shows one that’s not corrupted. only the resized algorithm seems to do this on lower dpi photos.
- renewiltord 4y agoI have my images synced to icloud and Google photos. It'll be interesting to get them out later and compare.
- orangepurple 4y agoRAID-Z2 is the long term solution for anybody capable enough of managing such a setup
- nullc 4y agoVery weird looking artifacts-- they don't seem to be structured in any way that makes sense relative to the format of jpeg.
- uwagar 4y agour photos are spam to the provider if u arent paying for storage. so they may compress them, reduce the quality over time and most may not notice.
- fy20 4y agoHas there been any progress on self hosted alternatives to Google Photos in the past couple of years? Last time I checked there was great systems for individuals, but the main reason why I use Google Photos is so I can have shared albums with family & friends where anyone can add their photos.
- choppaface 4y agoA little buggy but easy to run and very feature complete: https://github.com/LibrePhotos/librephotos https://github.com/LibrePhotos/librephotos
- wan_ala 4y agoNot exactly what you're talking about but IPFS seems cool for storing stuff.
- _a9 4y agoI started using PhotoPrism a couple months ago. My only gripe with it is that you can't drag-and-drop files directly into folders or any page. You have to manually go to the upload page, change/add the folder then use the file picker to choose the files. Other than that it does the job well and even has face scanning. https://docs.photoprism.app/ https://docs.photoprism.app/
- choppaface 4y agoOne of the best ways to deal with Google customer service is to simply send in a support case and immediately dispute the credit card charge. I’ve done this with Youtube TV several times (wrong recording, live the doesn’t work, their location check locked me out) and got free months of service. Don’t bother getting all frustrated in a big support thread. The engineers who caused these bugs DO NOT care about you. They are way way too busy solving dynamic programming problems. Chargebacks are the most effective tool here.
- OrangeMonkey 4y agoI hope to god noone does this. You are flirting with having your entire google account disabled. As this may end up un-personing you, I would suggest strongly taking another route.
- esprehn 4y agoWhat have you done a chargeback for with Google? There's stories of folks doing a chargeback and then having their account closed afterwards meaning you lose Gmail etc.
- LionRoar 4y agoI find the idea of old photo/data degradation strangely comforting. Even digital data seems to be unable to escape the law of entropy (don't mention the black holes). On another note; if corrupting data/photos was a wide occurrence now wouldn't we have heard about it in a big way?
- rasengan0 4y agoThis is normal. I found out that Google Photos has limited reliability. My 2007 photo was garbaged into a lower resolution. I have since recovered from the original drive. Back then Google Photos gave an option to store in lower res but still sufficient high quality or original quality. Don't trust either. Lesson: backup to your media you own or print it.
- crazygringo 4y agoI'm mostly curious in what compression algorithm results in artifacts like these, and assume somebody here on HN recognizes them? But we're 225 comments in (so far) and not a single answer. They're absolutely nothing like block-based JPEG, that's for sure. When I inspect the images Google Photos serves me in my browser, it is serving up JPEG (from the response headers). But is this an artifact that shows up in AVIF or WebP? I wonder if mobile clients are getting a different encoding. The only objective thing I notice is that the artifact lines tend to track a line of constant brightness, so you seem them appearing perpendicular to gradients of light and shade. And that each artifact line is black/dark on one side and white/light on the other -- and that the white edge seems to be in the direction of darkening, while the black is in the direction of lightening. Someone here who works with modern image codecs must be able to hypothesize what part of encoding/decoding must be bugging out here?
- a_e_k 4y agoLooks to my eye like some 8-bit underflow artifacts that I've seen. And the way the line follows the gradient reminds me of when I've broken a gradient-domain Poisson solver to reconstruct an image from gradients. Maybe they're decorrelating the image data via gradients to compress better and then compressing those along with a sparse set of primal pixels? (See, e.g., http://graphics.cs.cmu.edu/courses/15-463/2019_fall/lectures/lecture9.pdf http://graphics.cs.cmu.edu/courses/15-463/2019_fall/lectures... for a nice overview of gradient-domain image processing.)
- nagonago 4y agoClearly Google was storing their old photos in a damp basement, and they got water damage. But seriously, I am also very curious. I wonder if they are playing with an in-house compression algorithm.
- porquilho 4y agoit seems to me from the photo of the dog, it was an face recognition, algo or eyes recognition ago, but the algo also left the changes in the photo as it proccesed it. instead of putting it all in the server side.
- zxcvbn4038 4y agoIMO the best place to put photos, at least the master copies is in AWS S3. It is both reliable and redundant, and they don’t mess with your data. They charge you every month for storing it, and again for accessing it, but that is it. Use anything else you want for day to day usage - Cloudflare, Digital Ocean, Google Drive, etc. whatever fits your budget and needs. But sleep safe knowing your data has a backup. Same thing with YouTube - you never know when Google might decide to wipe out your channel, and if they do you’ll never know why they did it, and you’ll have no recourse after they have done it. There are plenty of stories on HN where this has happened. So the master copy of all your videos should be in AWS so you can start again if you need to. IMO.
- cypress66 4y agoWhy would you use S3 instead of backblaze, which is both cheaper, and designed for that purpose? In both cases we're talking of a backup of course.
- simonblack 4y agoIf you don't hold it yourself, you DO NOT OWN IT. I had a disaster when using DropBox way back around 2011. Ever since then I have been responsible for storing and backing up my own files. Nobody else gets to have them in their 'safe'-keeping.
- javier2 4y agoLooks like seam carving algorithm gone bad?
- fennecfoxy 4y agoWell this is why I've finally gotten a Synology NAS. It's been great so far although I do need to figure out why it powers on all the time (probably SMB shenanigans). But the main thing I got it for is to set up freezing folders full of RAWs I've taken to back-up to Glacier.
- xcjs 4y agoThe powering on all the time is probably due to a Wake on LAN packet or other power management feature of the network hardware of the device. Most operating systems provide a way of disabling the power management features of the network hardware, but I've never used a Synology device, so I can't tell you how to do that.
- puffoflogic 4y agoAny data you store on someone else's server without encryption isn't your data, it's theirs.