8 ms·
I disagree with the criticism. Experiencing the original artifacts of history, like the original unenhanced film, is still possible. And totally understand that
by mcv 6y ago
I disagree with the criticism. Experiencing the original artifacts of history, like the original unenhanced film, is still possible. And totally understand that it's vital that that is still possible. But experiencing history is not just about experiencing its old, degraded artifacts. It's also useful and enlightening to experience history as it might actually have been back then.
Of course that's partially an illusion, and it's important to be aware of that. But it's still a useful illusion. It's not for nothing that many ancient ruins have been restored so people can get a better impression of what they were actually like. With ruins this is actually a lot more questionable than with film, because you actually change the original historic artifact (which is probably why nobody repaints those ancient statues that probably used to have colour). But we do restore even ancient paintings that have been darkened by the passing of time.
With that in mind, why would we not take this opportunity to create a much clearer window to the past?
- ijidak 6y agoAs an African-American it was actually useful to watch some of these colorized videos and get a better since for how African-Americans integrated into the society at that time. Having it in color with sound made the video seem much more real and relatable. For me this just ads another facet to develop a better-rounded grasp of the era. Inserting the extra frames seems to slow the footage down and allow more time for digestion. I find it often difficult to track what's going on in old videos because of the off-speed of the recordings. But for the historians who spend a lot of time with these artifacts, I'm sure this feels like a bastardization of history. But as a lay person, it makes me want to devour more of these videos to get a fascinating and easier-to-digest look at life back in those times. Btw, this is the video I watched: https://youtu.be/hZ1OgQL9_Cw https://youtu.be/hZ1OgQL9_Cw
- roywiggins 6y agoThe dance scene from Hellzapoppin' is my favorite use of the colorization algorithms so far- it's so much easier to follow the dancers when the color pops them out of the background. Color contrast helps a lot. In that case the whole setup- the costumes, the lighting, etc, are all artificial in the first place, adding some fake color on top doesn't seem like a historical sin. https://www.youtube.com/watch?v=qzc7vY9VTnk https://www.youtube.com/watch?v=qzc7vY9VTnk Personally, I don't find frame interpolation to be as effective as colorization; the artifacts it causes are often more distracting than just leaving the framerate alone. It can create a sort of artificial floating feeling to the motion- a problem that people often complain about with native 60fps, and is much worse with interpolated 60fps. Algorithms keep improving though, eventually I might find them more satisfying.
- dnautics 6y agoI love the hellzapoppin' colorization but it does show that we have a long way to go -- you can see the colorization struggle with keeping the skin tone of the dancers consistent. obviously, this has a lot to do with training data bias.
- webmaven 6y ago> I love the hellzapoppin' colorization but it does show that we have a long way to go -- you can see the colorization struggle with keeping the skin tone of the dancers consistent. obviously, this has a lot to do with training data bias. The colorization is struggling to keep colors consistent on the costumes, walls, curtains... basically everywhere the lighting keeps changing a bit. Many things in the scene are constantly shifting minutely between pinkish-beige and bluish-grey, often making them look somewhat iridescent. I've noticed this sort of effect in many colorization attempts. This one does better than most. Training data bias is obviously part of the problem, but not the whole story. It might not even be the biggest cause. Image segmentation is one way to help the colorization stabilize.
- ijidak 6y agoWow! Just watched it. Agree. It is easier to track the dancers! I wonder if it has to do with the fact that our vision is color. Is there color data that the brain is using to process fast moving images that we lose in black and white? Maybe color moving over a different color is easier for the brain to process than two similar colors moving with relation to each other.
- asperous 6y agoI believe the average person will get most of their understanding of the past from time period movies and shows, purely constructed art forms. At least this upscaling "medium" is tied directly to what was captured at the time.
- Someone 6y agoAnd to the training set. It wouldn’t surprise me if machine learning resolution enhancement methods would add a smartphone or some out-of time branding such as Adidas stripes or a Nike swash to some movie scenes.
- wizzwizz4 6y agoIt already does blue jeans (as described in the article).
- devmunchies 6y ago> the average person will get most of their understanding of the past from time period movies and shows actually, that's how lots people get their understanding of the present, too. For example, I'd wager most people on HN have seen more horses in movies than in real life. I've never actually seen what the ocean looks like underwater with my own eyes. It's actually mind boggling how much of our worldview is shaped by content we consume. Think about how different your worldview would be if it were based solely on things you've experienced first-hand.
- eru 6y agoAnd that humans can do that is one of our big evolutionary breakthroughs. Apes can only really learn from what they experience themselves. Teaching involves some other ape guiding them through the act, at best. Humans can learn from other humans (or their artefacts like books and films etc), even when the activity in question is not performed. (There might be some of that learning with some apes. There's some with crows. But it's not nearly as prevalent as with humans.)
- LeifCarrotson 6y agoAgreed. Laypersons and experts alike understand that fact intellectually, we believe that the past was not black-and-white and silent. But our experience in school tells us that a scratchy, low-framerate video is probably from the past. We grow familiar with Hollywood representations of history. Not having any evidence to the contrary, we develop an alief [1] about the past that it was sepia and grainy and distant, when in truth, it was just as vibrant and real as today. No one except Calvin [2] would say that they believe that Gettysburg was black and white, but after looking at a lot of historical photos in a textbook that look like [3], if I go to Gettysburg, see [4], kneel behind the wall, and touch the stones I have a completely different level of comprehension and connection to the people who fought there that I, at least, am incapable of generating for myself from a black and white grainy photo. I completely understand that upscaling and AI can invent things that aren't there, like Gigapixel inserting the face of Ryan Gosling in this photo [5] or Xerox copiers replacing a 6 with an 8 [6], as well as more subtle changes like assuming greyscale pants that might have used no dye at all or natural, local dyes were always indigo blue jeans. The important question is whether or not the AI-invented, upscaled, colorized photos are closer or further from reality than what your brain invents without the aid of that tool. If you subconsciously invent a muted reality, or worse, your brain gets lazy and assumes it's only from a book and wasn't real at all, I think your ability to empathize with and comprehend the past would be better served by a 4k 60fps artist's impression. [1]: http://www.pgrim.org/philosophersannual/pa28articles/gendleraliefbelief.pdf http://www.pgrim.org/philosophersannual/pa28articles/gendler... [2]: https://www.gocomics.com/calvinandhobbes/1989/10/29 https://www.gocomics.com/calvinandhobbes/1989/10/29 [3]: https://www.battlefields.org/sites/default/files/styles/gallery_item/public/thumbnails/image/Then%20-%20Power's%20Hill%20from%20Spangler's%20Meadow.jpg https://www.battlefields.org/sites/default/files/styles/gall... [4]: https://www.battlefields.org/sites/default/files/styles/gallery_item/public/thumbnails/image/Now%20-%20Power%27s%20Hill%20from%20Spangler%27s%20Meadow.jpg https://www.battlefields.org/sites/default/files/styles/gall... [5]: https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-added-ryan-goslings-face-to-this-photo/ https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-a... [6]: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
- 6y ago
- sillysaurusx 6y agoThis snowball fight is so much better colorized (and maybe upscaled): https://twitter.com/JoaquimCampa/status/1311391615425093634 https://twitter.com/JoaquimCampa/status/1311391615425093634 The more illusions the better, I say. Can't wait for the VR recreation of famous battles WW2 battles extrapolated from grainy footage.
- wffurr 6y agoIt's not VR, but... https://www.youtube.com/watch?v=IrabKK9Bhds https://www.youtube.com/watch?v=IrabKK9Bhds
- ProfessorPleats 6y agoIt's not based on old footage, but A VR experience for WWI was created. https://www.warremains.com/ https://www.warremains.com/ It uses CG, but in an interview the head of production talked about how he kept trying to get his team to push the audio and visuals as much as possible since he really wanted to get the "viewer" to understand even a fraction of what it would have been like to be there. He wanted to make sure the viewer felt something instead of just being a fly on the wall like a lot of VR experiences.
- pier25 6y agoIt's an illusion, but so is the original footage. People back then didn't experience reality in BW either. I know it's silly, but I was kinda surprised the first time I saw early color photographies. I was so used to the BW or sepia old images that it had never occurred to me that, you know, people of that time did live in color too just like we do. See these images from the early 20th century: https://en.wikipedia.org/wiki/File:Rgb-compose-Alim_Khan.jpg https://en.wikipedia.org/wiki/File:Rgb-compose-Alim_Khan.jpg https://en.wikipedia.org/wiki/Color_photography#/media/File:Prokudin-Gorskii-12.jpg https://en.wikipedia.org/wiki/Color_photography#/media/File:...
- tommica 6y agoThat is so true, I've also gotten similar kind of realization with being happy - so serious in most pictures, as was the style then, but then a picture of a farm couple laughing reminds me that they infact were living their lives just like we are
- fossuser 6y agoThe seriousness in pictures was a cultural artifact of the time, I think partly because people treated photos like getting portraits? There's this photo from 1901 which seems really modern: https://static.boredpanda.com/blog/wp-content/uploads/2019/03/smiling-chinese-man-eating-rice-1901-1-5c98d17755668__700.jpg https://static.boredpanda.com/blog/wp-content/uploads/2019/0... The speculation is that he didn't know about the culture of looking serious.
- btilly 6y agoIt was also an artifact of the technology. Cameras tended to have long exposure times. If you have to hold a position for 30 seconds, it is easier to do so with a serious face than a laughing one.
- jacobr1 6y agoThe other aspect is the early photos required long exposures. It is hard to maintain a smile or stand still long enough, so a neutral face was requested by the photographer. Daguerreotype's required upwards of 20 minutes!
- jacobwilliamroy 6y agoIt's totally an illusion. Up-scaling is a lie. Colorization is a lie. It's literally missing information that can never be verified. It's like if I used machine learning to double the length of the federalist papers. It would just be absurd to treat that like it was a real piece of history.
- burnte 6y agoSo what?
- c0nsumer 6y agoThere's a bit of a difference between interpolating to fill in spaces between pixels with hints from what's next to them and adding whole new words and sentences.
- jacobr1 6y agoRight. A better example might be removing every 5th word from a famous work, say one of shakespeare's sonnets. Then run GPT3 on it to interpolate the missing words. I bet a reasonable fraction of would match the correct words. Some would be acceptable substitutions, maybe you lose some fidelity or wit from the original, but it doesn't impact the overall enjoyment or understanding of the work. Then will be some minority of cases where we get something weird or wrong. Maybe a word that matches the general meaning, but is modern slang for example. Or maybe it guessed something that is plausible in context, but is opposite in meaning that changes something substantial in the work. I would expect that last scenario to be very rare as a fraction of total word substitutions, but also quite likely over the course of a whole piece of work. On the other hand, if we were able to render have a new undiscovered play available constructed from some sort of damaged fragment, rather than my hypothetical test ... having something we could read or see performed rather than just analyze academically seems worth the chance of getting some (even important) nuance in the work wrong.
- jacobwilliamroy 6y agoThat's actually a worse analogy. You're obscuring what's actually going on here. I was being nice when I said it was like "doubling the length of the federalist papers." In a proper analogy, our fake document would be 24 times the length of the federalist papers. 96% of our document would be completely fabricated. FAKE. At that point you're learning more about the computer than whatever document you're viewing and it's the same for video. Scaling up from 480p to 4k requires the computer to fill in 96% of the information. That's really only good for entertainment value, not serious academic research into a historical period. Computers aren't magic time-traveling machines. If you want 4k reproductions of old films, just scan the actual film with better equipment. If you want colorization, that requires actual investigation and to figure out what colors were in use, and how to map them onto the video data. If no such investigation is possible then we're SHIT OUT OF LUCK. It happens. Move on. Stop revising history for entertainment.
- m463 6y agoIt might be as simple as saying "what historians are saying this? are they speaking for all historians? I just asked 3 historians and they love 4k."
- DennisP 6y agoSince most of the videos show before and after, I've seen more of the original artifacts than I ever did before.
- xenocyon 6y ago> experience history as it might actually have been I think the might in the above is key. If you add new information to take the place of lost information, you're taking a risk that the new information does not accurately mimic the information that was lost. The more bits of new information you add, the more this risk increases.
- willis936 6y agoI would like to see AI upscaled and colorized versions of the past when there are at least 15 9s assuring me that I won’t see Ryan Gosling on the front lines.
- jeffalyanak 6y agoThese people are just doing for photos and video what historians have always done for history in general. They are filling in the blanks with their best guesses according to the information present. As long as they aren't presenting their results as more real than the original images, I don't see it as a bad thing. In fact, I think that the field in general needs to be more open about just how much this happens with history in general. Our understanding of the middle ages comes from evidence far less clear than these photos and yet its rare to see professionals admit how much of the field involves countless evolving, educated guesses. Others may feel differently, but I find the field even more impressive with the understanding that historians are not seeing a clear picture.
- dnautics 6y ago> why would we not take this opportunity to create a much clearer window to the past. My only concern would be that by the very nature of deep learning we are seeding it with simulated information from the current era so we could be injecting temporal biases. Imagine the process of colorizing -- imagine there was a dye color that we couldn't manufacture prior to year X. And we train colorization on a dataset that whether known to the curator or not includes this dye. Are we creating images that are a misrepresentation of the past? On the other hand, "science" often does some pretty (I think) extreme extrapolation of things like dinosaur fossils, so maybe it shouldn't be such a big deal and it's just something we have to live with.
- asxd 6y agoDefinitely have to agree with you here, as well as other posters saying it's more relatable seeing enhanced and colorized footage. To me, history becomes much more engaging and thought-provoking when presented that way. Not strictly related, but it brings to mind the 3D mapping of Pharaoh Ramesses VI Tomb [1]. Walking around the tomb in VR was very eye-opening and really made me appreciate the magnitude of people having built that so long ago. [1] https://news.ycombinator.com/item?id=23467292 https://news.ycombinator.com/item?id=23467292
- 46388226363 6y agoI think this could easily be taken a step further. Who cares what historians think when they're so obviously vulnerable to contemporary politics? When a random group like the NYT can kickstart a politically motivated effort to rewrite history and the only meaningful objectors are the economists it's pretty obvious that American historians have lost the integrity needed to engage with their subject in an objective fashion. It's beyond laymen to understand which parts of historical study have been politically coopted and so the only solution is to assume all histories are now compromised. From that position, things like upscaling don't matter in this context because the integrity of the historical record was already sacrificed.
- flohofwoe 6y agoI think the problem boils down to the difference between a "pattern-matching" AI filling in details by "inventing" them, or a human restorator doing essentially the same. A human restorator most likely has more background knowledge from other sources, while the AI might fill in completely unrelated details it's seen in completely unrelated training material. As long as there's a disclaimer about the type of restoration that's been used, both approaches are fine IMHO. The audience needs to understand that both are just "projections". Similar to how we thought for a long time that ancient greece was "marble white", when in reality it was full of colors.