9 ms·
Sea-Thru: A Method for Removing Water from Underwater Images
- Scaevolus 7y agoNeat! This uses Structure from Motion to compute depth to each pixel and correct backscatter for the specific distance light is traveling from the subject. Typical flat color correction algorithms can help reduce the blue tint typical of underwater photography, but it's only physically correct for a narrow distance band.
- perf1 7y agoModern Image programs have a Dehaze algorithm that already can calculate a depth map using the haze. Wonder if this is better.
- Double_a_92 7y agoThe paper says that they used current dehazing algorithms as a base, and added more parameters to them.
- dorkwood 7y agoWhich programs? I had to generate some depth maps recently and couldn’t find a good solution.
- bufferoverflow 7y agoThat's not what the comment above said. They said some imaging programs create depth maps, and process images based on that. It doesn't mean it creates a depth map for the user, it can simply be an internal part of the whole process. Photoshop RAW pluging does dehazing. I don't know how it works.
- TrueDuality 7y agoThis seems like it's just color fixing using stereographic distance modeling and a known reference palette. I'm not sure what is new or novel about this? Maybe it's just new to oceanographic photography?
- Pfhreak 7y agoInventions often seem obvious or trivial in hindsight. Are you aware of this technique being used anywhere else? Because this seems pretty novel to me.
- TrueDuality 7y agoYeah both are common in astrophotography. Color correction using a reference palette is also pretty common when doing any kind of scientific imaging, though in lab setting the distance is usually well established and not relevant.
- Semaphor 7y agoTFA: > Once trained, the color chart is no longer necessary. As Akkaynak helpfully explained on Reddit, “All you need is multiple images of the scene under natural light, no color chart necessary.” On Reddit: > Just a clarification: the method does NOT require the use of a color chart. That part was not clear in the video. All you need is multiple images of the scene under natural light, no color chart necessary.
- TrueDuality 7y agoThat makes it way more relevant.
- jsjw7sbw 7y agoWhat annoyed me with that answer is they didn't explain why they needed the color chart at all. I would assume it's for training some model, which would lead to this method not working without the chart out of the box for example in muddy waters.
- knolax 7y agohttps://www.scientificamerican.com/article/sea-thru-brings-clarity-to-underwater-photos1/ https://www.scientificamerican.com/article/sea-thru-brings-c... Original article linked in current article, has more pictures/content.
- Markoff 7y agothe first picture in this article is extremely blurry compared to original
- weinzierl 7y agoWhile, as explained in the article, the method was developed to help scientists and not primarily to improve aesthetic qualities, I find most of the images very pleasing. The one thing that is disturbing though is images with visible "horizon" (for lack of a better word). I find the images that look like they were taken on land but don't have a sky where one would expect it somewhat uncanny.
- aitchnyu 7y agoI was initially disappointed by the shadows/refraction patterns of the surface waves are present in the images. Can we compensate for that?
- bschwindHN 7y agoFun fact, those patterns are called caustics: https://en.wikipedia.org/wiki/Caustic_(optics) https://en.wikipedia.org/wiki/Caustic_(optics)
- OisinMoran 7y agoI imagine if you had multiple shots you could median filter through time, similarly to how people remove crowds from busy landmark photos.
- bayesian_horse 7y agoNope. Not with this technique at least. Maybe you could train a CNN to do that...
- padwan 7y agor_wateralpha 0.1
- pronoiac 7y agoI kinda want to see this applied to the diver video. It could surface artifacts, or look surreal.
- johnpowell 7y agoI'm not really excited about technology anymore. But I found this absolutely amazing. The people that made the jankey whale pop out of the floor and raised billions have nothing on this. I'm not joking about how cool I think this is.
- TeMPOraL 7y ago"I really see this as a start of the artificial intelligence boom in marine science" - first words of the video in the article. I don't understand why to even say something like that. PR value? As I understood the paper, the presented method is very refreshingly clean of anything resembling what we call AI today. It seems to be a combination of good, old-school photogrammetry and image processing techniques - which is great, because with such methods they can actually ensure the result is physically correct.
- cameronbrown 7y agoAI just means "I want my paper funded & published" nowadays.
- ovi256 7y ago> PR value? Yes, the hype around AI / machine learning is such that everyone wants it to do magic and invigorate their field of study.
- jhncls 7y agoIn the paper [0] there is more info than just what's explained in the simplified blog post. A.I. is needed if you want to do the same without the color chart used for training it. Furthermore, it is not just color correction, also backscatter is taken into account. [0] http://openaccess.thecvf.com/content_CVPR_2019/papers/Akkaynak_Sea-Thru_A_Method_for_Removing_Water_From_Underwater_Images_CVPR_2019_paper.pdf http://openaccess.thecvf.com/content_CVPR_2019/papers/Akkayn...
- TeMPOraL 7y agoYes, I'm talking about this paper exactly. All I can see is photogrammetry and regular image processing stuff. There's no DNNs or other hot stuff. I guess the part where they're estimating parameters for the equations from photos counts as ML, but people have stopped calling this stuff "AI" quite a while ago.
- Acinyx 7y ago
- kawsper 7y agoA couple of days before I saw the Scientific American article someone dropped this project in our local development group: https://github.com/nikolajbech/underwater-image-color-correction https://github.com/nikolajbech/underwater-image-color-correc... the repository contains some test images, and they also provide a website to test it out: https://colorcorrection.firebaseapp.com/ https://colorcorrection.firebaseapp.com/ that states: "This web-application features a beta version of the Paralenz color correction algorithm, which features an open-source license". The two projects seems unrelated at first glance, but the timing is interesting.
- SimpleMinds 7y agoI've skimmed the description of the repo and it does look unrelated. The paper uses depth information while the repo is focused on color balance. The paper mentions this is inaccurate way of fixing the colors as the color information changes with distance too.
- ape4 7y agoThe name is pretty good.
- Markoff 7y agois it physically accurate image though? I think those untouched photos are accurate same as seen by naked eye sure it's more aesthetically pleasing, but it's already distorting reality personally I think photos should be most accurate representation of what healthy naked human eye see, no beatification, no bigger contrast or oversaturating despite making photos more appealing. if you twist reality where it will stop, where is the border of what is too much?
- ygra 7y agoThe goal for this might not be to be visually pleasing, but to recover the actual colors of what is underwater. And yes, in that respect it should be physically accurate. She uses a color chart and multiple images from different distances. The algorithm thus ensures that colors are neutral. This is not what the eye sees, but in general, very, very few photos are. Do you also model the fovea in photos you take and make everything blurry around the center in the pursuit of greater realism? Do you shun long exposures of the night sky, as our eye can't see the milky way in that detail anyway? It's a misguided goal in general, I'd say. Some photos aim to realistically depict a scene, some aim to artistically depict a scene, some aim to depict a scene to evoke certain emotions and either realistic or artistic might be valid choices for that. The list can go on, I guess. In this case, being able to see and convey the actual colors of what is being photographed has scientific and artistic value.
- deleted 7y ago[deleted]
- leni536 7y ago> personally I think photos should be most accurate representation of what healthy naked human eye see Under what lighting?
- Markoff 7y agono artificial lightning which would not be there just for photo
- stjohnswarts 7y ago
- jansan 7y agoDoes something like this exist for over-water photography? I remember once taking a picture of beautiful scenery in the Japanese mountains, and the result was a very disappointing, almost completely grayed out image.
- grenoire 7y agoShoot in RAW and adjust white balance to your taste, maybe tweak the colour space too. Professional editing software such as Lightroom in combination with high-resolution and fidelity RAWs will allow you to correct even the worst pictures, as long as it's not pure white or black.
- Karliss 7y agoHere is a recent research that does haze removal and other image decomposition http://www.wisdom.weizmann.ac.il/~vision/DoubleDIP/ http://www.wisdom.weizmann.ac.il/~vision/DoubleDIP/
- bayesian_horse 7y agoI've had some success with adaptive contrast ... basically you compute the difference between the original and a blurred image and then apply more contrast where that is high. I've done this in Darkroom and Blender (with the node compositor).
- laurent123456 7y agoWhether the modified photos look better or not is mostly a matter of taste I guess. Personally I prefer the non-modified photos for the underwater feeling they give. It's a cool technique though, and probably useful for scientific purposes.
- skocznymroczny 7y agoI'd be kind of curious about the reverse filter. I wonder how some of my photos would look like if I "underwaterified" them.
- fareesh 7y agoHow does one verify if the output is correct?
- burundi_coffee 7y agoMaths
- gus_massa 7y agoShe used a color chart for calibration. I think it should work, but I'm not an expert in image processing. An alternative can be to make some similar looking objects that are like fake coral blocks and use a boat to sumerge them in the sea and compare the corrected photos with the photos taken in air before. (The wet surfaces have a different look, and it would be important to use a diffuse light instead of direct sunlight, and perhaps other technical problems for the comparison.)
- dghughes 7y agoI'd assume that's the point of the colour reference card. You know what the colours should look like when not under water.
- leni536 7y agoThe goal of the transformation is to make the image look like as if it were photographed above water. A straightforward measurement would be to photograph certain subjects above and below water and compare the appropriate reference and transformed images.
- dylan604 7y agoI would be interested in an experiment of exporting video to a frame sequence, and then letting the software process each frame. Would it make the same decisions on each frame so that when played back as video there is no visible changes between frames? While the results are impressive for a single still, using this to see movement that occurs underwater would be amazing with the improved color.
- djsumdog 7y agoand you could also use it to just accuracy (or at least consistency) of the algorithm. Sounds like a good follow up paper.
- dylan604 7y agoIn the early days of VR, I did the equivalent of this test. The software to stitch stills was much more robust than the the nascent versions of software handling video. As an experiment, I ran the video through the video software, and then did the export to image sequence and had the stills software apply the same settings to the image sequence. When the image sequence was converted back to video, you could see how the stills software made different decisions for each frame resulting in a very psychedelic video. With more tweaking the trippy effect was reduced but not eliminated, and the video software was updated to become much more robust (and impressive). As a primarily video guy, I always laugh to myself (sometimes not to myself) at the amount of effort photo editors spend on a single image. I remind them that the video world has to do that same level of work, except x24 per second multiplied by number of seconds. Photoshop is cool, but Nuke is mindblowing
- 666lumberjack 7y ago>I remind them that the video world has to do that same level of work, except x24 per second multiplied by number of seconds. While this is true, video frames only have to stand up to scrutiny for ~40ms. You can take a lot of clever shortcuts when an onlooker only has a fraction of a second to spot the rough edges!
- jagged-chisel 7y agoI'd love a tool to reverse this process on normal photos so I can put random family events under water without drowning the family
- soylentcola 7y agoYou can do it in Photoshop with various color tweaks, blurs, and textures. It helps a lot to "stage" the original shots for background removal (or to look like they could already be in an underwater location).
- munificent 7y ago> It helps a lot to "stage" the original shots In particular, a fog machine will help emulate the haze and backscatter produced by water and the particles floating in it.
- neovive 7y agoThese are truly amazing results. Very excited to see some wonderful underwater photography in the coming years. Also, the name "Sea-Thru" is cleverly perfect!
- joshdance 7y agoThe branding of this is on point. Well done.