9 ms·

For those who think it's just another lame DL based instagram filter... The method proposed in the paper(https://arxiv.org/abs/1707.03491 https://arxiv.org/abs

For those who think it's just another lame DL based instagram filter... The method proposed in the paper(https://arxiv.org/abs/1707.03491 https://arxiv.org/abs/1707.03491) is mimicing a photographer's work: From taking the picture(image composition) to post-processing(traditional filter like HDR, Saturation. But also GAN powered local brightness editing).In the end it also picks the best photos(Aesthetic ranking) Selected comments from professional photographers at the end of paper is very informative. There's also a showcase of model created photos in http://google.github.io/creatism http://google.github.io/creatism [Disclaimer: I'm the second author of the paper]

zitterbewegung 9y ago
I thought of a cool application . You could use this to generate desktop photos for desktop environments that are either localized to a place you are near or the user could type in a location that they want to see .
- samstave 9y ago
  Or an actual photography class app where a user could tell google exactly where they are standing on streetview and have the app tell them the best direction/angle/settings to take their pic and walk them through WHY it is as such
  - anigbrowl 9y ago
    It can't do the latter, except in a very basic way of pointing to proximity to the training set. ML is great at imitating, but good look getting it (or the people programming it) to be able to articulate any of the underlying principles. This is a perennial problem in the arts, which might be why the examples on this page are pretty ho-hum from an aesthetic point of view.
  - NIL8 9y ago
    I bet this is something we will get eventually. As I think about it, my mind is flooding with ideas on how this would effect other areas of life.
placeybordeaux 9y ago
Is the code available?
bluetwo 9y ago
Did you have anyone say "Professional photographers don't apply HDR filters to single photos"?
- hoorayimhelping 9y ago
  Why would they say that? You can't have HDR with a single exposure. I think you're trying to say professional photographers don't apply tone mapping to single exposures, but that is false. There are times when it's not 'technically proper' to apply tone mapping to a single exposure, but it makes for a better picture. Really it sounds like you're trying to make a snarky comment referencing how some professional photographers complain about novices over using HDR on flickr. But who cares. You can have a professional quality photograph that uses what some pros think is incorrect tone mapping. Just like how you can have a novice hack together a working professional-level product that many professional engineers think is trash. If it works and meets the criteria, who cares.
  - wl 9y ago
    > You can't have HDR with a single exposure 14 stops of dynamic range in a single shot (RAW on a modern DSLR) counts in my book.
    - barrkel 9y ago
      HDR worthy of the name seems to require multiple levels of light gathering, to me. Either exposure time or aperture needs to vary. If you don't, you either blow out on bright light, or you have too much noise in dim areas. You might not notice if the scene is well and reasonably evenly lit though.
      - wl 9y ago
        You never vary aperture when taking bracketed exposures to merge to HDR. The depth of field would vary from exposure to exposure, making things inconsistent when you merge them. It's shutter speed you vary. My point is that with 2005-era DSLRs, I had to use HDR techniques (or strobes) to, say, correctly expose a room and the environment out a window. Shooting into the sun, I'd have to use HDR, polarizers, and/or ND grads to expose both the sky and the foreground. With my D750? I've got 4 stops more dynamic range and the situations where I need HDR have almost all gone away. I can get the same thing from a single RAW exposure. These days, HDR only seems necessary if you want to have ridiculous tone mapping where you pull in your global highlights to be within a stop or two of your global shadows and still keep the noise levels down.
samstave 9y ago
on the pro side: you could use this to determine the most photogenic scenes in the world. On the con side: only an AI photo-bot can find and capture the most photogenic scenes in the world.
vadym909 9y ago
Cool- Are you going to apply it to all images on Google Maps? How about an 'Auto Enhance" button on Google photos?
- eru 9y ago
  They already have that button. But it's doing rather boring things at the moment---but sure to improve in the future.
  - soylentcola 9y ago
    Yeah, it's more akin to Auto Tone/Auto Contrast/Auto Color in Photoshop: often a quick and easy way to tweak levels/curves on a photo that's already decent, but much more varied results on anything with mixed or more atypical colors and levels of light.
collinmanderson 9y ago
The images look really good. I'm curious: Is this creatism link what the algorithm considers the best images, or did you manually filter out some of the less good ones?
zitterbewegung 9y ago
I tried a style transfer network on one of your images . Here is what I got https://prisma-ai.com/p/3a91d082-feee-443e-8062-4e85d2720ea0 https://prisma-ai.com/p/3a91d082-feee-443e-8062-4e85d2720ea0
had2makeanacct 9y ago
Will this ever get implemented in Google Photos automatic editing?
- deepGem 9y ago
  Related - if not editing, use a panorama image taken from the phone to create stunning professional photographs.
  - chii 9y ago
    and in a few years, we find a whole bunch of professional photographers out of a job...
    - rangibaby 9y ago
      By that time I think everyone will be out of a job. At least photographers will still have a hobby ;-)
    - eru 9y ago
      You say that like it's a bad thing.
      - deleted 9y ago
        [deleted]
dsnuh 9y ago
My ex-wife is a professional photographer. She does weddings, boudoir, and portraits mostly. Here's the thing. If she had say, 2 shoots in a week, maybe a total of say 8 hours of shooting time, the selection, editing, and styling of those photos could take 3x or 4x as long as the time spent shooting. She would have to be editing constantly. I've always thought a perfect application for this sort of technology would be a model trained on that particular photographer's style, which goes through a batch of photos and selects the best candidates and then presents the user with a few choices for each photo it selected and styled. I think the ultimate would be a NAS with this capability embedded. The photographers I have met through her over the years seem savvy about photo tech, but don't generally use online storage solutions or know much more than how to admin a WP site at best. SOHO solution would be ideal imo. Great work! I'm excited to see where this goes.
- eru 9y ago
  I don't think the NAS will come at all: the economics just don't work out. It's more likely this will hit consumers hands via an automatically applied 'enhancement' in Google Photos first.
  - dsnuh 9y ago
    An on-prem storage solution specifically targeting these features at professional photographers would sell. Storage is always a problem with every photographer I have met. They keep stacks of external USB drives or burn archives to DVD. They don't trust or want to use online storage. A package that provided these models, trainable on that photographer's particular style, with autocropping, watermarking, etc. would sell very well among this group.
    - eru 9y ago
      I bet you could make millions. Or even tens of millions. Great for a start-up. But Google doesn't bother with such small fry---especially if it comes attached to a complicated hardware business model. At these small numbers, they are more likely to just give away access to the software for free.
oscii 9y ago
Thanks! Interesting work, indeed. I found the showcase photos appealing and nice. I briefly skimmed through the paper with hope to find some interpretation of the “aesthetics”, but couldn't find it. What is meant under “all aesthetic aspects” and “aesthetic quality”? Did you perform a critical analysis of the network? How do you know that the network learned “aesthetics”?
necessity 9y ago
Still lame, just cropping and increasing saturation/contrast. Some of the pictures in the blog post look worse than the originals, they are artificial.
CWuestefeld 9y ago
Apologies for being that guy... Was there any control for, e.g., the dynamic lighting filter being the factor that carried most of the water, rather than the other manipulations? FWIW, I did think your tool did a good job finding good compositions within mundane scenes. Looking at the originals, I have to admit that in many or even most of the cases, I wouldn't have spotted the opportunity - which points to an area I can improve my craft.

zitterbewegung 9y ago

I thought of a cool application . You could use this to generate desktop photos for desktop environments that are either localized to a place you are near or the user could type in a location that they want to see .

samstave 9y ago
Or an actual photography class app where a user could tell google exactly where they are standing on streetview and have the app tell them the best direction/angle/settings to take their pic and walk them through WHY it is as such
- anigbrowl 9y ago
  It can't do the latter, except in a very basic way of pointing to proximity to the training set. ML is great at imitating, but good look getting it (or the people programming it) to be able to articulate any of the underlying principles. This is a perennial problem in the arts, which might be why the examples on this page are pretty ho-hum from an aesthetic point of view.
- NIL8 9y ago
  I bet this is something we will get eventually. As I think about it, my mind is flooding with ideas on how this would effect other areas of life.

placeybordeaux 9y ago

Is the code available?

bluetwo 9y ago

Did you have anyone say "Professional photographers don't apply HDR filters to single photos"?

hoorayimhelping 9y ago
Why would they say that? You can't have HDR with a single exposure. I think you're trying to say professional photographers don't apply tone mapping to single exposures, but that is false. There are times when it's not 'technically proper' to apply tone mapping to a single exposure, but it makes for a better picture. Really it sounds like you're trying to make a snarky comment referencing how some professional photographers complain about novices over using HDR on flickr. But who cares. You can have a professional quality photograph that uses what some pros think is incorrect tone mapping. Just like how you can have a novice hack together a working professional-level product that many professional engineers think is trash. If it works and meets the criteria, who cares.
- wl 9y ago
  > You can't have HDR with a single exposure 14 stops of dynamic range in a single shot (RAW on a modern DSLR) counts in my book.
  - barrkel 9y ago
    HDR worthy of the name seems to require multiple levels of light gathering, to me. Either exposure time or aperture needs to vary. If you don't, you either blow out on bright light, or you have too much noise in dim areas. You might not notice if the scene is well and reasonably evenly lit though.
    - wl 9y ago
      You never vary aperture when taking bracketed exposures to merge to HDR. The depth of field would vary from exposure to exposure, making things inconsistent when you merge them. It's shutter speed you vary. My point is that with 2005-era DSLRs, I had to use HDR techniques (or strobes) to, say, correctly expose a room and the environment out a window. Shooting into the sun, I'd have to use HDR, polarizers, and/or ND grads to expose both the sky and the foreground. With my D750? I've got 4 stops more dynamic range and the situations where I need HDR have almost all gone away. I can get the same thing from a single RAW exposure. These days, HDR only seems necessary if you want to have ridiculous tone mapping where you pull in your global highlights to be within a stop or two of your global shadows and still keep the noise levels down.

samstave 9y ago

on the pro side: you could use this to determine the most photogenic scenes in the world. On the con side: only an AI photo-bot can find and capture the most photogenic scenes in the world.

vadym909 9y ago

Cool- Are you going to apply it to all images on Google Maps? How about an 'Auto Enhance" button on Google photos?

eru 9y ago
They already have that button. But it's doing rather boring things at the moment---but sure to improve in the future.
- soylentcola 9y ago
  Yeah, it's more akin to Auto Tone/Auto Contrast/Auto Color in Photoshop: often a quick and easy way to tweak levels/curves on a photo that's already decent, but much more varied results on anything with mixed or more atypical colors and levels of light.

collinmanderson 9y ago

The images look really good. I'm curious: Is this creatism link what the algorithm considers the best images, or did you manually filter out some of the less good ones?

zitterbewegung 9y ago

I tried a style transfer network on one of your images . Here is what I got https://prisma-ai.com/p/3a91d082-feee-443e-8062-4e85d2720ea0 https://prisma-ai.com/p/3a91d082-feee-443e-8062-4e85d2720ea0

had2makeanacct 9y ago

Will this ever get implemented in Google Photos automatic editing?

deepGem 9y ago
Related - if not editing, use a panorama image taken from the phone to create stunning professional photographs.
- chii 9y ago
  and in a few years, we find a whole bunch of professional photographers out of a job...
  - rangibaby 9y ago
    By that time I think everyone will be out of a job. At least photographers will still have a hobby ;-)
  - eru 9y ago
    You say that like it's a bad thing.
    - deleted 9y ago
      [deleted]

dsnuh 9y ago

My ex-wife is a professional photographer. She does weddings, boudoir, and portraits mostly. Here's the thing. If she had say, 2 shoots in a week, maybe a total of say 8 hours of shooting time, the selection, editing, and styling of those photos could take 3x or 4x as long as the time spent shooting. She would have to be editing constantly. I've always thought a perfect application for this sort of technology would be a model trained on that particular photographer's style, which goes through a batch of photos and selects the best candidates and then presents the user with a few choices for each photo it selected and styled. I think the ultimate would be a NAS with this capability embedded. The photographers I have met through her over the years seem savvy about photo tech, but don't generally use online storage solutions or know much more than how to admin a WP site at best. SOHO solution would be ideal imo. Great work! I'm excited to see where this goes.

eru 9y ago
I don't think the NAS will come at all: the economics just don't work out. It's more likely this will hit consumers hands via an automatically applied 'enhancement' in Google Photos first.
- dsnuh 9y ago
  An on-prem storage solution specifically targeting these features at professional photographers would sell. Storage is always a problem with every photographer I have met. They keep stacks of external USB drives or burn archives to DVD. They don't trust or want to use online storage. A package that provided these models, trainable on that photographer's particular style, with autocropping, watermarking, etc. would sell very well among this group.
  - eru 9y ago
    I bet you could make millions. Or even tens of millions. Great for a start-up. But Google doesn't bother with such small fry---especially if it comes attached to a complicated hardware business model. At these small numbers, they are more likely to just give away access to the software for free.

oscii 9y ago

Thanks! Interesting work, indeed. I found the showcase photos appealing and nice. I briefly skimmed through the paper with hope to find some interpretation of the “aesthetics”, but couldn't find it. What is meant under “all aesthetic aspects” and “aesthetic quality”? Did you perform a critical analysis of the network? How do you know that the network learned “aesthetics”?

necessity 9y ago

Still lame, just cropping and increasing saturation/contrast. Some of the pictures in the blog post look worse than the originals, they are artificial.

CWuestefeld 9y ago

Apologies for being that guy... Was there any control for, e.g., the dynamic lighting filter being the factor that carried most of the water, rather than the other manipulations? FWIW, I did think your tool did a good job finding good compositions within mundane scenes. Looking at the originals, I have to admit that in many or even most of the cases, I wouldn't have spotted the opportunity - which points to an area I can improve my craft.