5 ms·

The Klimt Color Enigma

76 points by emilwallner 5y ago

egypturnash 5y ago
When I skipped down to the bottom of this page to look at the actual art under discussion, it gave me a popup that told me I had earnt a fucking badge for reading this page all the way through. I hate this fucking future.
Palomides 5y ago
does anyone else feel this is disrespectful to artists? having a computer make up colors to apply to art? it's completely devoid of the human intent that chose the original colors
- hairofadog 5y ago
  Speaking only for myself, no! It’s just borrowing and playing with ideas, which artists do all the time. (I have an education in fine arts, for whatever it’s worth [0]) [0]: nothing, both literally and figuratively
  - Palomides 5y ago
    I'm fine with playing with ideas, but the fact that it's google smacks of "look, the computer can replicate/replace the human", which has a very different vibe than, say, Dali coloring Goya
    - hairofadog 5y ago
      Yep, good point.
    - mechanical_bear 5y ago
      The timing, considering it was posted by someone connected to Google and in light of recent news, seems suspect.
- Daub 5y ago
  Not exactly made up. Having only the lightness values to start with, The task is to define the hue and saturation. Though in theory there is no way to know for certain what they originally were, in practice certain lightness values are associated with some hues more than others. Also, saturation values tend to follow lightness values quite closely. Add to that the fact that Klimt (like most painters) employed a habitual pallet, you are half was to a good guess.
- canjobear 5y ago
  They have human Klimt experts helping too.
- alisonkisk 5y ago
  Klimt is dead, and the originals are missing.
- woodruffw 5y ago
  I empathize with this position, but I think this situation is closer to authentic restoration (which the art world is okay with, generally speaking) than an unguided machine transformation of a human work. (I've visited just about every Klimt exhibit in Vienna, including the one that these B&W photographs are in. I think they've done a remarkably faithful job here, down to accurately capturing the subperiod of Klimt's "gold period").
- googlryas 5y ago
  It seems different when we're just trying to guess what the actual colors were, since the artist is dead and the painting is destroyed, and we only have a black and white copy + textual description of the painting.
- aeturnum 5y ago
  No, absolutely not. I think that one has to be careful with labeling - these works are highly researched guesses based on lots of sources but are not "the article itself." But I think this process, where they are trying to be a thorough as possible in guessing what colors Klimt might have used mechanistically, is much more respectful than other techniques. Consider the recent Vermeer restoration[1] (hn discussion at [2]): 'restoring' the painting in this way alters the actual article and could be altering the intent of the artist. I support the work, but I think there's a basic element of uncertainty to it. At the end of the day, if you don't want to accept the machine's guess at the colors Klimt used, you don't need to! You have access to the same archival documentation of what his lost paintings looked like as everyone else. [1] https://hyperallergic.com/672345/vermeer-restoration-finally-complete-reveals-a-painting-within-a-painting/ https://hyperallergic.com/672345/vermeer-restoration-finally... [2] https://news.ycombinator.com/item?id=28468802 https://news.ycombinator.com/item?id=28468802
woodruffw 5y ago
Normally I roll my eyes at these kinds of ML applications (I've seen a lot of haphazard applications of upscaling and auto-colorization to archival footage, resulting in really garish and unrealistic transformations), but this effort appears to have been thoughtfully performed (with references to Klimt's contemporaneous works and living experts).
kwelstr 5y ago
Great work, I greatly admire Klimt's art and these paintings were such an important part of his artistic development and so sad they were destroyed on purpose so the other side wouldn't have them. That website is a pain to read on a desktop computer thou... maybe ok on mobile?
- jbaber 5y ago
  I'm a big Klimt fan and am very impressed with this (especially Jurisprudence). I was thinking, though: "this is so hard to see on mobile. Maybe better on desktop."
  - ambyra 5y ago
    I can confirm it is terrible on both. What a great article though.
tartoran 5y ago
Mistery is like a magnet for the imagination. I personally like when a piece of the puzzle is missing: it makes us think.
mechanical_bear 5y ago
Quick!!! We need some good PR! Push the Klimt story!
tudorw 5y ago
"The algorithm is trained on 91749 artworks from Google Arts & Culture" I'd like to have a play with that, is it open or available?
- emilwallner 5y ago
  Unfortunately not, but here are a few great public art datasets: https://www.artnome.com/art-data https://www.artnome.com/art-data. Rijksmuseum and wikiart also have available datasets.
  - tudorw 5y ago
    thanks :)
emilwallner 5y ago
Thanks for the appreciation! I made the machine learning model and wrote the article, happy to answer any questions. Here's more information: - 3D Gallery: https://artsandculture.google.com/pocketgallery/kAUxTZBD8McZyQ https://artsandculture.google.com/pocketgallery/kAUxTZBD8McZ... - Video overview: https://www.youtube.com/watch?v=1xYpIM_BVTI https://www.youtube.com/watch?v=1xYpIM_BVTI - This work will be part of an exhibition at the Museum of Rome starting from 27th Oct, where you can also see Klimt's 'Portrait of a Lady' that was missing for almost 23 years - More Klimt artworks and articles: https://artsandculture.google.com/project/klimt-vs-klimt https://artsandculture.google.com/project/klimt-vs-klimt
- boulos 5y ago
  Awesome work! Btw, you could have also submitted this as Show HN :).
- zvr 5y ago
  Amazing work! Congratulations!
- CamperBob2 5y ago
  Is there anywhere the actual restored artworks in the article can be seen without having to navigate a lot of Web 3.0 gimmickry, or without having to scale up some 2000-era images the size of postage stamps?
  - emilwallner 5y ago
    - Jurisprudence: https://artsandculture.google.com/asset/jurisprudence-recolored-with-artificial-intelligence-gustav-klimt/1QHKcdPmTK6p_Q https://artsandculture.google.com/asset/jurisprudence-recolo... - Philosophy: https://artsandculture.google.com/asset/philosophy-recolored-with-artificial-intelligence-gustav-klimt/fAE8Sorss5TLiA https://artsandculture.google.com/asset/philosophy-recolored... - Medicine: https://artsandculture.google.com/asset/medicine-recolored-with-artificial-intelligence-gustav-klimt/wQH01qjWyeIaEQ https://artsandculture.google.com/asset/medicine-recolored-w...

76 points by emilwallner 5y ago

When I skipped down to the bottom of this page to look at the actual art under discussion, it gave me a popup that told me I had earnt a fucking badge for reading this page all the way through. I hate this fucking future.

Palomides 5y ago

does anyone else feel this is disrespectful to artists? having a computer make up colors to apply to art? it's completely devoid of the human intent that chose the original colors

hairofadog 5y ago
Speaking only for myself, no! It’s just borrowing and playing with ideas, which artists do all the time. (I have an education in fine arts, for whatever it’s worth [0]) [0]: nothing, both literally and figuratively
- Palomides 5y ago
  I'm fine with playing with ideas, but the fact that it's google smacks of "look, the computer can replicate/replace the human", which has a very different vibe than, say, Dali coloring Goya
  - hairofadog 5y ago
    Yep, good point.
  - mechanical_bear 5y ago
    The timing, considering it was posted by someone connected to Google and in light of recent news, seems suspect.
Daub 5y ago
Not exactly made up. Having only the lightness values to start with, The task is to define the hue and saturation. Though in theory there is no way to know for certain what they originally were, in practice certain lightness values are associated with some hues more than others. Also, saturation values tend to follow lightness values quite closely. Add to that the fact that Klimt (like most painters) employed a habitual pallet, you are half was to a good guess.
canjobear 5y ago
They have human Klimt experts helping too.
alisonkisk 5y ago
Klimt is dead, and the originals are missing.
woodruffw 5y ago
I empathize with this position, but I think this situation is closer to authentic restoration (which the art world is okay with, generally speaking) than an unguided machine transformation of a human work. (I've visited just about every Klimt exhibit in Vienna, including the one that these B&W photographs are in. I think they've done a remarkably faithful job here, down to accurately capturing the subperiod of Klimt's "gold period").
googlryas 5y ago
It seems different when we're just trying to guess what the actual colors were, since the artist is dead and the painting is destroyed, and we only have a black and white copy + textual description of the painting.
aeturnum 5y ago
No, absolutely not. I think that one has to be careful with labeling - these works are highly researched guesses based on lots of sources but are not "the article itself." But I think this process, where they are trying to be a thorough as possible in guessing what colors Klimt might have used mechanistically, is much more respectful than other techniques. Consider the recent Vermeer restoration[1] (hn discussion at [2]): 'restoring' the painting in this way alters the actual article and could be altering the intent of the artist. I support the work, but I think there's a basic element of uncertainty to it. At the end of the day, if you don't want to accept the machine's guess at the colors Klimt used, you don't need to! You have access to the same archival documentation of what his lost paintings looked like as everyone else. [1] https://hyperallergic.com/672345/vermeer-restoration-finally-complete-reveals-a-painting-within-a-painting/ https://hyperallergic.com/672345/vermeer-restoration-finally... [2] https://news.ycombinator.com/item?id=28468802 https://news.ycombinator.com/item?id=28468802

woodruffw 5y ago

Normally I roll my eyes at these kinds of ML applications (I've seen a lot of haphazard applications of upscaling and auto-colorization to archival footage, resulting in really garish and unrealistic transformations), but this effort appears to have been thoughtfully performed (with references to Klimt's contemporaneous works and living experts).

kwelstr 5y ago

Great work, I greatly admire Klimt's art and these paintings were such an important part of his artistic development and so sad they were destroyed on purpose so the other side wouldn't have them. That website is a pain to read on a desktop computer thou... maybe ok on mobile?

jbaber 5y ago
I'm a big Klimt fan and am very impressed with this (especially Jurisprudence). I was thinking, though: "this is so hard to see on mobile. Maybe better on desktop."
- ambyra 5y ago
  I can confirm it is terrible on both. What a great article though.

tartoran 5y ago

Mistery is like a magnet for the imagination. I personally like when a piece of the puzzle is missing: it makes us think.

mechanical_bear 5y ago

Quick!!! We need some good PR! Push the Klimt story!

tudorw 5y ago

"The algorithm is trained on 91749 artworks from Google Arts & Culture" I'd like to have a play with that, is it open or available?

emilwallner 5y ago
Unfortunately not, but here are a few great public art datasets: https://www.artnome.com/art-data https://www.artnome.com/art-data. Rijksmuseum and wikiart also have available datasets.
- tudorw 5y ago
  thanks :)

emilwallner 5y ago

Thanks for the appreciation! I made the machine learning model and wrote the article, happy to answer any questions. Here's more information: - 3D Gallery: https://artsandculture.google.com/pocketgallery/kAUxTZBD8McZyQ https://artsandculture.google.com/pocketgallery/kAUxTZBD8McZ... - Video overview: https://www.youtube.com/watch?v=1xYpIM_BVTI https://www.youtube.com/watch?v=1xYpIM_BVTI - This work will be part of an exhibition at the Museum of Rome starting from 27th Oct, where you can also see Klimt's 'Portrait of a Lady' that was missing for almost 23 years - More Klimt artworks and articles: https://artsandculture.google.com/project/klimt-vs-klimt https://artsandculture.google.com/project/klimt-vs-klimt

boulos 5y ago
Awesome work! Btw, you could have also submitted this as Show HN :).
zvr 5y ago
Amazing work! Congratulations!
CamperBob2 5y ago
Is there anywhere the actual restored artworks in the article can be seen without having to navigate a lot of Web 3.0 gimmickry, or without having to scale up some 2000-era images the size of postage stamps?
- emilwallner 5y ago
  - Jurisprudence: https://artsandculture.google.com/asset/jurisprudence-recolored-with-artificial-intelligence-gustav-klimt/1QHKcdPmTK6p_Q https://artsandculture.google.com/asset/jurisprudence-recolo... - Philosophy: https://artsandculture.google.com/asset/philosophy-recolored-with-artificial-intelligence-gustav-klimt/fAE8Sorss5TLiA https://artsandculture.google.com/asset/philosophy-recolored... - Medicine: https://artsandculture.google.com/asset/medicine-recolored-with-artificial-intelligence-gustav-klimt/wQH01qjWyeIaEQ https://artsandculture.google.com/asset/medicine-recolored-w...