18 ms·
‘2001: A Space Odyssey’ rendered in the style of Picasso
- yxlx 10y agoLike someone said about this on /r/programming: >Pretty tight that computers can drop acid now. Anyway, here's a direct link to the video for mobile users: https://vimeo.com/169187915 https://vimeo.com/169187915
- tunnuz 10y ago"Oh my God, it's full of layers."
- ggchappell 10y agoCool. It needs some kind of averaging with nearby frames (or whatever), to avoid the constant flicker in areas of more or less solid color.
- minimaxir 10y agoGiven the movie being represented, the constant flicker is thematically more appropriate.
- woodman 10y agoIt isn't appropriate for cubism though, which is an attempt to render multiple perspectives to a single point of view. In this case time is the only extra dimension available, so it is a missed opportunity. But if the goal isn't a cubist rendering, just an interesting filter - mission accomplished.
- nsimoneaux 10y ago"It means nothing to me. I have no opinion about it, and I don't care." On the first moon landing, quoted in The New York Times, (1969-07-21). https://en.wikiquote.org/wiki/Pablo_Picasso https://en.wikiquote.org/wiki/Pablo_Picasso Curious about his feelings regarding this work. (I find it beautiful.)
- JoeAltmaier 10y agoFunny! That's exactly how I think of Picasso's work.
- prawn 10y agoHave you seen Guernica in Madrid?
- RBerenguel 10y agoOr for that matter, his "training period" works. I'm not a fan of abstract (Guernica, of course, is a symbol) and he was a very fine painter even taking off the weird eyes-out-of-face stuff he did (which is not to say it is bad, but it's not my style).
- mildbow 10y agoI have. And I still echo joe's comment.
- drzaiusapelord 10y agoJesus what a curmudgeon. I love how solidly he bought into the whole pretentious modern 'artiste' personality that is still being emulated today. I guess having a world weary 'fuck you' attitude is probably the social signaling you need to do to get your stuff into high-end galleries.
- scrumper 10y agoDid he buy in, or was he the source? A bit like how you can hear Chuck Yaeger in every US airline pilot's calm drawl.
- drzaiusapelord 10y agoEuropean bohemianism predates Picasso. I've always assumed that attitude was just another flavor of bohemianism.
- stepvhen 10y agoI have two opinions: 1) I don't think cubism transfers well into a motion picture format, 2) I think these experiments, as they are currently, attempt to merge two styles and end up with neither, and nothing novel in its place; there is little Kubrick or Picasso in the final piece. I think it's superficial and doesn't do either source justice.
- minimaxir 10y ago"It looks cool" is sufficient excuse for experimentation.
- Hermel 10y agoYes, but it is no excuse to call it "in the style of Picasso" when it clearly isn't.
- the_cat_kittles 10y agothis is such a quintessentially HN comment
- TaylorAlexander 10y ago"I am well informed and unimpressed." Also, for fairness, do the same with my comment: "Unlike this status oriented nerd, I can relate to you and the author on a human level."
- 10y ago
- 2bitencryption 10y agoI kind of want someone to do the same thing with a "NON-neural network" Picasso filter, like the ones in Photoshop and similar image editing programs. I want to compare how much the neural network's understanding of Picasso's style adds to the work (I imagine it's a lot, because this looks incredible).
- LoSboccacc 10y agoI'd like to see different styles personally. This generation of deep style will keep too much of the shape to be a picasso style. There is no decomposing and superimposing points of views because, well, the data to do it it's just not there.
- jamesrom 10y agoA whole new debate about copyright is around the corner.
- fratlas 10y agoSurely this comes under fair use because of modification?
- danso 10y agoYeah, it's hard to think of how this could not be an example of transformative use but I'm sure I'm not legally creative enough to assert that: http://fairuse.stanford.edu/overview/fair-use/four-factors/#the_transformative_factor_the_purpose_and_character_of_your_use http://fairuse.stanford.edu/overview/fair-use/four-factors/#... In a 1994 case, the Supreme Court emphasized this first factor as being a primary indicator of fair use. At issue is whether the material has been used to help create something new or merely copied verbatim into another work. When taking portions of copyrighted work, ask yourself the following questions: Has the material you have taken from the original work been transformed by adding new expression or meaning? Was value added to the original by creating new information, new aesthetics, new insights, and understandings?
- siegecraft 10y agoI see this as similar to colorizing a black and white movie, so maybe not.
- habosa 10y agoThe big changes frame-to-frame certainly add to the "trippiness" but I'd love to see this where the value function (or whatever it's called for ML) prioritizes reducing the frame-to-frame diff so that I could actually watch the full length movie like this.
- jamesdwilson 10y agoSerious question: how is this different than one of the many photoshop filters that could be applied iteratively to each frame?
- minimaxir 10y agoThere likely isn't a Picasso Photoshop filter that can match the fidelity of the style transfer used in the video. (I checked; the best you can get is a macro to automate faux-cubism)
- jamesdwilson 10y agook, if there was, would there be a difference?
- golergka 10y agoCan someone knowledgeable estimate, how far are we from rendering this in 60 frames per second? Can't wait to try it as a post-processing layer in game development.
- minimaxir 10y agoNot happening in this lifetime at a meaningful image resolution. The style-transfer implementations require very large amounts of GPU processing. If you pay attention to the video, the images are heavily upscaled.
- fudgie 10y agoA paper called Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks (http://arxiv.org/abs/1604.04382 http://arxiv.org/abs/1604.04382) uses a different approach with similar results and runs about 500 times faster giving 0.25M pixel images at 25Hz. It also seems to result in less randomness from frame to frame https://www.youtube.com/watch?v=PRD8LpPvdHI https://www.youtube.com/watch?v=PRD8LpPvdHI Code at https://github.com/chuanli11/MGANs https://github.com/chuanli11/MGANs
- golergka 10y agoThanks a lot! That's exactly the kind of comment I love HN for.
- TrevorJ 10y agoWould be interesting to see if they could reduce the temporal noise without compromising the overall effect.
- slavik81 10y agoI wonder if you could extract a 3D scene from the video and somehow style all the textures rather than styling the frame. Then reproject back and use that to style the frame.
- mgraczyk 10y agoI remember watching an interview with the creators of South Park in which they described the transition from animating using cardboard cutouts to a system with CorelDraw and other pieces of software which helped speed up the process. The bulk of the efficiency improvement came from carefully defining all the frequently used objects (characters, houses) once with movable components, and reusing those objects in the per-episode animation pipeline. I can easily imagine an animation system like the one presented here enabling another massive improvement in animation efficiency. In the same way animation software allowed South Park to reuse pre-drawn objects, a deep learning system could enable south park to carefully define the entire drawing style just once, then generate complete episodes based on simple story boards and animation directives. Fortunately, South Park already has a significant amount of training data available, specifically every South Park episode yet produced.
- jkscm 10y agoAnother interesting problem would be the generation of filler pictures (I don't know the correct term). Normally there is a person who draws keyframes at a much lower framerate. Other animators then fill in the frames between to increase the framerate.
- shiro 10y agoIt's called in-between. There is certainly an art form of its own, for a simple mechanical interpolation won't produce visually pleasing animation. Disney has some research on parameterizing in-betweening. These days 2D animation are replaced with 3D and the art of 2D in-betweening might be lost in future. It'll be interesting if DN can learn animation styles from existing footage (e.g. Kanada-style).
- veli_joza 10y agoThey're called inbetweens or tweens. According to recent HN article, they are outsourced to South Korea. Generating them with algorithm would be interesting, but often incorrect and against artist wishes. For example, objects in motion sometimes needs to be blurred; some characters need to have ghost duplicates; shapes get distorted and exaggerated. I think at this moment it is not possible to instruct algorithm to take additional suggestions (artist ideas) into consideration when creating output image.
- jdblair 10y agoThis is amazing. That said it doesn't have the distorted perspective I think is a hallmark of Picasso's work.
- jjcm 10y agoI remember when The Scanner Darkly came out there was a lot of talk about how they achieved the style of the film. Some of it was automated, but a lot still had to be hand done. I wonder if using deep learning systems we could achieve the same effect that film had with nearly zero human interaction. For those that haven't seen the movie, here's the trailer: https://www.youtube.com/watch?v=TY5PpGQ2OWY https://www.youtube.com/watch?v=TY5PpGQ2OWY
- madethemcry 10y agoI think you should know `Waking Life` (2001) as the prototype for that stylistic approach: https://www.youtube.com/watch?v=uk2DeTet98o https://www.youtube.com/watch?v=uk2DeTet98o
- mockery 10y agoThis is cool, but the frame-to-frame variance is distracting. I really want to see this reimplemented with temporal constraints a-la this paper: https://www.youtube.com/watch?v=Khuj4ASldmU https://www.youtube.com/watch?v=Khuj4ASldmU
- pvinis 10y agoI hadn't seen that before. This looks great.
- robbiep 10y agoWow
- grabcocque 10y agoI want to see them all now.
- deleted 10y ago[deleted]
- thope 10y agoI wish I had time to set up a computer and compile and run this https://github.com/manuelruder/artistic-videos https://github.com/manuelruder/artistic-videos
- onetwotree 10y agoOh wow, that looks amazing! I wonder how long it takes to transform those clips though...basic neural style transfer takes a few minutes on a standard gaming rig, so it probably takes a big machine or a few hours to do this.
- kodfodrasz 10y agoSo basically you take someone else's work. Run it with some content (someone else's work also), post it, and wow innovation. Actually in the last year myriads of similar things were created, and this is simply boring. This is as interesting as a random tumblr reblog. May be curious, but lacks any sense of achievement or originality.
- dredmorbius 10y agoIf you read up on research into innovation, you'll find that much of it does hinge on appropriating ideas from elsewhere. It's more like bacterial gene transfer than procaryote sexual reproduction, and very little like random point variations. John Holland, W. Brian Arthur, and Doyne Farmer (all associated with the Santa Fe Institute) have done much interesting work. Farmer talks of Theodor Wright, and his studies of aircraft manufacturing improvements during WWII, though much of Farmer's discussion seems to borrow heavily from Wright's browther Sowell, a geneticist looking at genetic drift and fitness landscapes. Quincy Wright's study of war also seems apropos. Interesting family.
- kodfodrasz 10y agowow I can write a for loop for the frames of a film and call the google open sourced deep learning style transfer program. INNOVATION!
- Udik 10y agoThere is something that escapes me regarding this very cool neural style transfer technique. One would expect it to need at least three starting images: the one to transform, the one used as a source for the style, and a non-styled version of the source. This last one should give the network hints on how to transform the unstyled version in the styled one. For example, what does a straight line end up being in the style? Or how is a colour gradient represented? Missing this, it seems that the neural network should be able to recognize objects in the styled picture, and derive the transformation applied based on a previous knowledge of how they would normally look like. But of course the NN is not advanced enough to do that. Can someone explain me roughly how does this work?
- vintermann 10y agoIt's right that it needs additional information to distinguish style from content, but they get that from selected layers from established, pre-trained neural nets for image recognition. I don't entirely understand why it works myself, but it seems to.
- brian_cloutier 10y agoFrom the way you described it, you could consider the pre-trained network to be the "missing image". It already has an idea of what images should look like so when it detects an object the "style" is what makes that object different than the stereotypical one it's already modeled.
- vintermann 10y agoRight. But it's more complicated than that, the choice of layer(s) to use matters a lot, and I have no idea why they do as they do. Seems it's a bit of dark magic to get it to work well - takes lot of aesthetic judgements too. I think Alex J. Champandard's implementation is probably the best one out there right now. It has a ton of knobs to twist and is very fast.
- gh2k 10y agoI'm learning about ANNs at the moment, so I'm not really getting this but I'd like to. Aside from the 'missing image' part, what is the fitness example here for training? How does the training process determine what a good image is given there aren't many (any?) examples of a source image -> picasso mapping?
- shiro 10y agoIt certainly has a wow factor, but once you pass the initial impression, it's interesting that the brain starts recognizing the content (motion of characters and objects) separately from the visual style, and even starts applying negative cubism filter so that we won't actually see the visual style anymore. (In other words, the brain treats those applied style as noise.) It could be a way to exploit the mismatch of content and style as certain form of expression; but it may be more interesting if we can modify the temporal structure as well.
- auggierose 10y agoAwesome. Just the black monolith should stay black :-)
- cooper12 10y agoYou just pointed out the biggest limitation with this. Even if it can apply the style, that's all it is doing. It doesn't understand the context or content of the film. Everyone knows in the first fifteen minutes of the film of the importance of the monolith and how imposing it is, but the algorithm doesn't. Even image recognition couldn't help you there. In visual art where filters and different media are used, it is rare to non-selectively apply them. Computers with cognition and creativity are a long, long time away. That is why I can't stand people who are scared of some skynet scenario; even if AlphaGo mastered Go or Deep Blue mastered Chess, real-life problems are not so strictly defined and require much more intelligence and insight.
- fractallyte 10y agoPossibly the finest painting software currently available is Synthetik's Studio Artist (http://synthetik.com/ http://synthetik.com/). Compared to Adobe's powerhouse software, it's relatively unknown, but that doesn't make it any less innovative. It uses an algorithmic 'paint synthesizer' to generate brushes (with hundreds of presets) and auto-paint canvases, and is designed for animation (rotoscoping) as well as static artwork. The output can be reminiscent of the style of the movie 'A Scanner Darkly', but the software is hugely flexible. Here are a couple of rather amazing examples: http://studioartist.ning.com/video/auto-rotoscoped-dancers http://studioartist.ning.com/video/auto-rotoscoped-dancers and http://studioartist.ning.com/video/dance-styles-animation-reel http://studioartist.ning.com/video/dance-styles-animation-re... Also, unlike most other 'painterly' software, the graphics are resolution independent - meaning that they can be scaled up to any size without loss of detail.
- rorygreig 10y agoI wonder how long it takes to render each frame. Eventually with fast enough GPUs you could render a video game in this style, now that I would like to see.
- elcapitan 10y ago"Poetry is what gets lost in translation", "Art is what gets lost in machine learning". I think it's interesting that it's possible to create basically filters from existing images, but then applying those filters to large amounts of images (like in this movie) quickly loses the novelty effect and is just as boring as any photoshop or gimp filter became in the 90s after seeing it 3 times. When I look at Picassos actual pictures, I am astonished and amazed with every new one I get to see. With these pictures, I get more and more bored with every additional image.
- jtr1 10y agoOne of the most striking things missing is Picasso's play on perspective. Often his paintings would look at the subject though multiple points of view at the same time. Or he would break apart shapes, reorient them, and put them back together to get at some underlying idea. Watching Dave jog along the perimeter of Discovery One in perfect perspective sort of undermines the whole effect. Even though the images are painted over with Picasso-like textures, colors, and shapes, it doesn't really _look_ like the real thing. That said, even if I think it falls short of capturing exactly what makes a Picasso a Picasso, I still think it's pretty cool.
- slr555 10y agoI am much more of an artist than a technology person and the rendering inconsistency the author refers to is one of the coolest aspects of the video. This is the kind of happy accident that gives work originality and makes it more than a slavish copy. Reminds me of Link Wray putting a pencil through the speaker cone of his amplifier.
- stcredzero 10y agoIs it just me, but have all forms of art simply melded with self-promotion? (Melded in the sense found in the movie "The Fly.")
- 6stringmerc 10y agoNot trying to over-state my qualifications to make the following claim, but I'm pretty sure Kubrick would have hated this. And, as such, have it destroyed.
- rurban 10y agoHis http://bhautikj.tumblr.com/tagged/drumpf http://bhautikj.tumblr.com/tagged/drumpf is much better. Donald Drumpf as sausage
- onetwotree 10y agoNeural style transfer is extremely fun to play with. If you have a system with a recent-ish graphics card (I'm doing fine with my GTX 970), put a linux on it and check out the many GitHub projects that implement this stuff (some of the tools will only work on linux). It's a great way to start learning about deep learning and GPU based computation , which are starting to look like very good things to have on your resume. Plus, you get to make cool shit like this that you can actually show to your friends. I'm getting more interested in the text generation stuff as well too - I'd love to make a Trump speech generator :-)