16 ms·
Drag Your GAN: Interactive Point-Based Manipulation of Images
- waqasy 3y agoDragGAN consists of two main components including: 1) a feature-based motion supervision that drives the handle point to move towards the target position, and 2) a new point tracking approach that leverages the discriminative GAN features to keep localizing the position of the handle points.
- ArekDymalski 3y agoAs a technology this is tremendously impressive, straight out of SF movie. However I wonder how it will impact our culture, fashion, standards of beauty etc. when more and more artists will be accepting the generated output then creating their own. Just like in case of music the invention of MIDI, synths and sequencers brought new styles but also boring and imagination-numbing standardization.
- Scene_Cast2 3y agoNeat concept. I wonder if something like that can be applied to diffusion models (the new kid on the block that is outshining GANs right now) - especially since the technique doesn't seem to be too dependent on the generative image implementation. Also, it's interesting that they're submitting to SIGGRAPH - kind of expected this to be in a more ML-ish conference.
- cubefox 3y agoProbably goes to show where SIGGRAPH is headed.
- nine_k 3y agoHello, post-truth world! More seriously, I think that digital photos, and especially low-res surveillance camera coverage, will soon be inadmissible in any reasonable court, because tools like this would allow to forge such evidence in very natural-looking ways.
- politelemon 3y agoSeeing how slowly laws move, the cynic says it's: should be inadmissible in any reasonable court, but will continue to be admissible, and will take a major set of incidents for changes to be enacted across many countries.
- nine_k 3y agoYes. The story if admissable DNA evidence is instructive and terrifying. https://daily.jstor.org/forensic-dna-evidence-can-lead-wrongful-convictions/ https://daily.jstor.org/forensic-dna-evidence-can-lead-wrong...
- imranq 3y agoThere are probably ways to embed cryptographic hashes within images. Any device that creates images from the real world could have secret keys that can be used to validate any image created by said device. We will still need a centralized party that holds the secret keys for validation through
- bick_nyers 3y agoWhat about the recording device itself? Refeed a video/frame/hash back to a security camera and it tells you if it was originally sourced from that specific camera or not.
- foota 3y agoThe sci-fi dystopian answer would be entangled photon lights and off site image recording that preserves entanglement :-)
- ChainReaktion 3y agoThis is the right approach, but there’s lots of complexity around transcoding. In the courtroom that’s less of an issue if you can get the original unmodified outputs, but broader applications need to think through what it means to be “verified”
- johndough 3y agoProject website mirror https://web.archive.org/web/20230519060439/https://vcai.mpi-inf.mpg.de/projects/DragGAN/ https://web.archive.org/web/20230519060439/https://vcai.mpi-... GitHub (no code yet, only demo GIF) https://github.com/XingangPan/DragGAN https://github.com/XingangPan/DragGAN arXiv https://arxiv.org/abs/2305.10973 https://arxiv.org/abs/2305.10973
- ortusdux 3y agoDemo video: https://twitter.com/_akhaliq/status/1659424744490377217 https://twitter.com/_akhaliq/status/1659424744490377217
- bbminner 3y agoThere's also an older work called PuppetGAN http://ai.bu.edu/puppetgan/ http://ai.bu.edu/puppetgan/
- Qweiuu 3y agoIt's similar but different. Your paper makes an existing body a puppet. The other one adjusts features.
- vagabund 3y agoThe semantic understanding feels much richer than diffusion based modeling, e.g. the trees on the shore growing to match the manipulated reflection, the sun changing shape as it's moved up on the horizon, the horse's leg following proper biomechanics as its position is changed. I haven't gotten such a cohesive world model when doing text-guided in-painting with stable diffusion etc. This feels like it could very conceivably be guided by an animation rig with temporally consistent results.
- orbital-decay 3y agoTemporal consistency for a guided scene is a separate problem. It's been kind of solved a couple years ago. [0] It can be used with animation rigs and simplistic tagged geometries, and it even works in near real-time. "Kind of" because training a model from scratch from a large dataset is not something you want to do for the actual job; what you want is a good style transfer mechanism that can extract features from as few references as possible. [0] https://isl-org.github.io/PhotorealismEnhancement/ https://isl-org.github.io/PhotorealismEnhancement/
- deleted 3y ago[deleted]
- sroussey 3y agoWould love to see this for architecture!
- tikkun 3y agoHas anyone built the "online photoshop that incorporates all of the latest AI image editing tools asap and sells access as a premium subscription with lots of GPU access for smooth editing" business yet? I'd be curious to know.
- ultra_nick 3y agoIsn't that stability.ai's business model?
- apodolny 3y agoPlayground AI (https://playgroundai.com/ https://playgroundai.com/) does a lot of this.
- echelon 3y agoThere are a million of these. It's a super crowded space. https://civitai.com/ https://civitai.com/ https://lexica.art/ https://lexica.art/ https://openart.ai/ https://openart.ai/ (Many more)
- vaidhy 3y agoThey are all for image creation. I would love to have one to edit my photos.. not generate images from prompt.
- nadermx 3y agohttps://ImageEditor.ai https://ImageEditor.ai
- dragonwriter 3y agoYou can edit your photos with the models and tools for image creation, because those tools can work on a source image (img2img, inpainting, outpainting) as well as just a prompt and blank canvas.
- jahewson 3y ago
- t3estabc 3y ago[dead]
- loandbehold 3y agoWhat do we need human actors for at this point? Everything can be generated by AI now.
- u385639 3y agoPlease try to make an original movie that meets the standard of, say, The Godfather, with AI.
- yamazakiwi 3y agoI understand your point but I think it would be easier with AI than without. Many movies are not made to the standard of The Godfather because they don't sell like MCU Movie #53 and if you include more humans in the creation you're more likely to run into the current system's restrictions. Making a movie as beloved as the Godfather would still be challenging of course.
- u385639 3y agoOf course it would be easier. I agree. I just take issue with the "why humans" thing because if anything, the recent advancements highlight just how big the human element really is. Can you imitate a Bach prelude? Sure. And only people who aren't actually familiar with his music would be impressed. Much of AI approaching "human performance", is it approaching the lowest bar. There's a Wittgenstein thing going on here. That an LLM can ace the LSAT or GMAT is mostly an indictment of those tests. A little off topic.
- famouswaffles 3y ago>That an LLM can ace the LSAT or GMAT is mostly an indictment of those tests. These kind of comments are always the funniest. You can just tell the person who makes them has never looked at those tests nevermind attempted them.
- u385639 3y ago
- mft_ 3y agoThe main link you provided is either not loading, or loading with missing video links, for me - maybe hugged to death at the moment? Github may be more resilient: https://github.com/XingangPan/DragGAN https://github.com/XingangPan/DragGAN
- krunck 3y agoI'm thinking that soon video and images are going to be just dead weight in journalism, adding nothing other than decoration.
- radarsat1 3y agoIt seems to be a pretty common thing now on news network websites, that at the top of the story, or perhaps somewhere in the middle, there is a video. But, you click on the video, and it's something entirely unrelated to the article. I feel like this as been going on for quite a while, nothing to do with synthetic media, but just an observation of an annoying pattern I've noticed.
- chpatrick 3y agoI think there has to be some kind of cryptographic signature solution, like "The BBC verifies that this image is authentic".
- mrshadowgoose 3y agoWe already have most of the required cryptographic primitives for this. PKI, trusted timestamping, secure hardware with remote attestation are some of the necessary building blocks. All that's really missing are camera sensors with built-in cryptography. And societal care. Our society seems to really like to whine about "the danger of deepfake images", but our actions reveal that we don't really give a crap, as we could solve this problem today if we really wanted to.
- chpatrick 3y agoI think it would also need to have some kind of embedded version control from camera through Photoshop so you know what was changed.
- mejutoco 3y agoIf you trust how the bbc tells the story, you also trust them when they say the image is true. Not a technical problem IMO.
- Zetobal 3y agoThe only thing that's new is the interactive interface the rest of it is old tech... You can use it on art breeder.com. Photoshop even has it in their face neural filter. GANs are not feasible for a variety of reasons you need to have models specific to your subject ie. why they change to a elephant model to manipulate the elephant.they are also not style agnostic but it's a great demo and the right time to release it. Just before the summit of the hype curve I bet one VC is dumb enough to throw millions at them.
- chatmasta 3y agoWith ChatGPT, the only thing new was the chat interface. In fact even Sam Altman mentioned this on Lex Fridman's podcast, IIRC - he said what he was most surprised about was the outsized effect the interface had on bringing LLM to the forefront of public consciousness, despite the existing maturity of the underlying GPT models. At least in that case it was OpenAI adding interactivity to its own existing models. But similarly, from a more holistic viewpoint, OpenAI productized existing research from Google. Transformer models were "old tech" since Google published "Attention is all you need" in 2017... and yet, when OpenAI managed to turn it into a usable product, suddenly they became the first movers and the company to beat. So I'm not convinced that only a "dumb" investor would fund an effort with a proven ability to productize "old tech."
- Zetobal 3y agoThe I don't understand the technology but will ramble about stuff until they just give up reading the comment approach ¯\_(ツ)_/¯
- nullc 3y agoBest demo would be to take some famous photo, find its position in gan-space then move it around.