9 ms·
Pose Animator: SVG animation tool using real-time TensorFlow.js models
- terpimost 6y agoGreat to see this thing open sourced. Adobe has a product for such things.
- nitsky 6y agoThis is awesome! Great work, thanks for sharing.
- ml_basics 6y agoThis is great! It's amazing to see what can be made out of open sourced machine learned building blocks
- kevmo314 6y agoAwesome stuff! I was just looking at face-api.js the other day, this kicks it up a notch.
- BIackSwan 6y agoThis is freaking amazing. So many potential applications. Great work!
- airstrike 6y agoI, for one, am ready for the metaverse
- pmayrgundter 6y agoExactly :)
- hprotagonist 6y agodo let me know when you’ve got a decent motorcycle model up and running.
- pmayrgundter 6y agoThe annotated SVG skeleton is a separate download, but that server is out of quota :( Someone have it?
- deleted 6y ago[deleted]
- yohann305 6y agoyou can find about 5 svg skeletons inside the github's folder: resources/illustration/ Have fun
- scruffups 6y agoWhy is this down voted? I hate it when people do that and offer no explanation. Totally useless input, even harmful. HN should force comment on down-vote.
- airstrike 6y ago> HN should force comment on down-vote This is actually a great idea
- scruffups 6y agoMy other comment on this thread followed others in congratulating the author. It specifically said "Bravo!" and guess what? It got down voted. So the explanation may be that it is useless input but believe me it is not useless to the author to have people congratulate them for the fruit of their labor. It's a human desire to be recognized by others. How is it not a positive comment? Why down-vote something as benign and empathic as saying congratulations? So force a comment on each down-vote and things might get a little bit better because at least we'll know it's not just random acts of hostility and that the down-voter has a rational reason for it, at least in their mind.
- dang 6y agoPosting like this breaks the site guidelines. Would you please read them? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html One reason that guideline exists is that unfair downvotes frequently get canceled by users who come along, see the situation, and make a corrective upvote. Meanwhile complaints like this linger on in the thread, inaccurate and off-topic—they don't garbage-collect themselves. As an example, I noticed your other comment and upvoted it before I saw this comment here. Similarly, other users have upvoted the GP. As with any stochastic process, there is a lot of error and spillage with downvotes. There's no way to perfect it; you have to ask whether the system is better off with it than without it. Forcing comments wouldn't help, and posting complaints certainly doesn't help.
- nine_k 6y agoNext step: output to a deep-fake synthesizer instead of SVG. Jump from animation to cinema.
- deleted 6y ago[deleted]
- scruffups 6y agoBravo!
- runawaybottle 6y agoAwesome, this is getting the idea mill in my head started. Animated short comics would be a great use case. Need some South Park vector models!
- TazeTSchnitzel 6y ago> This is not an officially supported Google product. Does that indicate the author is a Google employee who happened to make this in their free time, and has to say this somewhere by Google policy?
- youeseh 6y agoMaybe they used their 20% time on it.
- jpalomaki 6y ago"Unless your project is an official Google product, you must state “This is not an officially supported Google product” in an appropriate location such as the project’s README file." https://opensource.google/docs/releasing/publishing/ https://opensource.google/docs/releasing/publishing/
- rw2 6y agoWow, this would change how animation is done completely! If there existed an easy way to create animated character cartoons. It would launch a thousand southpark/rick and morty type of shows. A team with 10k can launch a show.
- tyingq 6y agoOn the other hand, I expect a rash of poorly done "explainer" type videos coming.
- jjeaff 6y agoHeh, literally my first thought when I saw the sample gifs was that I could use this to make some explainer videos. And yes, they would probably end up looking poorly done.
- nkozyra 6y agoI believe Adobe has had this for a long time in character animator https://www.adobe.com/products/character-animator.html https://www.adobe.com/products/character-animator.html
- egypturnash 6y agoSince 2015, according to Wikipedia. https://en.wikipedia.org/wiki/Adobe_Character_Animator https://en.wikipedia.org/wiki/Adobe_Character_Animator They won an Emmy for it this year, their post about it (https://theblog.adobe.com/adobe-character-animator-receives-emmy-award-for-technology-and-engineering/ https://theblog.adobe.com/adobe-character-animator-receives-...) notes that it was used for a live broadcast of the Simpsons, and a Showtime series called "Our Cartoon President" (https://www.sho.com/our-cartoon-president https://www.sho.com/our-cartoon-president).
- dharma1 6y agoAdobe Character Animator is not bad, has skeletal rigging and tracks mouth but doesn't do deformations like this one
- 6y ago
- p4bl0 6y agoIt would be cool privacy-wise to have some sort of virtual webcam that could share such an animation rather than what the webcam actually sees.
- njsubedi 6y agoIt sounds like a cool idea. Probably a platform to act on a play/script as the characters themselves? Like, a social theatre kind of thing with pre-animated background and stuff, where you only do your character part.
- p4bl0 6y agoAha, that's a completely different application than what I had in mind (just an alt webcam that's better than a static picture for communication while still protecting your visual identity), but it would be amazingly cool =).
- phkahler 6y agoOBS studio can do that.
- coderesearch 6y agoPlease more details - I can not find any feature on the OBS website similar to what we see here. Do we need some plugin / extension? Please give us some urls, thank you!
- sho_hn 6y agoThis is fairly easy to wire up on Linux using v4l2loopback and pyfakewebcam. I am currently using a little setup that uses OpenCV to acquire frames from the real camera, TensorFlow/BodyPix to compute an alpha mask for the foreground (me) and then OpenCV again to transform and composite myself behind news desks and into car infotainment screens and the like, eventually writing it to a virtual webcam I can use from Zoom (over its own virt bg feature this adds the layering and perspective transforms), Jitsi, Teams, etc. The above looks like another fun thing to add. Time to go full Who Framed Roger Rabbit? ...
- ravila4 6y agoThis is very cool! I wonder if using input from multiple cameras, plus a 3D rig can improve the accuracy of the rendering.
- Budabellly 6y agoCertainly, there are lots of apps leveraging the TrueDepth sensor on iPhones that reach an even higher level of fidelity as far is pose estimation is concerned. The limiting factor for accuracy in a lot of these technologies is the actual rigging process of the characters, probably because that is very difficult to standardize or generalize across different geometries, art styles, animation drivers, 2D vs 3D, etc.
- saadalem 6y agoThat's what a Side-Project should look like, curious to see the blog post !
- ghego1 6y agoThis is incredibly awesome
- zanybear 6y agoI would really like this imposed on a Teams, Zoom, Lync, WebMeeting. This way you cam still be in pajamas...
- crazygringo 6y agoThis is like creating a live version of author drawings, e.g. of New Yorker contributors: https://www.newyorker.com/contributors https://www.newyorker.com/contributors How interesting would it be to have your own live emotionally expressive avatar for videoconferencing, when you don't want to worry about your hair, makeup, lighting, or general visual state at all?
- jfyne 6y agoDavid Foster Wallace ruminated on this in Infinite Jest. "The proposed solution to what the telecommunications industry's psychological consultants termed Video-Physiognmoic Dsyphoria (or VPD) was, of course, the advent of High-Definition Masking. Mask-wise, the initial option of High-Definition Photographic Imaging — i.e. taking the most flattering elements of a variety of flattering multi-angle photos of a given phone-consumer and‚ thanks to existing image-configuration equipment already pioneered by the cosmetics and law-enforcement industries — combining them into a wildly attractive high-def broadcastable composite of a face wearing an earnest, slightly overintense expression of complete attention." Always thought it was fascinating that he came up with this in 1996!
- draugadrotten 6y ago"You can look like a gorilla or a dragon or a giant talking penis in the Metaverse. Spend five minutes walking down the Street and you will see all of these. Hiro's avatar just looks like Hiro, with the difference that no matter what Hiro is wearing in Reality, his avatar always wears a black leather kimono. Most hacker types don't go in for garish avatars, because they know that it takes a lot more sophistication to render a realistic human face than a talking penis. Kind of the way people who really know clothing can appreciate the fine details that separate a cheap gray wool suit from an expensive hand-tailored gray wool suit.", Neil Stephenson, Snowcrash, 1992.
- viraptor 6y agoYou can do that with some premade and custom models via https://facerig.com/ https://facerig.com/ already. They're not perfect, but they aree pretty expressive.
- 6y ago
- marcsto2 6y agoThis is awesome!
- iseanstevens 6y agoThis is great :) 10 years ago I'd been using Animata http://animata.kibu.hu http://animata.kibu.hu + kinect, this rolls it all into one web based thing. cool stuff. Would it be easy to move it to a local GPU accelerated version for more FPS?
- soylentgraham 6y agoHmm tfjs does use the gpu, and there are plenty of small models for posenet to run fast on mobile. Maybe this is an old version. (Or maybe the bottleneck is in the face or svg stuff) But to answer your question, yeah it could be faster (everything could be faster given enough time! :)
- mkchoi212 6y agoWow really cool! I'm wondering how much work it is to define the specific anchor/interest points on the SVG for the correct mapping to occur. But I guess since "modern" illustrations are quite minimal, said work probably shouldn't take too long.
- tianshuo 6y agoHead-turning doesn't work that great, live2d looks much better here.
- gitgud 6y agoWorks okay for a browser application, but are there any native alternatives for this?
- mbalex99 6y agoIs this very similar to the tech that powers Adobe Character Animator?
- garfieldnate 6y agoIIRC there was a similar thing done for one of the VR systems recently, but for the hands instead of for the face. Has anyone seen any open models or software for tracking hand shape and location in this way? I would love to use it in connection with sign language processing.
- sethjgore 6y agoYep! I do have something bookmarked somewhere. I use sign language myself and do want the same thing. Curious about your background
- garfieldnate 6y agoIf you find that bookmark could you share it with me? One project I've looked at is sign puppet [1]. It has all of the animation basics needed; the tough thing is inputting the parameters to animate the puppet. Traditionally capturing sign language data for the computer requires really sophisticated tracking equipment (gloves, etc.). Being able to do it with a web cam could be a game changer! I studied linguistics and CS in school, and I learned a little JSL to speak with deaf friends in Japan. I think sign language processing is a really neat combination of computer vision/graphics and linguistics. Lately there have been so many great advances in speech processing, but there hasn't been a huge leap forward for sign language processing, though I feel there should be. Deaf people are already really disadvantaged in many places, and getting left behind technologically doesn't help. I really resented looking for JSL books in the "disabled" section of the book stores in Japan, and when I spoke with some people about JSL, they didn't believe it was its own language. Even just linguistic work for sign languages is limited; I haven't seen a single reference grammar (re: comprehensive documentation) on any sign language. I think the difficulty of working with sign language data makes it more daunting to work with. (Paucity of speakers is certainly not a deterrent for linguists.) [1] https://github.com/aslfont/sign-puppet https://github.com/aslfont/sign-puppet