7 ms·
Scientists Uncover Invisible Motion in Video
- 0x09 14y agoDiscussion from last June: http://news.ycombinator.com/item?id=4062216 http://news.ycombinator.com/item?id=4062216
- speeder 14y agoSomeone posted this same link yesterday, but I think I am with a ban that make my upvotes don't count, since the arrow go away and the vote count don't change. Goad that someone tried again and made this go to front page, I think it is vert amazing and awesome, maybe animators will be able to use this tech to learn how to avoid uncanny valley
- laurencei 14y ago"The system works by homing in on specific pixels in a video over the course of time. Frame-by-frame, the program identifies minute changes in color and then amplifies them up to 100 times, turning, say, a subtle shift toward pink to a bright crimson" So really the title should be "Scientist amplify motion in video" - because that is what is occurring. There is nothing "invisible" being discovered - it still needs a visual change.
- JonnieCache 14y agoInvisible here obviously means invisible to the human eye. This kind of deliberate point-missing is toxic to conversation, and it just makes you come across as childish.
- Evbn 14y agoAsking for the nations paper of record to use English properly isn't so bad.
- Volpe 14y agoOoh thanks for that, I was confused by the title and the video, but your pointless pedantics cleared it all up for me. Thanks!
- floobynewb 14y agolol
- nathan_long 14y agoYou can argue that this was "imperceptible" rather than "invisible". But you could say the same thing about things we only see with the aid of telescopes and microscopes. "The amoebas weren't invisible; that light was already bouncing up to your eyes." But really, invisible means "unable to be seen." Unless you can demonstrate that it's possible for a person to see these things unaided, they are invisible.
- archivator 14y agoThe paper from last year's SIGGRAPH - http://people.csail.mit.edu/mrub/papers/vidmag.pdf http://people.csail.mit.edu/mrub/papers/vidmag.pdf
- Osmium 14y agoIs there an open source implementation of this yet?
- floobynewb 14y agoI was thinking about doing this. It's very do-able, there's nothing really complicated going on here. I wanted to do it for mobile devices, but this is going to be pretty resource heavy and I couldn't work out how to do the necessary processing in OpenGL ES...
- cyrus_ 14y agoThis is the code referenced in the video: http://people.csail.mit.edu/mrub/vidmag/ http://people.csail.mit.edu/mrub/vidmag/
- jbrooksuk 14y agoI imagine that the only real problem is it'll never be able to be 100% real time as you need to sources of colour. I guess you could get pretty close though.
- ndr 14y agoIt is intrinsically delayed because it's based on the deltas from the previous state, and also they show a feedback loop in the filtering phase. In one of the example of the color amplification you can see the head moving and a little bit of lag between the two sides, but I can't tell if they have been intentionally synced or this was done truly in real time.
- timthorn 14y agoPhilips launched an iPad app to do this over a year ago: http://innovation.philips.com/pressreleases/nurturing-bg_vital_signs_camera/index.html http://innovation.philips.com/pressreleases/nurturing-bg_vit...
- shurcooL 14y agoHere's the app in question. http://appshopper.com/healthcare-fitness/vital-signs-camera-philips http://appshopper.com/healthcare-fitness/vital-signs-camera-... It's $1 and seems to have mixed reviews. I haven't tried it myself.
- Geee 14y agoThere's an app for that: http://cardiio.com/ http://cardiio.com/ (by HNer, http://news.ycombinator.com/item?id=4062555 http://news.ycombinator.com/item?id=4062555)
- rubberbandage 14y agoI’ve been using it for a few months and it’s surprisingly accurate. It’s fascinating to see a video demonstration of what Cardio does with all its math and image processing, since the only feedback in the app is the silly phrases above the stethoscope. I’d love a switch to show the processed video instead of the heart rate!
- mattvot 14y agoSomeone should take the code and make a site/app that can take YouTube videos in and enhance. I would try, but video processing goes way over my head.
- tommy_m 14y agohttp://videoscope.qrclab.com http://videoscope.qrclab.com
- konstruktors 14y agoSomeone should port it over to HTML using canvas, native video and lots of javascript.
- bluetidepro 14y agoThey mention that they posted the code last year, in August. Does anyone have a link to that code? This is so incredible to me!
- gallamine 14y agoMATLAB implementation here: http://people.csail.mit.edu/mrub/vidmag/ http://people.csail.mit.edu/mrub/vidmag/
- reginaldo 14y agoLast June, when this first came up, I commented: I was thinking about the implications of using this technique to analyze e.g. political speeches and try to catch people lying on the act. Your application (winning on card games) seems very interesting too Now, with the Google glass getting closer to being a real thing in the market, the possibilities are endless (for the good and for the bad). Unfortunately, my mind is kind of twisted and I think about the bad first. Must be a side effect of all the security issues I'm researching. For instance: # Google glass + Eulerian magnification + facial expression recognition = Instant "Lie to Me"-like[1] microexpressions expert. # Google glass + Eulerian magnification + TSA agent: "picking" suspects by the way their pulse react as they get closer to the agent using the "apparatus". Of course, the real criminals would just take some kind of drug to avoid being detected... http://en.wikipedia.org/wiki/Lie_to_Me http://en.wikipedia.org/wiki/Lie_to_Me
- Someone 14y agoOr have a camera with software like this and a low-intensity video projector aimed at their face. The projector would beam the anti-phase of the signal that the camera detects. Better yet: project a signal on the guy next to you in line that makes him a prime 'I am a terrorist' suspect. Another bad: when walking on the street, if your heart rate goes up when seeing a mean looking individual approach you, that individual could pick you out for a 'give me your wallet' treatment. I think that would decrease the risk of getting caught for the criminal.
- bostonpete 14y ago> could pick you out for a 'give me your wallet' treatment Well, this is small-time stuff and I think it's unlikely that such thieves would start employing technology like this. The payoff for this type of robbery is just too small to draw in a lot of innovation. Plus, I'm not sure heart rate is such a useful data point in deciding who to rob. My heart rate could go up because I'm scared, or because I'm amping myself up for a fight, or because I'm getting ready to draw my weapon in self-defense...
- 14y ago
- addandsubtract 14y agoCan someone explain how they are amplifying the video of the eye moving and the baby in the crib breathing? If they are only analyzing the changes in color, how are they amplifying movement of shapes with that information?
- gallamine 14y agoThey are amplifying a vector of choice. That vector could be change in color or change in relative pixel positions (as in physical motion). In other words, it's not color based.
- peripetylabs 14y agoComputer science that can improve people's wellbeing -- in this case, medical diagnostics -- is particularly amazing. I can see this being used by emergency workers to take pulse quickly without having to fumble with electrical leads. The algorithm is also very simple and elegant.
- hatcravat 14y agoI too think this is a very exciting sub-field. EMTs and the like already have something like this, only better: It's called a pulse oximeter, and it determines both heart rate and blood oxygen saturation by measuring the absorption ratio of two wavelengths of light. Commercial devices clip on your finger and cost about $50 (the whole device is about the size of two fingers).
- eru 14y agoI've seen those. But having even a mediocre version of the same available as software for each smartphone will only improve what the casual user or even first-aider is capable of. I wonder whether it would make sense to give people first aid instructions via an app? I guess when actually panicking, people would work better talking to an experience person over the phone. At least for the time being.
- hartator 14y agoWaho, I think this project is fascinating. Anyone wants to work with me something like 2/3 days to release a web app, that transpose their code in a more "webby" way? (We have their own solution: http://videoscope.qrclab.com/ http://videoscope.qrclab.com/ but with all the respect this guys deserve! It's unfortunately far from perfect.) I am based in Austin, my email is hartator_AT_gmail.com
- 31reasons 14y agoHow are they able to change the video frame to show the motion amplification. In the last example, they are amplifying baby's breathing and showing it using some changes into the video frame itself. How are they able to stretch baby's cloths and body without knowing its 3D geometry, even wrinkles on the baby's cloths seems to shift. I am puzzled.
- etherextraction 14y agoSame questions myself
- TheZenPsycho 14y agoHuman perception of 3D geometry comes from motion. This is an algorithm to amplify motion. And that's really all that is needed. See "Structure From Motion" and "Optical Flow" for more information about this. Note that this is technology that is used in some video codecs. You may have noticed in some corrupted videos, the corruption artefacts can sometimes move as if they are mapped on a detailed 3d model of the moving objects. But really all it is are 2D motion vectors moving blocks of color around on a predetermined optical flow path. In this algorithm, exaggerating that same 2D optical flow path has the perceptual effect of somehow "knowing" something about the 3d geometry. you could more accurately say the original motion is constrained by the 3D geometry. (more) here is a kanye west video that uses the optical flow/video corruption effect intentionally for artistic purposes. http://vimeo.com/4578366 http://vimeo.com/4578366 see particularly the sequence starting at 1:00 to see unrelated background "texture" getting motion mapped onto kanye's moving face.
- 31reasons 14y agoThanks for the detailed explanation. The kanye video just looked like a corrupted low res video, I didn't understand that one. But i get the idea, its about extrapolation of moving pixels in 2D.
- antirez 14y agoI bet the iPhone app will be great. A few months ago I had some weekend fun implementing this algorithm in a trow-away iPhone application, and indeed I was able to observe the color change in my skin, and when watching to the veins of my arm the movement was greatly amplified. I was missing a lot of the filtering required in order to amplify more and performance was a bit poor (frame rate) since it was just an hack. Something like that done in the proper way will be cool.
- mistercow 14y agoWhat I find neatest about this (if I recall the paper correctly), is that it's essentially the same concept as an unsharp mask filter, only taken over time instead of over space.
- hmsimha 14y agoMy first thought when watching the segment where a clip from Batman is shown is, if this can be applied to movies, it may ruin some of the magic when the video picks up the microscopic motions of supposedly 'dead' characters (as the actors are still breathing and pulsing). I think this is groundbreaking technology though -- I've read that there are subconscious responses to seeing things we like, such as a delicious food or an attractive individual of the preferred gender; a widening of the pupil or an increase in heart rate and body temperature. Devices that capture these changes can have applications in everything from marketing to security to courting.
- bsaul 14y ago"My first thought when watching the segment where a clip from Batman is shown is, if this can be applied to movies, it may ruin some of the magic when the video picks up the microscopic motions of supposedly 'dead' characters (as the actors are still breathing and pulsing)." Because you'd need fancy algorithm to believe that actors in movies aren't really dead ?
- hmsimha 14y agoI was pointing out that as cool as this is, you wouldn't realistically want to watch a movie you hadn't seen before with this kind of augmentation. Part of accepting a fictional story is suspending disbelief.. movies are designed to make this as easy for the viewer as possible. That being said, seeing a dead character breathing would disrupt that suspension of disbelief. It was the first thought that crossed my mind, but I found it interesting enough to share.
- pla3rhat3r 14y agoI find this fascinating especially that the code is open source. Makes me wonder what kind of applications can be built. Maybe even using some sort of Leap Motion application in conjunction with the amplification algorithm.
- 31reasons 14y agoCan blood pressure be monitored in realtime with this technique ?
- eru 14y agoBlood pressure would be pretty hard, I believe. But pulse is easy.
- dredmorbius 14y agoPulse, yes, as is demonstrated in the video. Blood pressure, in a calibrated sense, is a lot harder to measure without directly tapping into a vessel or putting a cuff on a patient, both of which are invasive and/or annoying over a long period of time or for chronic / long-term patients. But a few derivative measures of circulation should be determinable. Both skin color and temperature will relate to blood oxygen levels and circulatory efficacy. It should be possible to note when a patient's circulation is no longer effective, a low blood-pressure situation or heartbeat irregularity.