14 ms·
Through-Wall Human Pose Estimation Using Radio Signals
- enervate 8y agoI guess now we know how that beginning scene in altered carbon could work
- qwantus 8y agoMore suicides to come. Humans need to feel in control of their own lives.
- aeleos 8y agoIt’s pretty amazing that they only trained it to create pose estimation from radio signals, and that its ability to see through walls is only a byproduct of them not blocking RF. Given how much information is out there that we can’t sense I wonder if this area will be a main source of AI advancements. Another example of something like this is using videos of plants that vibrate slightly with sound waves to recreate the original sound.
- thirduncle 8y agoCountermeasures?
- aeleos 8y agoI bet some RF absorbing material or something that reflects RF at weird angles would probably break it. Or an active RF transmitter.
- kawfey 8y agoanything metal in the way. I'm sure the image would be a lot less clear if the wall had metal studs instead of wood.
- sudhirj 8y agoMake all walls a Faraday cage / concrete. If you can’t get cell or radio reception inside, you probably aren’t leaking anything either.
- deleted 8y ago[deleted]
- John_KZ 8y agoBuild an electronic warfare device that makes it look like hundreds of people are inside, all pointing guns outwards.
- femto 8y agoCarry something metallic, so the neural network no longer classifies your reflection as human?
- sudhirj 8y agoLove it. This is a game mechanic / movie superhero device that's been around a long time, just love the fact that this does it so simply and with so much promise. As sensitivity and the quality of the neural net increases, combined with AR, this is going to make super soldiers out of SWAT teams.
- alexbeloi 8y agoI hope you're not doing anything illegal, citizen.
- gknoy 8y agoMaybe SWAT can use this to validate that it's actually the right house when serving a no-knock warrant. I'm cynical enough to feel like there's no way this is good for any of their targets, though. Edit: Kolpa makes a really good point! This seems like a really good counter to SWATting, where the claim is that someone is holding hostages or something -- this could pretty effectively demonstrate that to be false if they see one person in an apartment sitting at a computer.
- sudhirj 8y agoNo possible improvement to a SWAT team is ever good for a target. But I think this will prevent false positives a bit. Knowing the posture and stance of the people inside a room can help with where you aim as you go in.
- noonespecial 8y agoI'd be more afraid that they'd "fear for their lives" at every broomstick and fishing pole they saw through a wall and would start shooting before they even got to the door.
- John_KZ 8y agoThis. It's quite possible that if it looks like you're holding something, they'll just shoot through the wall before even entering.
- 8y ago
- visarga 8y agoWith this paper our last shreds of privacy are being taken away, but they presented the news with nice background music, so there's that.
- chaosmachine 8y ago"Attacks only get better".
- bufbupa 8y agoMeh, that seems a little sensationalist. Any significant metal would likely throw this in a loop. You could even buy paint with graphite in it to block weak RF signals outright. IIRC most military bases and data centers protect against this sort of leakage.
- jwilk 8y agoThe skeletons appear to have very long legs, but no torsos. I wonder if it was made intentionally that way to make them look less serious.
- randcraw 8y agoI also wonder how accurate their estimates are of arm and leg poses, much less their motions. I do believe they can detect the location of a torso. I don't believe they can do the same for forearms, hands, legs, or feet, or their precise motions. I suspect the articulated pose and motion of their animated stick figures were extrapolated from progressions of articulations of limb poses that were learned from the light-based training videos, and then they were overlaid onto the wifi-based signal of only the moving trunk. All that animation does look cool, though.
- ggg9990 8y agoLook at the list of names on this paper. We better fix our immigration system for high skill individuals.
- jjoonathan 8y agoDevil's advocate: "let's fund PhDs with respectable salaries instead of green cards."
- hello_1234 8y agoWe are talking about MIT. Their acceptance rate is around 6%. I am sure they get plenty of applications from the smartest American undergrads. Assuming that they choose the best and the brightest, it makes sense that they would have students from all over the world. Americans don't have a monopoly on intelligence.
- throwawayjava 8y agoYou are both correct. Many Americans choose not to go into academia (at every level) because the other options are much more lucrative. It is true that many of America's best minds do not pursue a PhD in large part because of the paltry stipend. It is also absolutely true that the one top CS PhD program I know of does not compromise in admissions and does not suffer from a lack of highly qualified applicants. > Americans don't have a monopoly on intelligence. I think it's possible to agree with this point while still believing that American CS Ph.D. programs -- especially those outside the top N -- are not as attractive to domestic applicants as they probably should be.
- RIMR 8y ago>Americans don't have a monopoly on intelligence. Sadly, we seem to be quite deficient in that department compared to other first world countries...
- kolpa 8y agoHow does that solve the problem of attracting talent?
- sebringj 8y agoThrough-Wall Human HeadShot Estimation Using Radio Signals...
- ttul 8y agoAmazing what a bit of military funding can achieve.
- bitL 8y agoWe need multiple home robots to fake our presence now!
- gaius 8y agoThere are no legitimate uses of this technique. It only applies if you lack the consent of the target. If have that, there are many easier and better ways to monitor them.
- oh_sigh 8y agoSWAT team dealing with a hostage situation in a closed room
- murdockq 8y agoA personal home security system could make better use of this instead of motion sensors in each room.
- godelmachine 8y agoIt can also be used to monitor a baby's respiration
- grkvlt 8y agoAs mentioned elsewhere, detecting earthquake victims maybe, a terrorist or hostage situation where the targets definitely aren't giving consent, military situations and covert surveillance for law enforcement and counter espionage all spring to mind as legitimate and useful applications.
- zlynx 8y agoJust like people running open WiFi access points getting upset about anyone reading their packets. They don't think about what they can't see. A building that doesn't block radio and IR (and others) is essentially transparent. Putting an emitter like a WiFi point inside is like turning on a light. What we need is more attention put to building better walls.
- mathnmusic 8y agoDoes it matter if our WiFi points are always on?
- matte_black 8y agoPresence detection can easily toggle WiFi on and off based on if people are present.
- mt3ck 8y agomaybe so... but only external walls right and even then dont you want wifi in the back yard?
- gohwell 8y agoDoes it only detect moving bodies?
- sudhirj 8y agoDepends on the training data and the room config. The system sees a difference in an empty vs occupied room, movement doesn’t seem to be strictly necessary. The heat map video looks more like radio spectrum signatures, which might even be working on still frames - which would imply that movement isn’t necessary at all. It would help the AI, though. Easier to classify moving bodies, the speed and gait is a signal.
- lolc 8y agoWow great work! I'm stunned. Not really by the pose-recognition, that's just a bonus.
- miguelrochefort 8y agoI'll repeat it again. - Machines that see through walls are coming - Microscopic drones are coming - Mind-reading machines are coming - Private key cryptography will become obsolete Fighting for privacy is backward. It's unsustainable. We must work toward a post-privacy world. The sooner we do it, the smoother the transition will be.
- musage 8y agoSmelly air is "coming", so prepare for it by eating lots of onions? Nah. > I've heard quite a lot of people that talk about post-privacy, and they talk about it in terms of feeling like, you know, it's too late, we're done for, there's just no possibility for privacy left anymore and we just have to get used to it. And this is a pretty fascinating thing, because it seems to me that you never hear a feminist say that we're post-consent because there is rape. And why is that? The reason is that it's bullshit. > We can't have a post-privacy world until we're post-privilege. So when we cave in our autonomy, then we can sort of say, "well, okay, we don't need privacy anymore, in fact we don't have privacy anymore, and I'm okay with that." Realistically though people are not comfortable with that. Because, if you only look at it from a position of privilege, like, say, white man on a stage, then yeah, maybe post-privacy works out okay for those people. But if you have ever not been, or if you are currently not, a white man with a passport from one of the five good nations in the world, it might not really work out well for you, and in fact it might be designed specifically such that it will continue to not work out well for you, because the structures themselves produce these inequalities. > So when you hear someone talk about post-privacy, I think it's really important to engage them about their own privilege in the system and what it is they are actually arguing for. -- Jacob Appelbaum
- Sammi 8y agoI'm scared. What is a post-privacy world?
- miguelrochefort 8y agoA world where you don't have to lie and keep secrets. A world where people freely share knowledge and resources.
- himom 8y agoAlmost like The Eraser (1996), just add squad railgun rifles and X-ray-like targeting.
- hathawsh 8y agoAm I correct in assuming they are using a Walabot sensor as mentioned in footnote 2? https://walabot.com/ https://walabot.com/ EDIT: I see from section 3 that they created their own FMCW antenna array that is somehow similar to a Walabot.
- godelmachine 8y agoMay I ask what's FMCW? Of course I can straightaway google but I would like to hear an intuitive explaination.
- hathawsh 8y agoI had to Google it too. :-) http://demonstrations.wolfram.com/FrequencyModulatedContinuousWaveFMCWRadar/ http://demonstrations.wolfram.com/FrequencyModulatedContinuo...
- allthenews 8y agoGreat, as if I weren't already self-conscious about having my roommates overhear me, now they'll be able watch me masturbate through the walls too.
- hammock 8y agoWi-fi has been researched for this capability since at least 2015: https://www.youtube.com/watch?v=fGZzNZnYIHo https://www.youtube.com/watch?v=fGZzNZnYIHo
- freyir 8y agoIt’s a new paper from the same group. In 2015, the tracker person was just shown as an amorphous blob on the screen. In the recent paper, they display a stick figure modelling the movements of the person’s limbs.
- wcunning 8y agoI've noticed, recently, that literally every major chain store I go into blocks my cell signal. I get full bars outside, perfect LTE, and about 5 ft past the glass doors, I'm suddenly cut off. Specifically, Meijer grocery stores and Home Depot are the worst offenders. Coincidentally they offer free in store WiFi to track me with, so I turn off my wifi radio on both smartphone and smartwatch before walking in the door... Note, this was not the case in the exact same physical stores a couple years ago, and they weren't closed for remodeling in the interim. Does anyone know if this sort of Faraday cage technology for large buildings is easily available for me to install in my own home? If so, what's it called, what does it cost and where do I get it?
- lgats 8y agoI believe you can achieve the same result in your home by adding a sheet metal roof, metal cabinets, and refrigerators along every wall.
- cabaalis 8y agoIt's a rather unfortunate coincidence, isn't it? Certainly it's not their fault that their buildings just happen to be constructed in such a way, and their products arranged such that it blocks signal causing customers to not be able to compare prices and have to use their approved internet connection. /s I see this in the Targets in my town. I just expect signal to drop to zero.
- bufbupa 8y agoFinally, a legit purpose for my tinfoil hat!
- DoreenMichele 8y agoDuh. You clearly didn't read the manual.
- amenod 8y agoI know you're joking, but that's an interesting idea... I wonder how would the system "see" your head then? Unless it is trained on people wearing tinfoil hats (and aluminium suits) it might produce interesting results.
- thedirt0115 8y agoDo the people that write these papers ever release their source code or trained model? I'm curious about both other researchers reproducing as well as being able to download something and play with it.
- freyir 8y agoThey’re using custom hardware to make this work. This isn’t always clear from the press release. For example, they mention that the technology works at “WiFi frequencies.” That’s true, but they’re sending a radar signal using two huge antennas, not a Wi-Fi router.
- RyanCavanaugh 8y agoIn a 5-4 decision in https://en.wikipedia.org/wiki/Kyllo_v._United_States https://en.wikipedia.org/wiki/Kyllo_v._United_States , the SCOTUS determined that police can't use thermal imaging on a house without a search warrant. I wonder what the vote would have been if the justices could have seen videos like this.
- kolpa 8y agoThis seems nearly identical issue; why would you suspect the vote would differ?
- themodelplumber 8y agoSo this MIT method would be OK, since it's not thermal?
- fermienrico 8y agoIt is still radiation of photons. Just different wavelengths (IR vs Radio).
- viggity 8y agoFWIW, I don't think the cops were using thermal imaging to detect humans within a house - they were using it to detect abnormally high heat - from high temperature lamps used to grow pot. Originally they'd talk to the power company to see who is drawing irregularly high levels of power, but the growers started using diesel power generators to power their grow lights.
- JTbane 8y agoSometimes I wonder is this could be mitigated by literally covering your walls in aluminum foil. Maybe those "tinfoil hats" are on to something.
- binarymax 8y agoJust get aluminum siding, which was very popular before people switched to vinyl.
- poundtown 8y agohuh huh huh he said smaller micro movements...
- godelmachine 8y agoI would like to get more research of this sort. Would someone kindly point me towards it? Thanks.
- adw 8y agoA good place to start would be the paper's references.
- godelmachine 8y agoI went to the website and casually looked down. There was a "see also" section. Browsed through those and remembered Prof Dina Katabi demonstrating this concept before Prez Obama in the White House, few years ago. Will go through references too :)
- adw 8y agoI come at this entirely from the ML side, not the hardware one, so that stuff is a mystery to me :-)
- Jach 8y agoI built and extended this for a college project years ago: https://ocw.mit.edu/resources/res-ll-003-build-a-small-radar-system-capable-of-sensing-range-doppler-and-synthetic-aperture-radar-imaging-january-iap-2011/index.htm https://ocw.mit.edu/resources/res-ll-003-build-a-small-radar... It's not a bad place to start. And cheap if you want to build your own too (some third party even sells the "cantennas" so you don't have to fiddle with tuning that part (which is kind of infeasible without a network analyzer anyway)).
- godelmachine 8y agoThanks Jach
- packeted 8y agoAwesome. I was playing Batman Arkham City yesterday and this is "Detective Mode" in the real life!
- TremendousJudge 8y agoYeah, I wonder what they are going to use this technology for. "Helping earthquake victims" probably
- con22 8y agowe have nothing to hide now!
- barbegal 8y agoWhat this video (or even the paper) doesn't show is the large antenna arrays that are used to capture the heatmaps. A previous paper called "Capturing the human figure through a wall" [1] shows the type of antenna array used and in this paper they use two such antenna arrays to capture two planes to increase the accuracy. [1] https://www.semanticscholar.org/paper/Capturing-the-human-figure-through-a-wall-Adib-Hsu/2b2d03f8b96aa1e306fb941e0318d403efbde4be https://www.semanticscholar.org/paper/Capturing-the-human-fi...
- dznodes 8y agoMIT's new anti-fapping radar!
- concernedctzn 8y agoI wonder how similar it is to the tech used in this paper on gesture recognition through walls: http://wisee.cs.washington.edu http://wisee.cs.washington.edu [2013]
- barbegal 8y agoThey are quite different. Wisee uses a single reciever and uses doppler shift to identify the gestures whereas this "through wall" system uses an array of transmitters and receivers that identify reflections from a position in space. In layman's terms this system is more like a conventional camera but working in the radio spectrum whereas Wisee is like a microphone that can pick up changes in frequencies around it.
- maxander 8y agoWe could be using this technology to see tumors and other pathologies through patients’ skin. Instead we’re using it to give soldiers and spies x-ray vision.
- mabbo 8y agoWhy can't it be both? Science and technology advance in often unexpected directions and for new purposes all the time. Satellites in orbit came from ICBM research, GPS from military location tracking, the tiny electronics in your phone are the grandchildren of at least one miniature spy device.
- gus_massa 8y agoNo we can't use this radio signals to see tumors, because science is not magic and wishful thinking will not solve the technical problems. Let's try to name a few: The radio signals are absorbed by wet salty meatbags, so you can find them using radio signals, but you can't see thru them. If you raise the power to compensate the absorption, you will cook the meatbags, like a microwave oven. If you were lucky to get a signal at the other end, the radio signal can distinguish between meat-bone-air-water, but not between tissues like meat and tumors that have a somewhat similar composition. A tomography use contrast to try to distinguish them. A magnetic resonance is better, but it use a huge magnet (and in some cases it use contrast too). And also, the radio frequency that they are using has a wavelength of an inch approximately. So roughly it can resolve things that are an inch long, so it must be a big tumor to be visible. Perhaps you can try with a higher frequency that has a smaller wavelength. And if I understand correctly, this device needs a lot of calibration, so you need version of the part of the body without the tumor and a version with tumors of different sizes and positions to calibrate the device and then try to use it in the real subject.
- jamesough 8y agoUse ultra-wideband radio pulses, an array of a few thousand 3D printed UWB antennas and a phased-array scanning beam. You'll get a machine you could fit in the back of a truck that can scan a city block in real-time to mm precision. Sweet dreams, citizen.
- antisthenes 8y agoSo Dark Knight style imagery, all from 1 truck? What's the range?
- jamesough 8y agoDepends on UWB bandwidth+power, how good your processing is + how many antennas you have... You could also get way better with a fleet of trucks spaced over a large area, or a city-sized swarm of drones, or satellites...
- femto 8y agoPotentially not that simple, as there are fundamental limits on the information capacity of a MIMO system, based on the surface area of the volume occupied by the antenna array (in wavelengths squared). The capacity limits apply whether you use the array for communications or radar, as both are governed by Information Theory. Not to say that it can't be done, but the maths exists to definitively prove whether it can be, so no need for speculation and tinfoil. My gut feeling is that you could scan a small volume from afar, but not a whole block. No, I haven't done the maths.
- jamesough 8y agoYeah this is why you'd use UWB -- the capacity is insane. Also the raw information theory calculation makes sense only if you have zero prior information about the sample you're reconstructing. But 'people in buildings' is a pretty strong prior. And it's not like you have to do any extra work, just put a simulated raw signal through a neural net and regress to groundtruth data. But let's do the math, I'm curious!
- rl3 8y agoThe defense applications for this are tantalizing. However, given the staggering amount of room clearing that's been taking place in US military operations for the past 15 or so years, I'd be shocked if there wasn't already an existing capability to detect bad guys through walls in a CQB environment.
- dsfyu404ed 8y ago>I'd be shocked if there wasn't already an existing capability to detect bad guys through walls in a CQB environment. Existing and exist in a state you can deploy are two totally different things that often happen many years apart. Usually something exists as an impractical wish list item for a long time before some other development pushes it into the realm of being practical to use.
- rl3 8y agoYeah, it's just that western special operations kick in so many doors on a daily basis that one would think such tech would be fast-tracked.
- godelmachine 8y agoMay I ask what's CQB?
- rl3 8y agoClose Quarters Battle
- Animats 8y agoThe paper [1] doesn't show the antennas, but the text seems to indicate the antennas are quite large, as large as the area being observed. Plus, you have to have two perpendicular axes of antenna, so you have to be behind two walls, or one wall and a floor or ceiling. That's a big setup. It might be useful if you wanted to count people in a crowd going in and out through a big opening, like a subway station or mall. [1] http://openaccess.thecvf.com/content_cvpr_2018/papers/Zhao_Through-Wall_Human_Pose_CVPR_2018_paper.pdf http://openaccess.thecvf.com/content_cvpr_2018/papers/Zhao_T...
- workinthehead 8y agoGreat. More murder tech. There's always funding for murder tech.