13 ms·
mmWave radar, you won't see it coming
- hornej 5y agoI've been following and building mmWave radar stuff for a little over a year now and I wanted to write an intro to help anyone get up to speed on how it works, why it's special, and what's been happening in the space.
- synthos 5y agoHow is the power consumption? Is it sensitive to noise or interference? The VR headset application appeals to me. If headsets could be dumb terminals they would be a lot lighter and untethered wouldn't suffer from performance loss.
- hornej 5y agoPower consumption for radar can get down to around 0.5mW (edit) But I imagine for 60GHz WiFi it's probably pretty power hungry. At this point there's not much noise in the mmWave spectrum (it won't be competing with 2.4GHz and 5GHz anyways). And mmWave doesn't travel that far which is one of the main reasons the FCC unlicensed it in the first place. Might be worth keeping an eye on UWB for AR/VR headsets https://www.embedded.com/wireless-transceivers-use-uwb-for-low-power-low-latency-data-transfer/ https://www.embedded.com/wireless-transceivers-use-uwb-for-l...
- BenjiWiebe 5y agoChecking some point to point wireless radios and it doesn't look like 60Ghz needs much more power. Well, it might be a few times as much, but definitely the same order of magnitude.
- mateo1 5y agoPlease write up how to make a simple passive detector so people can have a chance to know if they are being tracked. This is going to be in every home in 10 years embedded in phones, TVs, smart speakers. All public venues. Trams, buses, subways. Everywhere. But most importantly homes. Every. Single. Person. 24/7. Breathing rate. Abnormal motions. Current location. Current activity. Maybe, just maybe if we raise a some awareness we can curb the proliferation a bit. Surely I can't be the only one who sees this as a nightmare.
- hooande 5y agogreat article, I really appreciate it! sorry that the knuckleheads here got caught up on privacy instead of discussing this interesting technology
- will_walker 5y agoChallenges to keep devices affordable aside ($3.65 is a lot for a BOM add on), Radar is one of the first non-human senses I’ve had to design for. From an ML perspective it offers significant advantages over static images, being independent from ambient lighting, and sample rates are higher than low cost cameras without ISO noise.
- idontwantthis 5y ago> Because mmWave radar doesn’t process visual light, what it sees is not personally identifiable. This makes it an ideal choice for monitoring where a camera would not be appropriate. Is 5mm resolution not enough to identify a face? What’s the difference between a 3d model of a person constructed from high res radar and a black and white photo? The radar sounds even more identifiable. Edit: Couldn’t you construct a mosaic and have much higher resolution? And also remove their clothes?
- krasin 5y agoI worked with mmWave radars previously, but not professionally. In my opinion, it would be quite challenging to extract enough features from the current generation of mmWave radars, such as TI IWR1642 ([1]) which have around 4 receivers and 2-3 transmitters, because the incoming data, while having a lot of temporal resolution is very limited in spatial resolution. With greater number of antennas something like what you describe is theoretically possible, but becomes cost prohibitive. By limiting the number of transmitters / receivers, we can have an almost perfect privacy monitoring. For example, it would be possible to detect a attack / a fight in a public bathroom, while not exposing anyone. 1. https://www.mouser.ec/datasheet/2/405/1/swru521c-1954464.pdf https://www.mouser.ec/datasheet/2/405/1/swru521c-1954464.pdf
- R0b0t1 5y agoTemporal res can be turned into spatial res.
- krasin 5y agoTheoretically: yes. Practically, there are limits to that.
- jcims 5y agoTo an extent, but depending on the sampling rate and frequency and your ability to control the area being observed, there is still a lot of information available for modeling and biometric identification. In the extreme you can detect things like heartbeat, rate of breathing, etc. For example, I have a CDM324 24Ghz radar module here on my desk. I set it up to 'watch' me type this comment from across the keyboard. This is an extremely simple module that I have powered by a bench power supply and the IF routed directly to the audio input on my desktop. It was sampled with audacity and amplified a bit to help with visibility. Towards the end of the recording I'm expecting a flat spot followed by regular motion followed by 'noise' as I pause motionless for ~10 seconds or so, then take about 5-6 exaggerated breaths, then resume typing. This is with zero design, the wrong frequency for the job, and next to zero signal conditioning. (post: i've included a zoomed in image of the 'motion demonstration' to show still/breathing/typing, then zoomed in on typing to show the detail, then a spectrogram and waveform of me reaching up to scratch my ear.) https://imgur.com/a/0JmENYu https://imgur.com/a/0JmENYu Bonus: yo check out my soundcloud - This is what the doppler signal actually sounds like: Scratching my ear - https://soundcloud.com/buckrunner2/scratch https://soundcloud.com/buckrunner2/scratch Talking directly at the module - https://soundcloud.com/buckrunner2/talking https://soundcloud.com/buckrunner2/talking
- senectus1 5y agoman that mmwave stuff built into phones and watches looks great. Would love to see that on my watch and Galaxy Fold3... less touching but gesture control. Want.
- ipsin 5y agoSo far these applications seem pretty innocuous, but for things like gesture & other control systems, can mmWave be jammed or spoofed?
- hornej 5y agothere is a risk of being jammed or spoofed which is why there has been a bunch of research done in the automotive industry. https://www.caee.utexas.edu/prof/bhat/ABSTRACTS/SecurityOverview_mmWave_V2X.pdf https://www.caee.utexas.edu/prof/bhat/ABSTRACTS/SecurityOver... But I don't think there is much risk in gesture control.
- mycall 5y agoHow hard is it to make stealth clothing, so your body doesn't reflect back to the radar system?
- jacknews 5y ago"One of mmWave's major advantages is its privacy-preserving nature." Ha, haha, hahaha! It can measure your breathing, heart signature, gait, etc. With machine learning this data will certainly be enough to identify you.
- chrismorgan 5y agoA technical error in one of the diagrams: Hall effect sensors are not binary, but one-dimensional, and are used so; their output voltage is directly proportional to the magnetic field strength. As for PIR sensors, fundamentally they’re certainly not binary, but I have no idea if they perhaps expose only a binary signal. A better example of a binary sensor might be an on/off switch.
- choonway 5y agothe only true binary sensor i can think of are good old fashioned mechanical switches.
- hornej 5y agothats a good point about the hall sensors. i'll update that. PIR sensors also output an analog signal, but in practice they are binary.
- deleted 5y ago[deleted]
- stingrae 5y agoin practice they don't have to be binary
- 5y ago
- jacknews 5y ago"The promise of ambient computing is that technology gets out of the way while becoming more helpful at accomplishing what we want. It anticipates our needs by understanding the context of our environment and situation. It serves us, rather than us serving it. " This is the optimistic case. Perhaps more likely is that 'ambient' computing will smother us in surveillance, ads, 'nudges', restrictions (you seem tired or drunk, you may not drive), etc.
- visarga 5y agoIt will probably be both good and bad, but for certain is that we're going to live inside a computer. Every move monitored.
- jasondclinton 5y agoI've been waiting for someone to finally make a consumer stud finder that works and this just might finally happen after this deregulation of the spectrum. The construction industry would hand over wheelbarrows full of money for a BLE phone-paired puck that located studs and pipes behind drywall. Someone could even sell a "pro" version that combined the data from multiple pucks to render a 3D point cloud of the inside of the wall on the user's phone.
- mycall 5y agoWould this be it? https://blog.vayyar.com/walabot-diy-2-launch https://blog.vayyar.com/walabot-diy-2-launch
- neurodyne 5y agoThey also seem to have development hardware available https://walabot.com/makers https://walabot.com/makers
- acmegeek 5y agoWould the Walabot fit the bill for you? I bought one when it first came out and seemed pretty good. I haven't messed with it in a couple years though. https://walabot.com/see-through-walls https://walabot.com/see-through-walls This seems like a pretty good review: https://www.bobvila.com/articles/walabot-stud-finder/ https://www.bobvila.com/articles/walabot-stud-finder/
- jpm_sd 5y agoI tried one and it's pretty disappointing. Plenty of room for improvement.
- hornej 5y agoThis stud finder uses UWB radar (lower frequency) https://www.amazon.com/Bosch-D-Tect-Floor-Scanner-Technology/dp/B005EM93R0/ref=sr_1_1?crid=2780PXCJLCI3H&keywords=bosch+dtect+150&qid=1643776912&sprefix=bosch+dtec%2Caps%2C184&sr=8-1 https://www.amazon.com/Bosch-D-Tect-Floor-Scanner-Technology...
- leohonexus 5y agoVery dumb question: If mmWave is operating at 60GHz, how would it be possible for processors (which are operating at less than 1/10th of that clock rate) to process that sampling rate?
- 01100011 5y agoYes. The relevant information may be contained in a narrow swath of bandwidth riding within that 60GHz. Say you use 60GHz as a carrier but only care about changes happening below 10kHz. You can separate out that from the returned signal and only process 20k samples/second.
- colechristensen 5y agoYes. The key here is the difference between frequency and bandwidth. The radar isn't using the entire 0 Hz to 60 GHz band, it will be using something like a few GHz or sometimes much less. i.e. 60-64 GHz Also much of the processing isn't done on general purpose processors but in FPGAs or ASICs which can do lots of the parallel computation very fast. For comparison 802.11ac has a max of 120 MHz channels, and that's cheap and everywhere these days (but runs on 5 GHz). There are lots of basically tricks to reduce the amount of data in successive stages so that the processing is quite possible.
- teraflop 5y agoThe same way you can receive/transmit a 5GHz wifi signal using devices running at a fraction of that clock rate: most of the critical high-frequency signal processing happens using analog electronics. The actual sample rate of the digitized signal is much lower.
- BenjiWiebe 5y agoThe other comments are good answers to your question, but here's a fun little fact: if you actually want to directly read a 60Ghz signal, you actually need to sample it at 120Ghz to get it all! Good old nyquist.
- iam-TJ 5y agoIn summary, 60 GHz is the CARRIER frequency (the central frequency) around which a modulated signal is centred or otherwise aligned. The modulated signal BANDWIDTH (channel width) is where the signal information is encoded and is going to be measured in Mega-Hertz (MHz). E.g. for 802.11a/n/c the CARRIER is in the 5.x GHz band but the modulated signal BANDWIDTH will be one of 20, 40, 80, 160 MHz around a central frequency [0]. The 'base' 20MHz bandwidth at best performance (signal to noise ratio) will be able to modulate (encode) at a raw 54Mbits/second. Intel publish [1] an easy-to-read set of tables that show the relationships between standards, frequencies, bandwidths, MIMO, and modulation schemes. [0] simplified - actually there is a (frequency-hopping or direct sequence) spread-spectrum method [1] https://www.intel.com/content/www/us/en/support/articles/000005725/wireless/legacy-intel-wireless-products.html https://www.intel.com/content/www/us/en/support/articles/000...
- imnotlost 5y agoPrivacy is a political/legal problem, not a technology problem.
- hornej 5y agotechnology is a political/legal problem. the printing press, gunpowder, the internet. technology has always been in the middle of political/legal problems. can't be evil > don't be evil.
- finite_jest 5y agoIt is a technological problem too. Telegram has a much better UI/UX than Signal, and not having end-to-end encryption by default is part of the reason.
- transpute 5y agoWire has a good UI/UX with always-on E2EE.
- argomo 5y agoA cryptopunk would say just the opposite: that privacy is a technical problem that, properly solved, alleviates us of political concerns. Both views seem to miss the mark. The crypto folks see mathematical certitude while ignoring the dynamics of the social/legal context in which it must operate. By contrast, those who argue that "artifacts don't have politics" would absolve technologists of an ethical duty to consider how their products impact society. In short, privacy required both technical and legal protections as well as a society that demands it as a fundamental human right
- finite_jest 5y agoRight, and a standard response to cryptopunk fundamentalism is pointing at rubber-hose cryptoanalysis [1]. [1]: https://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis https://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis
- transpute 5y ago> Because mmWave radar doesn’t process visual light, what it sees is not personally identifiable. With machine learning improvements, identification and more are possible. From a 2021 paper on IEEE 802.11bf Wi-Fi Sensing, https://arxiv.org/pdf/2103.14918.pdf https://arxiv.org/pdf/2103.14918.pdf > Indeed, it has been shown that SENS-based classifiers can infer privacy-critical information such as keyboard typing, gesture recognition and activity tracking ... since Wi- Fi signals can penetrate hard objects and can be used without the presence of light, end-users may not even realize they are being tracked .. individuals should be provided the opportunity to opt out of SENS services – to avoid being monitored and tracked by the Wi-Fi devices around them. This would require the widespread introduction of reliable SENS algorithm for human or animal identification. Prior discussion of WiFi Sensing: Jan 2022, https://news.ycombinator.com/item?id=29901587 https://news.ycombinator.com/item?id=29901587 May 2021, https://news.ycombinator.com/item?id=27123493 https://news.ycombinator.com/item?id=27123493 400+ research papers on wireless activity sensing, showing steady improvement in detection techniques, https://dhalperi.github.io/linux-80211n-csitool/#external https://dhalperi.github.io/linux-80211n-csitool/#external Today, specialist devices or targeted attacks can monitor human activity through walls and closed doors. But that's a world away from the ubiquitous transparency scheduled for commodity WiFi 7 in 2024. If regulators or lawyers don't step in, homes and some businesses may need RF shielding.
- waterproof 5y agoYeah. Up front they say: > what it sees is not personally identifiable Then later on in the article: > [the new HomePod could] [detect] the user's identity based on their gait, height, and physical profile Uh-huh. …did they read their own article?
- kwertyoowiyop 5y agoIt just depends on whether they’re talking to Congress or the Police.
- parsimo2010 5y agoRight on. The whole "mm wave radar is detail rich but gives privacy" line is a contradiction. Any data that is detailed enough to be useful is detailed enough to be a privacy concern. While it might not be the same as leaking nude photos from someone's camera, radar data is something that needs to be carefully thought out. Radar data from a car can provide information about people's driving behavior even without location data- do they follow too close, do they make reckless lane changes into tight spots, do they exceed the regular flowing traffic speed, etc. Radar data from inside a building used to track people can tell when you are at your desk or somewhere else, who you congregate with in the break room, how long you spend in the bathroom, etc. If it's detailed enough to assist you, that data can also be used to monitor you.
- dathinab 5y ago> mmWave radar doesn’t process visual light, what it sees is not personally identifiable that is not how "personally identifiable" works as long as you can (re-)identify a person with it it's personally identifiable data. Even if identification is done by using a mmWave radar and some simple ML.
- avidphantasm 5y agoYes, it seems much too early to say that this is privacy preserving just because visible wavelengths aren’t involved.
- Symmetry 5y agoUsing radars with only distance and relative speeds data, no direction at all, to identify equipment has a long history in the military. https://en.wikipedia.org/wiki/Radar_MASINT#Non-Cooperative_Target_Recognition https://en.wikipedia.org/wiki/Radar_MASINT#Non-Cooperative_T... Humans come in greater variety than airplanes but recognition, especially with things like gait analysis, should be quite possible. Now, I'm skeptical that a photo of a person contains enough information to allow for mmWave recognition though, but I'd assume video does.
- dathinab 5y agoInteresting, though it's enough that two independent mmWave radars can re-recognize a person to count as identifiable. No need to involve visual data. Especially if mmWave radars become common place.
- fooblaster 5y agoI've been somewhat disappointed that the learning material around mimo radar processing that can be done with popular TI automotive radar chips (and others) is somewhat limited after you get through with the absolute basics. TI's documents are a good starting point: https://training.ti.com/sites/default/files/docs/mmwaveSensing-FMCW-offlineviewing_0.pdf https://training.ti.com/sites/default/files/docs/mmwaveSensi... But it doesn't really go into the different schemes for sending out orthogonal waveforms from the transmitters, the design of 2d arrays, much of the basic signal processing (windowing, matched filtering), etc. I was pleased to see https://hforsten.com/mimo-radar-antenna-arrays.html https://hforsten.com/mimo-radar-antenna-arrays.html pop up on hacker news quite a while ago, which was very interesting. Goes into how to plot the beam width for a mimo array, with some nice python code. I want more, but I'm afraid I won't get it unless I read through a phd thesis. Any ideas/tips?
- lokimedes 5y agoDo any of you see avenues for a conventional radar company to contribute to this emerging trend? We produce high-end FMCW radars (http://dopplerradars.com http://dopplerradars.com) and I have been looking for ways to align us more with Tech and less with Defense.
- transpute 5y agoArms race: help evolve 802.11bf with deep understanding of risk management. Sell souped-up localization and CSI countermeasure devices, e.g. for industrial laboratories. Work with two celebrity neighbors on a sanctioned, comedic public demo of seeing through home and business walls, in advance of Wi-Fi 7 launch. Publicity will mean a larger market for both attack and defense products. If it's done early enough, it could materially influence the WiFi Sensing standard and thus devices racing to be the first to support not-yet-official WiFi 7. There are legitimate use cases, but consent and bounded scope are critical for mass-market acceptance. Non-naive experts can help. http://www.orca-project.eu/ http://www.orca-project.eu/ https://ans.unibs.it/projects/csi-murder/ https://ans.unibs.it/projects/csi-murder/ > Imagine that someone wants to illegally track the position of a person inside a laboratory, for instance to measure how much time is spent doing different activities at different desks, as depicted in the upper picture. How much effective can this attack be? ... With CSI-MURDER, the localization becomes impossible because results will seem random, thus preserving the person privacy without destroying Wi-Fi communications
- bradleyy 5y agoPresence detection. Make a $50 device I can run on a battery/Power over Ethernet/etc. Make it all locally manageable, and make it work with HomeAssistant, so I don't have to deal with it "calling home" or shipping data outside my LAN. I'm not a huge fan of the current crop of presence detection, and mmWave seems like just the thing. Eventually, person detection would be fantastic, but it MUST NOT call home outside the LAN. I don't want that junk going on the internet in any way, shape, or fashion.
- bradleyy 5y agoUse cases: I need a presence detector that's able to detect children near my pool. I need a presence detector that can, via gait detection (or other), detect when a specific person crosses a threshold (e.g. autism or Alzheimer's with a tendency to run off)
- lifeisstillgood 5y agoI have a theory / idea I call MOOP - massive open online psychology - the idea is that our phones / radar devices monitor our daily interactions - tone of voice, body gestures - to get a state of our anger / love / etc between our families (and work). And then uses simple epidemiology to help guide / train us. The basic idea is your phone becomes your life coach. This "privacy protecting" radar (which does not seem privacy protecting at all, just avoids leaking images which is just bad PR avoidance) looks like the same direction.
- bredren 5y agoNbd, but moop means garbage at the burn. I agree this will happen, I suspect Apple is not only in the best position to exploit mmWave hardware but already well down the road in related interaction design. Apple Watch already offers limited physical sense-based life coaching in the form of hand washing duration tracking and feedback. Apple Watch’s assisted interaction with UI that quietly debuted in iOS 15 was an important set of physical sense-based interaction primitives in production. They suggest to me the company has likely progressed capabilities sans mmWave.
- lifeisstillgood 5y agoI assume moop is a slang term then? what's the burn? And yes, I am sure many companies are thinking (or at least have people in them thinking like this). A few issues are the legal and cultural framework (I am quite happy for my personal data to be shared with accredited Health (ie NHS) researchers, but not with the Apple Watch product manager) Epidemiology at this level is going to have transformative effects, but it needs to be trusted. I mean the only difference between a utopia of guided humans and a surveillance state is who gets the data and what they do with it. the people in the free country abs the ones in the oppressive totalitarian state will be wearing the same devices with the same radar.
- bredren 5y agoAt Burning Man, moop is an acronym for Matter Out Of Place, a reference to the leave no trace principal of the event. People will run after a small piece of material. Again, NBD, but in tech circles this might be the first thing people think of with that acronym. I do think Apple captures plenty of meta data, but I do not know how much physical motion is getting sent back. One of the goals of the secure enclave was to store biometrics. And despite some notable concerns, the company has generally aligned its business model and branding to be privacy focused. I can't think of another large company I'd trust more to handle this kind of information. >Epidemiology at this level is going to have transformative effects... The transformation is already underway. When Apple delivered Google Maps on iPhone with its pulsing blue dot, those with the technology instantly leapt ahead of those without it. I wonder how much stress has been relieved just from people knowing where they are. I agree the technology can be misused, even to oppress. But this is true with any tech. My concern is more about the disparity of those who have this tech and those that don't. The enhancements could start to show in a matter of years and quality of recommendations could coalesce into what will look like a new plane of daily living.
- tomjonesey 5y ago
- HNSucksAss 5y ago
- telephone2 5y agothis is terrifying? even more surveillance, but passively, all the time, in spaces that are deemed too private for cameras.
- telephone2 5y agoSo even more surveillance, but passively, all the time, in spaces that are deemed too private for cameras.
- xorandor 5y ago> mmWave radar used inside the cabin as a sensor to detect a child left behind in a hot car Call me a cynic but since the power button stopped being a power button I have trust issues. In the end we'll have mmWave chips our company devices that can tell middle managers if we've been at our desk for all 28,800 seconds of the work day and exactly how hard we've been working, that's a more likely outcome than saving a kid in a hot car.
- oezi 5y agoYou could do that today with normal tech today. That's why you need laws in place that prohibit work place monitoring.
- musingsole 5y agoYou could argue that today's circumstance is why you DON'T need explicit laws prohibiting workplace monitoring. Anyone with the desire could set up such systems today. But they don't. Because they're expensive, inaccurate, and ultimately not all that useful in the social context. There are places that do employ such monitoring, but they're a corner case. Trust is a core commodity in the workplace. It's usually more profitable to establish a trustworthy workgroup to accomplish some goal than it is to accomplish some goal WHILE monitoring the workgroup's every move for infractions.
- mikem170 5y ago> There are places that do employ such monitoring, but they're a corner case. Don't amazon warehouses already monitor workers in this much detail? Just yesterday I read someone talking about how managers get reports containing, amongst other things, how often a person was standing still. Lots of companies squeeze their employees like this, especially low paid employees.
- telephone2 5y agoAmazon has contracted firms to produce boxes for their facilities which include 'gait recognition' code.
- fagnerbrack 5y ago> "Consumer Technology Association (CTA) announced Ripple, an open API for developing general-purpose consumer radar systems" Why using the name of a cryptocurrency for something that has nothing to do with it??
- deleted 5y ago[deleted]
- pmx 5y agoRipple crypto doesn't own that name in any domain other than crypto so it can be used in other applications without an issue. The name makes a great deal of sense in this context since mmWave uses tiny waves; otherwise known as ripples.
- jldugger 5y ago> In October 2019, Soli shipped in the Pixel 4, marketed to consumers as "Motion Sense." It provided users with a faster face unlock along with some basic touchless gestures like controlling music. Google must have decided it was still too early as they haven’t included Soli in any of the Pixel phones since. Wasn't the blocker regulations in India?
- hornej 5y agoquite possibly. i didn't dive down that rabbit hole so just said 'too early' ¯\_(ツ)_/¯ would be interested if you have any links about it
- harvie 5y agohttps://github.com/jdesbonnet/RCWL-0516/ https://github.com/jdesbonnet/RCWL-0516/
- nynyny7 5y agoHm, the article makes it sound as if mmWave radars are a fairly new addition to cars. But 77 GHz radar sensors have been in cars since over a decade now, thanks to suppliers such as Bosch and Continental.
- giantg2 5y ago"One of mmWave's major advantages is its privacy-preserving nature. Because mmWave radar doesn’t process visual light, what it sees is not personally identifiable." Bullshit!
- dmd 5y agoYup. I work with MRI data. When de-identifying brain scans we have to strip the face off [0] because that's easily identifiable. [0] https://github.com/PeerHerholz/BIDSonym https://github.com/PeerHerholz/BIDSonym for example
- droopyEyelids 5y agoPerhaps they mean it skirts current privacy rules through legislative technicality.
- transpute 5y agoGood point. In that case, legislation can be amended to close previously-invisible loopholes.
- argvargc 5y agoIf we won't see it coming, and as many commenters have pointed out, it will indeed comprise personally-identifiable information - despite the (absurd?) claims - how does this all fit with the GDPR right to be forgotten?
- teddyh 5y agoThe science fiction novel Snow Crash mentions and features mmWave radars frequently.
- hornej 5y agoMy original intro was about Snow Crash. Still waiting for mmWave on my skateboard http://pages.erau.edu/~andrewsa/Project%203/Thompson_David/Snow%20Crash%20Project%203/Thompson.html http://pages.erau.edu/~andrewsa/Project%203/Thompson_David/S...