4 ms·
High-resolution efficient image generation from WiFi Mapping
- jychang 1y agoThe image examples from the paper are absolutely insane. Is this just extremely overfitted? Is there a way for us to test this? Or even if the model isn't open source, I'd pay $1 to upload a capture from my wifi card on my linux box and upload it to the researchers and have them generate a picture and see if it's accurate
- RicDan 1y agoYeah this seems too insane to be true. I understand that wifi signal strength etc. is heavily impacted by the contents of a room, but even so it seems farfetched that there is enough information in its distortion to lead to these results.
- esrh 1y agoA lot of wifi sensing results that have high-dimensional outputs are usually using wideband links... your average wifi connection uses 20MHz of bandwidth and is transmitting on 48 spaced out frequencies. In the paper, we use 160MHz with effectively 1992 input data points. This still isn't enough to predict a 3x512x512 image well enough, which motivated predicting 4x64x64 latent embeddings instead. The more space you take up in the frequency domain, the higher your resolution in the time domain is. Wifi sensing results that detect heart rate or breathing, for example, use even larger bandwidth, to the point where it'd be more accurate to call them radars than wifi access points.
- deleted 1y ago[deleted]
- tylervigen 1y agoThat is not how it works. The images of the room are included in the generative training model. The wifi is "just" helping identify the locations of objects in the room. If you uploaded a random room to the model without retraining it, you wouldn't get anything as accurate as the images in the paper.
- fxtentacle 1y agoFYI the images are not generated based on the WiFi data. The WiFi data is used as additional conditioning for a regular diffusion image generation model. So what that means is the WiFi measurements are used for determining which objects to place where in the image, but the diffusion model will then fill in any "knowledge gaps" with randomly generated (but visually plausible) data.
- jstanley 1y agoI'm confused about how it gets things like the floor colour and clothing colour correct. It seems like they might be giving it more information besides the WiFi data, or else maybe training it on photos of the actual person in the actual room, in which case it's not obvious how well it would generalise.
- owainBren 1y ago[dead]
- f_devd 1y agoThis is what GP eludes to, the original dataset has many similar reference images (i.e. the common mode is the same), and the LatentCSI model is tasked to reconstruct the correct specific instance (or a similarly plausible image in case of the test/validation set)
- Aurornis 1y ago> I'm confused about how it gets things like the floor colour and clothing colour correct. The model was trained on the room. It would produce images of the room even without any WiFi data input at all. The WiFi is used as a modulator on the input to the pre trained model. It’s not actually generating an image of the room from only WiFi signals.
- gblargg 1y agoIt wouldn't generalize at all. The Wi-Fi is just differentiating among a small set of possible object placement/orientations within that fixed space, then modifying photos taken appropriately, as far as I can tell.
- equinox_nl 1y agoI'm highly skeptical about this paper just because the resulting images are in color. How the hell would the model even infer that from the input data?
- orbital-decay 1y agoThat's just a diffusion model (Stable Diffusion 1.5) with a custom encoder that uses CSI measurements as input. So apparently the answer is it's all hallucinated.
- pftburger 1y agoRight but it’s hallucinating the right colours which to me feels like some data is leaking somewhere. Because no way wifi sees colours
- moffkalast 1y agoWell perhaps it can, a 2.4Ghz antenna is just a very red lightbulb. Maybe material absorption correlates, though it would be a long shot?
- jstanley 1y agoYou can't even pick colour out of infra-red-illuminated night time photography. There's no way you can pick colour out of WiFi-illuminated photography.
- steinvakt2 1y agoIf it sees the shape of a fire extinguisher, the diffusion model will "know" it should be red. But that's not all that's going on here. Hair color etc seems impossible to guess, right? To be fair I haven't actually read the paper so maybe they explain this
- defraudbah 1y agodownvoted until you read the paper
- malux85 1y agoPSA: If you publish a paper that talks about high resolution images can you please include at least 1 high resolution image. I know that is a subjective metric but by anyone’s measure a 4x4 matrix of postage stamp sized images are not high resolution.
- nntwozz 1y agoOne step closer to The Light of Other Days. "When a brilliant, driven industrialist harnesses the cutting edge of quantum physics to enable people everywhere, at trivial cost, to see one another at all times: around every corner, through every wall, into everyone's most private, hidden, and even intimate moments. It amounts to the sudden and complete abolition of human privacy--forever."
- nashashmi 1y agoSo privacy is a mathematical function using variables of cost, capability, control, reach?
- esrh 1y agoThis is my paper (first author). I think the results here are much less important and surprising than what some people seem to be thinking. To summarize the core of the paper, we took stable diffusion (which is a 3-part system of an encoder, u-net, decoder), and replaced the encoder to use WiFi data instead of images. This gives you two advantages: you get text-based guidance for free, and the encoder model can be smaller. The smaller model combined with the semantic compression from the autoencoder gives you better (SOTA resolution) results, much faster. I noticed a lot of discussion about how the model can possibly be so accurate. It wouldn't be wrong to consider the model overfit, in the sense that the visual details of the scene are moved from the training data to the model weights. These kinds of models are meant to be trained & deployed in a single environment. What's interesting about this work is that learning the environment well has become really fast because the output dimension is smaller than image space. In fact, it's so fast that you can basically do it in real time... you turn on a data collection node and can train a model from scratch online, in a new environment that gets decent results with at least a little bit of interesting generalization in ~10min. I'm presenting a demonstration of this at Mobicom 2025 next month in Hong Kong. What people call "WiFi sensing" is now mostly CSI (channel state information) sensing. When you transmit a packet on many subcarriers (frequencies), the CSI represents how the data on each frequency changed during transmission. So, CSI is inherently quite sensitive to environmental changes. I want to point out something that most everybody working in the CSI sensing/general ISAC space seems to know: generalization is hard and most definitely unsolved for any reasonably high-dimensional sensing problem (like image generation and to some extent pose estimation). I see a lot of fearmongering online about wifi sensing killing privacy for good, but in my opinion we're still quite far off. I've made the project's code and some formatted data public since this paper is starting to pick up some attention: https://github.com/nishio-laboratory/latentcsi https://github.com/nishio-laboratory/latentcsi
- phh 1y agoIs there a survey of SoTA of what can be achieved with CSI sensing you would recommend? What is available on the low level? Are researchers using SDR, or there are common wifi chips that properly report CSI? Do most people feed in CSI of literally every packet, or is it sampled?
- nashashmi 1y agoWhere is the color info coming from? It can’t come from WiFi. Is that being fed in using a photo?
- brcmthrowaway 1y agoSo the applications of this work is.. surveillance. Why are there people working in this space?
- cracki 1y agoSo they trained a model on a handful of poses? OK cool. Any "unknown" state of the scene is bound to confuse it.
- Lumoscore 1y agoHonestly, this whole field of research—turning WiFi signals into visual data—is one of the coolest and scariest things happening in tech right now. Basically, researchers figured out how to use the invisible radio waves from your Wi-Fi router to create surprisingly clear pictures of whatever is around it, even if there are walls in the way. Your router is constantly firing out radio signals, right? When those signals hit a person, a dog, or a chair, they bounce off and create a unique echo pattern. This echo pattern is called CSI (Channel State Information). It's a precise digital "shadow" of everything in the room. Turning that messy echo pattern into an actual picture used to be super difficult and slow. But now, they use a fancy type of AI—the same kind that generates images when you type a prompt—to do the heavy lifting. The AI is super smart and knows how to instantly translate that invisible echo pattern into a high-resolution image. So the Big Picture is, It's like they've figured out how to use your average home Wi-Fi to "see" without light or a camera, and they can do it so efficiently (quickly and cheaply) that it might become a normal thing. It’s pretty wild, and the applications are huge—especially for things like monitoring the health of older people without putting cameras in their rooms. Of course, it also means walls don't stop surveillance anymore, which is kind of unsettling!