13 ms·
Nvidia GPUs can break Chrome's incognito mode
- ericlamb89 11y agoawesome -- also I'm glad you censored the porn but not the porn title ;)
- pstrateman 11y agoThere's performance overhead to clearing memory. If you were them would you take the performance hit?
- fixermark 11y agoThat's a loaded question. If I (with training in how to design an OS and the risks of handing nonzeroed pages to another process) were them? It'd be part of my standard process for designing a memory repurposing library. But I can 100% understand how this mistake gets made; I wouldn't be surprised if it wasn't an explicit performance decision.
- Retr0spectrum 11y agoGPUs have loads of memory bandwidth. I can't imagine a framebuffer taking more than a few microseconds to clear. For example, Nvidia claims that the GTX 980 has a memory bandwidth of 223 GB/s. (1920 * 1080 * 3)/223e9 = 27us. Clearing all 4GB of VRAM would take 4/223 = 18ms. This would have a negligible impact on user experience in most cases. I guess the driver could also erase memory in the background as soon as it is deallocated, with zero user impact.
- qb45 11y agoPerformance is important because people use benchmarks to choose between vendors. And even if it's <0.1%, there is strong "optimization mentality" in those companies (because perf matters) so it's unlikely to happen in the current climate.
- exDM69 11y agoIt's high bandwidth but also high latency. If a page was cleared every time it was allocated, it would cause very unpredictable performance because the CPU would have to tell the GPU to clear memory and then wait for the GPU to finish before any other operations on the buffer could be done. This definitely isn't something that should happen for every allocated page. It might be acceptable if this happened only to pages previously used by other processes. But it would still be unpredictable and could cause unwanted stalls in the middle of a game session, for example. Also note that memory bandwidth is typically the bottleneck in modern games. The best place to do this would be in the browser, clearing out any textures and buffers before deallocating them if the contents are deemed private.
- Dylan16807 11y ago>because the CPU would have to tell the GPU to clear memory and then wait for the GPU to finish before any other operations on the buffer could be done If you're doing write-only operations, the CPU can queue them behind the clear. If you do a read, then the CPU has to wait whether you clear or not. Latency doesn't matter. Clearing can be slotted in with other operations, such as first use.
- psi-squared 11y agoAs the article points out, every modern OS clears (main) memory before handing it over to a new process. The cost is often mitigated a bit by using spare CPU cycles to zero out free pages, and by keeping a buffer of such pages. You only need to pause to zero pages if you have sustained 100% CPU usage for a long time - and that's pretty rare on most machines. GPUs probable even more so. GPUs generally have less memory, many fewer (but larger) allocations, and way higher memory bandwidth than CPUs, so it shouldn't be a problem for them to do this.
- AaronFriel 11y agoThere's performance overhead to doing everything, but one way to simulate zero'd virtual memory is to simply map them to a zero page and, when a full page write occurs to simply write the page, and when a partial write occurs to zero the rest. I am not familiar with GPU internals enough, but my understanding is that the GPU should be smart enough to know that a given texture or framebuffer will occupy n full pages, and so when either is written in its entirety, the zeroing only has to occur at the edges. (I would assume that the write would start on a page, but I don't know anything about GPU internals.) Caveat emptor: I will reiterate I know very little about memory internals. It seems like a bigger issue is that GPU memory is not virtualized and all users get access to the same memory. It's as if three decades of understanding the utility of virtual memory were forgotten.
- TeMPOraL 11y ago> It seems like a bigger issue is that GPU memory is not virtualized and all users get access to the same memory. It's as if three decades of understanding the utility of virtual memory were forgotten. I think you're forgetting the most important thing here - GPUs are meant to be fast. Virtualization will add like what, an order of magnitude to the access times?
- pcwalton 11y agoGPUs have had MMUs for a while (though they don't recover from page faults the same way CPUs do, I don't believe).
- crzwdjk 11y agoLast I checked, they pretty much don't recover from page faults, they just abort whatever "program" you're trying to run, so you can't really use the MMU for clever things like demand paging. But that's not the point of the GPU's MMU in the first place.
- exDM69 11y ago> Last I checked, they pretty much don't recover from page faults, they just abort whatever "program" you're trying to run Correct. When the GPU page faults, it causes a CPU interrupt and the driver will handle the interrupt. It's not possible to resume execution on a GPU in a timely manner so the only option is to terminate the process that caused the page fault. > so you can't really use the MMU for clever things like demand paging Recent GPU generations support "sparse" or "tiled" memory where the GPU can detect if a load or a store would access non-resident memory and then act accordingly. This requires a specialized shader and some CPU-side logic to actually stream in the memory. This can be used to on-demand paging for textures and buffers as well as implement workarounds to reduce visual artifacts from streaming.
- shirro 11y agoYes. Security and correctness should always come before performance. Performance first is the thinking that got us security vulnerabilities everywhere.
- fixermark 11y agoIt's also the thinking that got us Doom. ;) There are use cases where performance trumps security; the only issue here is that "multi-app semi-trusted computing environment" isn't one of them.
- nkozyra 11y agoInterested in details about this; where were Doom's major security issues? I love reading about Carmack/Doom development in general, maybe I've missed the part where caution was thrown to the wind and security was ignored.
- trbvm2 11y agoSeems like there never were security issues. At least, none that were talked about widely. I think that should be expected for a single player game. Almost all games are hack-able in some way of course but hacking a single player game is mostly an exercise in replay-ability. On a related note: Doom apparently does contribute to security proof of concepts though. http://www.techtimes.com/articles/15606/20140916/security-expert-hacks-canon-pixma-printer-to-play-doom-highlights-security-weakness.htm http://www.techtimes.com/articles/15606/20140916/security-ex... Which makes me wonder if the non-clearing memory issue exists for the printer's video driver and whether that could be used to retrieve something like a saved password or ssh key.
- PhasmaFelis 11y agoWhat security compromises do you claim Doom made to achieve greater performance?
- Hydraulix989 11y ago
- to3m 11y agoIt only needs to do it when first handing out a particular bit of memory to a particular process. The vast majority of the time, a process will be receiving memory it's had before (and no clearing is required). When not, it will be initializing the memory as part of the creation step (and no clearing is required), or it will be doing something it's not going to be doing all that often, such as creating a whole new frame buffer (and the clearing isn't a problem). I'm not convinced this would be a huge performance hit. Modern GPUs are not exactly slow at clearing memory either. Maybe the system doesn't pass enough information through to the driver to let it determine this, though...
- hn_123_throw 11y agoSerious kudos to the author for posting this even though he mentions viewing pornography. I had a similar problem on iOS. When I load Safari, there's usually a flash of the previous screen (probably cached as a PNG), then the page loads. I think it looks junky; I'd prefer a "loading" screen. It would flash the previous screen whether I was in private mode or not. So porn would flash on my screen. I didn't file a bug report or mention it on my twitter because I'm a little afraid of the reception. So, again, thank you charliehorse55. edit: i said "cached as a PNG" but that's just what I thought prior to reading this article. it could be many things, including this bug.
- 0x0 11y agoI've seen that too, sometimes the tab previews in the tab overview often show outdated thumbnails even if you wait a while - it never settles to the correct thumbnail.
- mahouse 11y agoBelieve it or not everybody watches porn.
- VeilEm 11y agoThis is totally not true. :/
- chirau 11y agoOf course excluding the blind
- cbd1984 11y agoIt isn't that everyone watches porn, it's that watching porn is just as moral as watching any other form of entertainment. Anti-porn crusaders aren't necessarily hypocrites. They also don't need to be hypocrites to be wrong.
- mahouse 11y ago
- 0x0 11y agoThis sounds like fun, especially if webpages can use WebGL to read old buffers back into javascript variables - and then AJAX them out silently in the background. (preserveDrawingBuffer + canvas.toDataURL() + ajax ?) Edit: Also, "google chrome incognito mode is apparently not designed to protect you against other users on the same computer".. what? Isn't that the only thing it can and should protect against? It's not like it can protect against non-local users (i.e. HTTP network interceptions)
- brtmr 11y agoThis is exactly the reason why the webgl standard strictly forbids allocating buffers without clearing them first. Otherwise anything the user looked at since the last power cycle - including emails, passwords, private keys, ... -could be extracted by visiting a website.
- 0x0 11y agoHow long until we see the first infoleak bug where some combination of OS+driver+browser+webgl-command-sequence misses a buffer to clear - or "optimizes" it away - or fails to bounds-check a texture coordinate - etc? :)
- chucky 11y agoWe've already had these kinds of issues with webgl. Here's one that I found through some googling: http://www.cvedetails.com/cve/CVE-2014-3173/ http://www.cvedetails.com/cve/CVE-2014-3173/ You don't need webgl for this kind of infoleak either, regular good old 2d canvas also supports allocating memory. It also supports reading the current state of all of the pixels in the buffer through Javascript, so if you have an exploit that gets you an uninitialized canvas you can easily send whatever memory contents you got back to your server for later analysis.
- anon4 11y agoNot always. I was trying to write an android application to serve as a frontend to a site by launching a background webview, drawing the elements I'm interested to a canvas and sending the pixels back to the application. (Un)fortunately, after you draw an HTML DOM element to a canvas, you're forbidden from reading the canvas pixels back and there's no flags you can set on the webview to let you do it.
- kevingadd 11y agoOP, it looks like this is on OS X (from the screenshots), in which case you should probably report it to Apple as well. The driver stack on OS X is a mix of Apple and NVIDIA code.
- rocky1138 11y agoBack in 2000, a very similar problem was my first lesson in frame buffers. I was watching some adult material using Quicktime on Windows 98. A few hours later, I wanted to show my mom something on my computer. As it loaded the new video in Quicktime, the last frame of the porno sat there in inverted colours until the new video began to play. I had closed Quicktime hours ago... what was that still doing there in memory? Needless to say it was very awkward.
- nailer 11y agoI was a Linux user in the early 2000s and learnt about framebuffers in a similar but less embarrassing way: sometimes X would crash, and before the root weave was drawn, the last thing you were doing when X crashed appeared for a moment. X crashed a lot back then, so everyone learnt pretty quickly.
- mmsmatt 11y agoI dual booted Win 98 and various Debian flavors. Once I got OpenGL going, I would sometimes see the "last" frame of a Windows session flash as X started up.
- TeMPOraL 11y agoI learned about it by accident too, but fortunately in a less embarrassing way - when I was first learning OpenGL I've noticed that displaying uninitialized framebuffer could sometimes cause "random noise" to show up. "Random noise", that when the window size was just right, suddenly turned into a screenshot of the game I was playing moments ago. :).
- mrsteveman1 11y agoSimilar happened to me, except it was the entire family staring at a 50" TV waiting for me to start a movie Christmas eve.
- deleted 11y ago[deleted]
- TeMPOraL 11y agoInteresting. I've never thought of it as a security issue. But it's something that's around since forever. I've seen old framebuffers containing stills from games or videos showing up when resizing OpenGL applications 15 years ago. Video cards don't clear memory for the same reason nothing is ever deleted by default - it's a waste of time.
- netheril96 11y ago> Video cards don't clear memory for the same reason nothing is ever deleted by default Modern operating system zeroes memory pages all the time. It is a security measure, and ensuring security is by no means a waste of time.
- brtmr 11y agoPrevious discussion on the same subject, about a post written by me: https://news.ycombinator.com/item?id=9245980 https://news.ycombinator.com/item?id=9245980 Basically, this issue is not restricted to NVidia GPUs or specific operating systems - This can be reproduced on Windows, Linux and OSX. Basically the concept of memory safety does not exist in the gpu space - which is the reason why the webgl standard is so strict about always zeroing buffers. The issue of breaking privacy and privilege boundaries on a multiuser system is very real, and there is no workable solution. This seems to be one of those problems where a lot of people are aware, but no one is sure how to fix it and so it just stays how it is.
- charliehorse55 11y agoYeah, I remember reading that when it was first posted. Glad to see other people are aware, but it's disappointing nothing has been done. The fix is pretty simple, the GPU manufacturer just needs to update their driver to zero the VRAM like an OS would with RAM.
- cbd1984 11y agoAre device drivers ever updated, much less for security issues? Seems like an obvious way to increase the take, in any event: "Chip X has a security flaw. Chip Y does not. Buy chip Y now or Evil People will own-zor all your cash-zors."
- 13of40 11y agoSometime around 1995 I had a brand new 80486 Dx-100 I was putting together, and just for kicks I decided to pull the CPU out while it was running. I expected some kind of epic C-64 style rainbow gibberish crash, but it actually just froze, with the Windows 95 desktop still on the screen.
- dogma1138 11y agoWDDM 2.0 gpummu is supposed to ensure that the memory has been zeroed between different applications that use virtual GPU memory. If this is the case there might be a compliance issue on nVidia side which makes me wonder if webgl is vulnerable also. WebGL was amended to request a zero when provisioning or disposing of a buffer but it relies on the API which is handled by the driver if nVidia is taking some shortcuts to save time it might be possible to leech stale memory this way.
- exDM69 11y ago> WDDM 2.0 gpummu is supposed to ensure that the memory has been zeroed between different applications that use virtual GPU memory. Which Windows version introduced this WDDM version? Could it be that OP is running an older version? > WebGL was amended to request a zero when provisioning or disposing of a buffer but it relies on the API This is indeed a tricky situation. All modern GPUs do "zero bandwidth clears" which means that upon clearing, nothing gets written to the actual framebuffer, the memory is just marked "cleared" (by writing some special bits to the L2 cache, for example). This makes it difficult to reason whether there's any sensitive content left in the framebuffer. edit: nevermind, the OP seems to be using OSX, so it's not WDDM. Additionally, the OSX GPU drivers are written by Apple.
- dogma1138 11y agoYeah this was confusing he said that it was an Nvidia issue which is why I thought it was on Windows. As far as WDDM goes 2.0 requires that for sure I'm pretty sure this was part of the original WDDM GPUMMU spec also but I can't really find those details anymore on MSDN since most of the pages refer to 2.0 atm.
- stordoff 11y ago> Of course, it doesn’t always work perfectly, sometimes the images are rearranged. I've seen the same behaviour on OS X with an Intel GPU: https://i.imgur.com/3fagsYx.jpg https://i.imgur.com/3fagsYx.jpg (screenshot of the contents of a browser tab - pretty sure it was Chrome. The Rooster Teeth page you can see parts of had been closed hours prior)
- rawnlq 11y agoGoogling shows that this bug previously received a $1000 bounty in 2012 and should already been fixed: https://code.google.com/p/chromium/issues/detail?id=152746 https://code.google.com/p/chromium/issues/detail?id=152746
- scurvy 11y agoReminds me of when network card drivers would use random bits of memory to pad out minimum Ethernet frames. Oh hey, there's your sensitive data going out in an ICMP ping request.
- maaaats 11y agoDo you have more info about this? Sounds crazy!
- 0x0 11y agoFor example, http://www.securitytracker.com/id/1008910 http://www.securitytracker.com/id/1008910 It's basically heartbleed in your ethernet driver in 2003.
- Dylan16807 11y agoOut to the boundary of your (wired) LAN.
- scurvy 11y agoThat didn't excuse it then, and it doesn't excuse it now. Sniffing the keys to the kingdom in the clear is bad news bears.
- logn 11y ago> Google marked the bug as won’t fix because google chrome incognito mode is apparently not designed to protect you against other users on the same computer (despite nearly everyone using it for that exact purpose). What's the purpose of incognito mode then? It doesn't protect you from your ISP, websites, or users on the same computer. I'm not sure what other use case there is.
- netheril96 11y agoIs this issue specific to Google Chrome? Does Safari has the same problem? In principle, I'd expect so, but if Safari surprises me, maybe I will jump ship.
- bradhe 11y agoUh, anyone else think this title is a bit sensationalist? I was expecting something a bit more along the lines of actually leaking usable private data, not just displaying a rastered frame. Even further this has very little to do with chrome. The only way chrome could actually fix this issue would be if it nuked the frame buffer when it released it. This is a fine idea, but if I was a dev in that context I would assume the OS would make stronger guarantees than that?? If anything, this is an edge case Chrome devs (and other developers) could protect themselves against if they were so inclined, but I'm not surprised they didn't assume they needed to protect against this.
- djur 11y agoThe primary use case for Incognito Mode, as far as I know, is so a user can casually use a browser without leaving inadvertent artifacts of their usage on the machine. Having a page you visited in Incognito Mode be visible in Chrome after Incognito Mode is closed seems to be precisely the kind of thing users expect the feature to prevent.
- cookiecaper 11y agoIf you associate the Chrome browser with your Google account (not just logging in to a Google site, but going to settings and putting the information in there), the history will be synced across several devices. Incognito can be used to prevent pages viewed on your phone or laptop from going to your desktop's history or vice-versa.
- netheril96 11y agoIt would be nice if a software exists that allocates many many GPU memory pages and clears them, so we can be safe after viewing p-something.
- cmrx64 11y agoCheck out this paper if you're interested in a solution to this and other problems (somewhat amusingly, using the GPU): https://www.cs.utexas.edu/~sangmank/pubs/lacuna.pdf https://www.cs.utexas.edu/~sangmank/pubs/lacuna.pdf In particular, they refute via counterexample the arguments that VMs or secure deallocation alone are sufficient.
- rubberstamp 11y agoThis is a hack that is caused by hardware, due to some way in which its currently designed. As such, I doubt if software patches would be able to fix it. A while back I read an article on arstechnica that described stealing encryption just by touching exposed metal parts of laptop. http://arstechnica.com/security/2014/08/stealing-encryption-keys-through-the-power-of-touch/ http://arstechnica.com/security/2014/08/stealing-encryption-...
- byuu 11y ago> Google marked the bug as won’t fix because google chrome incognito mode is apparently not designed to protect you against other users on the same computer. That's nonsense. For most users, that's exactly what it's used for. I really think Google is dropping the ball here. I know it's not their bug, and they shouldn't have to work around it in an ideal world, but this is a pretty clear leak of data outside of private mode. It wouldn't impact performance in any noticeable way (you're closing the window anyway at this point), and would just be an extra safeguard. Very short-sighted of them to ignore this bug. Perhaps we could ask distro maintainers to add patches for this to their builds of Chromium.
- Zekio 11y agothey could just fill the buffer with images of the nvidia logo free commercials for themselves?
- Animats 11y agoThis should be easy to fix at the driver level. Window close and and GPU resource release are not operations that occur often enough that memory clearing would affect performance.
- chris_wot 11y agoAnd this is why Linux folks hate proprietary drivers. At least the developers of open source drivers would fix this. Probably very promptly! There's a reason Linus Torvalds flipped the bird to NVidia. Here is a perfect example of the reason why closed source drivers suck.
- vermilingua 11y agoSame issue with Samsung GS5... on Snapchat... with disastrous results...