14 ms·
Nvidia CEO: We'll be solving VR immersion problems for the next 20 years
- moloch 10y agoThe actual quote was: “First of all, VR displays are a little too cumbersome. It has to be much more elegant, being connected by a wire has to be solved. The resolution has to be a lot higher. The physical worlds do not behave according to the laws of physics. The environment you’re in isn’t beautiful enough. We’re going to be solving this problem for the next 20 years.” "Solving over 20 years" / "20 years from being solved" are very different things.
- vessenes 10y agoThe actual quote is excellent (and already on this page). He's right -- the list of things to be done is obvious, and will take time. And, I'd like to imagine he has some enjoyment at the thought -- if you saw his keynote showing off a precomputed(ish) raytraced walkthrough of their new office space, he was really enjoying himself and the possibilities. What Hsun has is an incredibly good window into the silicon side of the problem, and he knows how many chip cycles he's got to address the amount of computation needed for physics, visual realism and presumably a latency budget to allow wireless support. Plus he can estimate what screen tech will be doing. It's a prodigious amount of R&D, and it is a bit breathtaking. Finally having a reason to do it, and a taste of how appealing it can be is pretty cool.
- _yosefk 10y agoYou mean how many chip cycles he needs, right? How many he's got we all know: maybe 2. Then CMOS scaling is over. (Not that things will completely stop evolving, but they'll surely slow down.)
- vessenes 10y agoIn 20 years? I think he's got at least 4, maybe 5. Of course, I don't know how big those evolutions will be in terms of advances, but I'm just a silicon dilettante, nothing like an expert.
- Bartweiss 10y agoI think the dispute here is that there are two different standards in play. He has time for 4-5 cycles, but Intel only has concrete plans for ~2 cycles. After that the existing die structures can't be meaningfully reduced anymore, and everything is up in the air. Maybe we get some wild, big-but-fast optical chips, maybe we get a breakdown of Moore's law. Of course, 3d chips look fairly achievable and might eke out a few more cycles, but things will get interesting sometime before that 20 year deadline.
- dogma1138 10y agoI don't think Intel plans for more than 2 fabrication cycles regardless of some gloom and doom omg quantum physics go away plox prophecy. 14nm FinFet GPU's are only coming in now, and they'll stay on 14nm for at least 3-4 years. 10nm GPU's will probably come out in 2019-2020, 7nm GPU's will come out in 2025 or so, and 5nm GPU's will come out in 2030, that's already ~15 years and considering that everyone is predicting that we'll have to stick to processes longer as the small we get the more time it takes to actually optimize them for good yields and performance you probably have 3-5 years there too. So even if we do not adopt any major process changes like switching to graphene, solid state photonics to replace the metal pathways and even optical transistors or w/e else people are cooking up to enable us to continue to improve our integrated circuits he can still be correct about the time frame. Also regardless of what happens or not he was talking about true VR immersion this means that allot has to change other than just how good and convincing the graphics are this also includes learning to develop VR applications and as VR is a very new media that alone can take 20 years.
- cm3 10y agoDo you have a link to the keynote video?
- mozumder 10y agoAnd then you have the fundamental design problems of VR, where it blocks out your environmental awareness, causing people to crash into walls, leading to liability issues: https://imgur.com/umYTJP1 https://imgur.com/umYTJP1 The first generation VR also faced the same liability issues. Specifically, the Atari Jaguar VR was cancelled because of this. VR will never make it as a consumer product. Ever.
- refulgentis 10y agoOh my. http://gizmodo.com/5846133/the-best-ipod-prediction-ever-made http://gizmodo.com/5846133/the-best-ipod-prediction-ever-mad... comes to mind. Want to put money on that? Say, 40 million unit sales a year within a decade?
- gfodor 10y agoHave you tried the Vive?
- ChrisClark 10y agoHe obviously hasn't. :)
- hodwik 10y agoAgreed. I was very VR skeptical until I tried the Vive last month. Now I'm convinced this is around the corner.
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- adamnemecek 10y agoLol, future technology prediction "responsive level design".
- alberthartman 10y agoSome of this may be misdirection. VR is getting tremendous attention and will be very competitive. Look how he backed away from mobile. Complaining about wires and resolution? They're getting new revs every 12-18 months now. Look for hidef wireless hmds in 20-36 months, not 20 years. No physics? Look again. I think he wants to avoid competitive low margin businesses when better choices are available.
- milesokeefe 10y ago>Look for hidef wireless hmds in 20-36 months, not 20 years. How do you suppose that will happen? On-device computing hardware or a wireless tether? The latter appears to only be maybe possible with microwave radio, but requires constant line of sight which is a huge issue. The former may be possible depending on your definition of hidef. IMO I doubt it though, much of what makes VR immersive is having a HMD lightweight/comfortable enough that you forget it's on. That's hard enough when the only hardware in the HMD is essentially screens and an IMU. I can't imagine the added weight of CPU/GPU/batteries/etc not making a significant enough impact on comfort to make it not worth it, in the next 20-36 months. For reference, my standard of hidef VR right now, not in 20-36 months, is Vive level resolution/FOV/tracking accuracy and latency; I wouldn't put GearVR in that group, or any platform without submillimeter positional tracking for that matter.
- Negative1 10y agoNot interested in Mobile unless it is for autos? That is a pretty big declaration from one of (if not the) worlds biggest GPU manufacturers. Does this spell the end of the Shield or is that not considered mobile?
- m_mueller 10y agoIt seems apparent to me that the strengths of desktop chip manufacturers don't translate well to mobile. Both Intel and Nvidia have had tremendous problems to compete against ARM / PowerVR. While Nvidia has done a better job, I imagine they are still loosing money on that front. Maybe this is the time for them to acknowledge that and instead concentrate on markets where local data parallel computations are actually useful and currently feasible (due to power requirements). Maybe mobile might get there as well (real time on-chip image recognition), but it's still years off I think.
- CountSessine 10y agoBoth Intel and Nvidia have had tremendous problems to compete against ARM / PowerVR. While Nvidia has done a better job Has Nvidia had any success at all in mobile? At least Intel has managed to get some Atom chips in some obscure Dell tablets and Asus mobile phones. I'm very curious about this - Nvidia's whole mobile business looks like a flop and Huang's circumspect dismissal of it sounds like sour grapes to me.
- kbaker 10y agoThey had some wins with the Tegra line... Tegra 3 was pretty big in tablets and phones a few years back. Not sure how well it's going currently. But much better than mobile Atoms for sure.
- m_mueller 10y agoWell at least they have some performance benchmarks to show for [1] (I wasn't able to find a comprehensive sustained performance-per-watt comparison though, which would be much more interesting) and at least they sold more than half a billion dollars worth of Tegras in 2015. Looking at [2] that's at least substantially more than Intel sold when their mobile business crashed and burned in 2014 [3]. It's however not as much as Intel sold in 2013 and the last quarter of 2015 wasn't good for Nvidia either. Maybe it's fair to say that Nvidia is just learning the same lesson with a bit of delay, but at least they have a very strong pivot going after the car market with rapidly increasing compute requirements. [1] http://wccftech.com/nvidia-tegra-x1-benchmarks-apples-a8x/ http://wccftech.com/nvidia-tegra-x1-benchmarks-apples-a8x/ [2] http://marketrealist.com/2015/03/why-nvidia-continues-to-focus-on-tegra/ http://marketrealist.com/2015/03/why-nvidia-continues-to-foc... [3] http://www.extremetech.com/computing/227816-how-intel-lost-the-mobile-market-part-2-the-rise-and-neglect-of-atom http://www.extremetech.com/computing/227816-how-intel-lost-t...
- chris_va 10y agoThe really big, really difficult one will be enabling dynamic focal depth. The lack of focal planes makes it difficult to imagine VR providing a lifelike experience. Some volumetric/panoptic displays are getting closer, along with eye tracking, but this seems like one of those things you cannot fix in software. Just adding higher pixel resolution, improving "physics", or creating a more beautiful environment will not solve this. So, 20 years is probably right.
- hodwik 10y agoBut 'all' you have to do is track the pupil, and push focus on the object they're looking at. We're already rendering the focal length live in most games, there's no reason we couldn't shift the focal length affording to what the viewer's pupils are pointed at. Alternatively, they could use a modified biology, where the direction the head is pointing chooses the relative DOF, basically you render a fairly wide depth of field around whatever the head is pointed at. Anyway, you can get a pretty lifelike experience with just some basic atmospheric perspective. Look at photography with very wide depth of field [1], so everything is in focus simultaneously. You sorta just get used to it, and accept it. [1] http://cdn.shutterbug.com/images/0814picture03.jpg http://cdn.shutterbug.com/images/0814picture03.jpg
- deleted 10y ago[deleted]
- modeless 10y agoAs someone who works on eye tracking for VR headsets, this is much easier said than done. Doing it well requires levels of accuracy, latency, and reliability that are not available in any eye tracker on the market today.
- rjvs 10y agoWhat about SMI already tracking pupils at 250Hz? https://www.youtube.com/watch?v=Qq09BTmjzRs https://www.youtube.com/watch?v=Qq09BTmjzRs That tech is still expensive and I don't know how accurate or reliable but if it is good enough for foveated rendering, I would have expected it to be good enough for focus as well. Care to elaborate as to why it wouldn't work, or is it just beyond your horizon of "today"?
- kordless 10y agoIn 20 years we might be able to implement something that works with our visualization systems to produce images at will...at least for those of you without Aphantasia.
- _greim_ 10y agoCan't remember where I read this, but the idea is that the brain compresses a vast amount of data down to a small amount of meaningful information. VR's current approach is to meticulously spoof that vast datafeed in such a way that the result gets compressed down to something that seems meaningful. Which is a lot of work. Why not go in post-compression and merely spoof the meaningful bits? VR's job could be a lot easier. I don't know how it would play out, but speculating, you might render a low-poly monochromatic "skeleton" scene and then simply suggest to the brain ways to flesh it out with feeling and texture. Show the user photos of war-torn Europe in 1947 beforehand for example. Even if it means direct neural interfaces, considering that we're 20 years out using the current approach, maybe that isn't so far fetched.
- exhaze 10y agoThis is the premise of a 90's sci-fi novel about VR (https://en.wikipedia.org/wiki/Labyrinth_of_Reflections https://en.wikipedia.org/wiki/Labyrinth_of_Reflections). In the novel, a character discovers a graphical sequence that puts people into a hypnotic state in which they can be shown a low-res virtual environment which their brain then fleshes out.
- mxfh 10y agoAren't these highly compressed and encoded concepts called books and language already?
- captainmuon 10y agoI always wondered if you could use adversarial neural networks to find images that the brain recognizes as something else, just like you can fool NNs. You could use this to find low-poly representations of your objects that are indistinguishable from the real ones. Probably you'd have to calibrate this for every user though.
- usrusr 10y agoYou might need to reset the user occasionally during training, otherwise that adversarial neural network would hunt a moving target.
- lewisl9029 10y agoI just came across this today (VR Backpacks): http://www.anandtech.com/show/10370/hp-and-msi-demonstrate-backpack-pcs-for-vr-gaming http://www.anandtech.com/show/10370/hp-and-msi-demonstrate-b... At first I thought the idea was pretty ridiculous, but now that I've given it a bit more thought, I'm beginning to think having to carry around a backpack like this might not be such a bad compromise between the traditional stationary wired VR setup and the holy grail of fully wireless VR.
- jamesrcole 10y agoWhich makes me think, eventually technology will probably enable the processor to be in the headset itself. Then possibly an implant.
- Alphasite_ 10y agoThere are already headset only solutions, but who knows how well they run.
- Alphasite_ 10y agoThere are already headset only solutions, but who knows how well they run.
- pavlov 10y agoThe Microsoft Hololens integrates the computer running Windows into a fairly light AR headset.
- empath75 10y agoThink about it for arcade-like experiences, not for the home.
- deleted 10y ago[deleted]
- greenspot 10y agothe piggy trough is so full of cash, cream and credit, that they ignore credible assessment(s) about the VR industry from the top comment of that page Edit: Thanks for the downvote
- naringas 10y agowhat about AR? e.g. a hololens?
- mtgx 10y ago30 years. At least if you're talking about mapping everything at the same scale, graphic fidelity, resolution, price, and field of view as in VR, then AR will always be 10 years away compared to VR, in my opinion. If you're going to make it much more limited than VR (like say only show a chess table in front of you, or some text on top of stuff, but not much else), and make it cost several times more, like the Hololens does right now, then yeah, I guess you could make AR seem almost as usable as VR, but it will never be a 1:1 comparison this way. If you're doing 1:1 comparisons, then it should be about 10 years behind VR because you have to transpose the same world that you do in VR but over the real world.
- dclowd9901 10y agoHa, 20 years. I'd guess more around 100, and graphics card companies, unless they pivot, won't really be a part of it. If we're talking about true VR, it's the replacement of reality, e.g. piping experience directly into your neurons. For instance, how are we going to augment smell or touch with a headset and a video card? Sensory augmentation will probably start with dream augmenting tech (headsets that are able to give you "good" dreams by learning your neural firing patterns), then as delivery systems, nanotech, processing and brain knowledge gets better, we'll see it evolve into waking experience augmentation.
- bradhe 10y ago> If we're talking about true VR, it's the replacement of reality, e.g. piping experience directly into your neurons Yeah that's not what anyone here is talking about.
- dclowd9901 10y agoI'm sorry, what was your definition of virtual reality? I sort of read it as "a reality that is virtually reality itself." My point was those problems might be on the horizon of nvidia in the next 20 years, but its not even scratching the surface of what VR needs to be.
- adrusi 10y agoOK, yes, that is the ultimate vr destination, but like bradhe said, thats not what anyone's talking about. Like you said, that's more like 100 years off (I think its a lot more before the tech is really all there AND people are OK with the seriously invasive surgury that will be required, splicing the optic nerves and such. But its more reasonable to think well at least get a near-perfect audiovisual virtual reality headset in 20 years. If the numbers you're throwing around here are accurate, then there will be an 80 year stagnation in vr immersiveness, when the headsets are great but anything motion related feels fake, but computer/neural interfaces still limited to prosthetics, or at least out of reach for consumers and not powerful enough for vr. 80 years is a lot longer than TVs have been around, there's plenty of time for graphics card companies to rule the world.
- mtgx 10y agoI think many have misinterpreted what he was saying, especially if they haven't read the actual article. Nvidia's CEO is saying that VR won't be a "completely solved problem" for at least another 20 years. And that's very much true and has already been known by VR enthusiasts. Carmack was even saying stuff like you need a 12k or 16k display to have the "ideal" resolution in VR, years ago. Well, we're not going to get those kinds of displays in VR headsets anytime soon, and even if we do, that's orders of magnitude more performance that's needed compared to what we're using in today's PCs. On top of that, you also need "virtual reality" to be as close as possible to actual reality in terms of graphics fidelity, so that's a few orders of magnitude more in terms of performance needed. If you want "Matrix-style VR", then yeah, that won't be achieved for at least another 20 years. The headsets will also have to become "glasses-like" to increase adoption and make them more convenient, which is going to take at least another decade, too. All of that said, while I do think the 1440p resolution is way too low, and probably even 4k won't be the "optimal" resolution for VR, I do think VR will start becoming "mainstream" within the next 5 years. I think many took his comments to mean that VR won't be mainstream for another 20 years, but that's not what he was saying.
- ricksplat 10y agoHow many years from John Logie Baird's first invention did it take to get TV right. Mainstream? Colour? Picture & sound quality? Ditching the cumbersome CRT ... best part of a hundred years but people where ready to use it long before that and pay big wodges of cash for it too. I'd say there'll be a big market for sub optimal VR :)
- martinsb 10y agoI wander why Huang did not mention -- in my opinion -- the main issue with VR - the fact that many people (including myself) cannot use those headsets for a longer time than say 10 minutes without getting dizzy and sick. Or are these the consequences of those issues he mentioned - like "the physical worlds do not behave according to the laws of physics"?
- kybernetikos 10y agoHave you tried experiences where you do all the moving yourself - like most vive games? I never had terrible VR sickness, but when I used the DK1 I could feel my body change temperature by itself fairly quickly and then would need to breathe carefully to stay ok. In Vive where I'm controlling the movement I have basically no noticeable sickness. I know that's not everyones experience, but if you haven't tried room-scale vr, I recommend it.
- rictic 10y agoI can second that. I don't know of anyone[1] who's gotten VR sickness from a room scale Vive game. Only doing teleportation and 1:1 tracked motion has solved the nausea problem for essentially everyone. [1] Including my partner who gets nauseous from even short car rides, to say nothing of boats, airplanes, or the DK1.
- jshmrsn 10y agoI am not an expert in VR, but I know one of the main causes of sickness from VR headsets is the latency between the user's head movement and the simulated camera movement. Basically, there is lag between when you move your head in real life to when the VR headset displays that new view angle to your eyes. Reductions to this latency is an ongoing effort and companies like Nvidia are definitely in that space.
- jobigoud 10y agoWe already have infinitesimal latency thanks to predictive tracking in both the high end headsets. The main source of sickness nowadays is vection.
- rl3 10y agoEventually it'll make more sense to just use the human brain as a renderer. Perfect photorealism is fairly worthless without a solid method of interaction and the ability to literally suspend disbelief. Direct neural interfaces will be capable of everything, not just rendering. That said, I can't help but think of a scenario where we have functional neural interfaces but haven't yet mastered the human brain's grammar, and thus resort to "driving" the brain with traditionally-rendered images rather than feeding it the information required to create a scene from scratch. In such a scenario, the requisite quality of the traditionally rendered images might be interesting. Would photorealism be required, or simply a low-quality rendering to convey general structure? Presumably the brain would enhance the latter case to the point of photorealism—or at least to the dream equivalent.
- richmarr 10y agoAgreed, considering the amount of processing and information loss that happens in the optic nerve (in people with healthy vision) you could save yourself a lot of processing, and get a higher fidelity result (potentially indistinguishable from reality).
- visarga 10y agoIf I can tap into the brain, I'd make it like in Matrix, to be able to upload skills and connect minds together wit AI.
- ricksplat 10y agoI expect it would be some kind of a merge of the two (without severing the optic nerve that is) - neurons work by accumulation so you would effectively have the two realities summed - which would be neat for augmented reality. Two fully immerse you could close your eyes or put on dark glasses. A friend told me this is how visual hallucinations on LSD work ...
- ricksplat 10y agoSomething like this would be the key breakthrough. For all all their versatility the human eyes have not evolved to be (have not been designed for - depending on your viewpoint :) an optical interface for digital systems. Most of the hard work at the moment is in working around the "limitations" of our visual pipeline. Rudimentary technology that does just this has been around for a while as assistive devices for blind people [0] albeit only rudimentary shapes and movement can be portrayed. I believe ultimately the most effective technology will be that which integrates directly with the visual cortex, which I read somewhere has neurons arranged somewhat isomorphically with the visual field [1] I think this should be sufficient to stimulate both higher cognitive and the more reflexive automatic visual systems [2] though IANAN and really haven't look at this area in depth for a number of years. To get into the realms of pure science fiction about it [3], this could even be done by way of the pineal gland, which has some optical features that are largely unused. Perhaps it was put there like that for a reason ... [0] http://www.fightingblindness.ie/cure/retinal-implant-technology/ http://www.fightingblindness.ie/cure/retinal-implant-technol... [1] https://en.wikipedia.org/wiki/Retinotopy https://en.wikipedia.org/wiki/Retinotopy [2] https://en.wikipedia.org/wiki/Two-streams_hypothesis https://en.wikipedia.org/wiki/Two-streams_hypothesis [3] http://www.crystalinks.com/thirdeyepineal.html http://www.crystalinks.com/thirdeyepineal.html
- hathym 10y agoI imagine that in 20 years, they will invent something that plugs directly to the optical nerve or your brain to simulate virtual reality.
- deleted 10y ago[deleted]
- kriro 10y agoI still think interaction is the major shortcoming and probably will be for the foreseeable future. Most of this seems to be a technology-psychology hybrid. I want to grab a virtual coffee mug with sufficient feedback so I don't grab through it, feel the weight of it etc. Another part is finding good interaction paradigms which I suspect will fall out of better technology being available and in more hands (easy to prototype interactions). Wide spread first gen VR-Headsets and primitive interaction means (controller) will probably be good enough to drive that.
- vegabook 10y agoNvidia of course has a stock price incentive to pitch a very long period of further development ahead.
- LoSboccacc 10y agoso, what about the need of having a huge open space to appreciate virtual environments in freedom?
- milesokeefe 10y agoIt shouldn't need to be /too/ huge with smart redirected walking algorithms. But still probably too big for the average person to have available to themselves privately. This is where I feel most people in the VR space agree that "VR arcades" will become a lucrative industry.
- golemotron 10y agoI suspect that both VR immersion and self-driving cars will always be "just around the corner." We want to believe in them, but in both cases there are obstacles that may be insurmountable or will at least force us to scale back our hopes. In the former (VR) they are technical/biological and in the latter (self-driving cars) they are technical/environmental. Or visual systems are too tightly integrated with our motion systems and other sensory systems to be handled independently, just as traffic lanes are too exposed to other influences (unpredictable people and things) to allow vehicular autonomy. In both cases we want to see the thing we want to do by itself but there are deep interdependencies.
- milesokeefe 10y agoThe difference is that your brain fills in the gaps when there are very minor discrepancies between sensory systems. That's why presence exists with current VR platforms.
- Raphmedia 10y agoCan anyone paste me the list of shortcomings that VR has to solve according to him?
- milesokeefe 10y ago• VR displays are a little too cumbersome. [They have] to be much more elegant, being connected by a wire has to be solved. • The resolution has to be a lot higher. • The physical worlds do not behave according to the laws of physics. • The environment you’re in isn’t beautiful enough.
- Raphmedia 10y agoThat's what was in the article... I was expecting a longer list somewhere. Those are not game killers. If points like that were game killers, we wouldn't have had the Atari or the Nes.
- clarkrinker 10y agoJust got our Vive today. Only 1/3 people threw up.
- TrevorJ 10y agoIs there a video of this talk available?