13 ms·
HoloLens secret sauce: A 28nm customized 24-core DSP engine built by TSMC
- eganist 10y agoLay speculation here: If the first run is on a 28nm process, does this suggest the second generation on e.g. Intel's 14nm process might yield a drastically more powerful HoloLens model in the current form factor, or at least a more compact one capable of all the same (assuming the optics were also compacted) for revision 2? There's a whole ton of stuff I'm not taking into account such as the physical size of the current HPU, but my guess is that this is the largest point of improvement just going by the process alone.
- dcw303 10y agoWould they want to compact the optics though? I have absolutely no knowledge of the physics involved, but it looks like the projection style that is used has a very limited field of view that won't be resolved with more processing power. I don't see how shrinking the optics could help this. Because of this, it looks like future immersive improvements will be restricted, but I would be very happy to have it explained to me why I am wrong :) Edit: my pessimism around FOV was from reading this: http://doc-ok.org/?p=1274 http://doc-ok.org/?p=1274
- Eridrus 10y agoThe main thing people complain about in the HoloLens is the field of view; I don't know how much these chips are the bottleneck for that, but it seems like a problem you can throw more cores at, so I'm sort of expecting them to solve it that way, but I don't know a whole lot about hardware.
- hackcasual 10y agothat's almost certainly a limitation of the projector
- jimminy 10y agoI haven't tried them so I'm going by the models and demo's I've seen. I think the field of view issue is a physical limitation, not software, and only slightly hardware related. The peripheral vision is blocked out by the components. To achieve a wider field of view you have to shift components further back out of viewing range. and then modify the screens to wrap around in the new viewing frame. I don't think this has any impact of field of view for that reason.
- markingram 10y agoI believe it's not the cpu or gpu limiting the field of view. if you read the patents behind the waveguides, the real limitation seems to be in the waveguides. They need to make them thin enough for light to pass through, while at the same time refract light into you eyes at angles that don't result in major image distortions.
- sbierwagen 10y agoe.g. Intel's 14nm process Intel doesn't let anyone else use their fabs. TSMC has their own fabs, and their own fab tech. 14nm is outdated, by the way. TSMC says they'll start shipping 10nm parts by the end of the year: http://en.ctimes.com.tw/DispNews.asp?O=HJZ4GC65UYSSAA00NW http://en.ctimes.com.tw/DispNews.asp?O=HJZ4GC65UYSSAA00NW
- mappu 10y ago> Intel doesn't let anyone else use their fabs. I don't think this is true any more.
- sbierwagen 10y agoYou're talking about the Altera deal a while ago? Intel bought them last year.
- eximius 10y agoDid they just buy a license for ARM or announce they'd let people use their fabs for ARM?
- Eridrus 10y agoWhat about http://www.intel.com/content/www/us/en/foundry/overview.html http://www.intel.com/content/www/us/en/foundry/overview.html ?
- HammadB 10y agoYeah Intel Foundry has been alive and chuggin for a while now, fabbing chips for others.
- kyrra 10y agoThat changed a week ago (Intel making chips for others). http://www.recode.net/2016/8/16/12507216/lg-chip-manufacture-korea-intel http://www.recode.net/2016/8/16/12507216/lg-chip-manufacture... Also, it's important to understand that chip process measurement is not standard across the industry. 14nm has different measurements across different components across the vendors. See: http://www.extremetech.com/computing/221532-tsmc-will-begin-10nm-production-this-year-claims-5nm-by-2020 http://www.extremetech.com/computing/221532-tsmc-will-begin-...
- mappu 10y agoDepending on what format the IP was provided in, and the level at which Microsoft made their customization, it's not necessarily possible to just transplant it onto a newer process node.
- typon 10y agoNo. In fact pure performance will very likely be lower on the 14nm process. They might get better power/performance, but I would be willing to be that they won't even achieve that. The only thing guaranteed is higher density aka more of those Tensilica cores.
- Meegul 10y agoWhy would that be the case? 14nm FinFet designs tend to have a higher clockspeed (at least at the lower TDPs) while still consuming less power.
- BooneJS 10y agoPure speculation: 28nm masks and manufacturing are significantly cheaper than 16FF and newer, which likely helps meet the cost of HoloLens at the volume they're forecasting. Moving to 16ff or Intel's 14nm allows more for processing cores at a fixed die size, but also makes it more expensive. There's a reason it's called the bleeding edge. :)
- mud_dauber 10y agoCorrect. My co backed off 16FF in favor of 28nm for exactly this reason.
- john_reel 10y agoWhat is your company and product? I’m just curious.
- AndrewKemendo 10y agoHolographic Processing Unit (HPU) chip used in its virtual reality HoloLens specs It seems to be insurmountably hard for press to understand that virtual reality and augmented reality are related but distinct concepts. Beyond that, interesting teardown. The 10w number is great, but my guess is a lot of the high power processing happens in the sensor suite because it's using 4 repurposed IR sensor receiver combos to relay depth data in a highly structured way. That means this processor is the glue between the IMU and RGBD camera combo. I think in the end this can't scale down to consumer side with this approach, not to mention the other hindrances to scaling down with the protection system.
- brador 10y agoIsn't VR just AR with the camera covered?
- sitharus 10y agoNot really. VR is an entire immersive environment to itself, it doesn't have to interact with reality. AR has to blend in to the environment around, so it's a different class of problem. It has to locate points in the scene to add things to and project in a believable manner.
- angersock 10y agoTurn off the lights. It's VR.
- sitharus 10y agoNot really, turn out the lights and you'll stop seeing AR at all. VR will, for example, create a room with a table and put a chess game on that table. This all happens within the VR author's control and designs. AR has to analyse the scene and find the table, find the angle the table is at relative to the eyes, compute the position and perspective of the table and the chess board to look correct on the table. Both have the same end result - images projected to a user's eyes - but how they get there is quite different.
- oaf357 10y agoOnly 10W for all that? While I'm a little sceptical that is an impressive number.
- Cyph0n 10y agoThey should thank Cadence for the IP cores I guess. On a similar note, an RPi model B consumes ~5W, which is damn good for a full-fledged computer.
- bigiain 10y agoYeah but - 10W all getting radiated inside a single 12x12mm BGA package? Isn't that a lot? It sounds to me like that'd be running very hot... I had a project with a Raspberry Pi2 which was only drawing ~2.5W all up (including the wifi dongle), and it got flaky when the sun shone on the case and got it a bit warm.
- aidenn0 10y ago10W is probably on the high-end for passive cooling of such a package, but zero problem with a fan.
- imeron 10y agoFor reference: an Intel Atom (e.g. Braswell) CPU has a TDP of 6W and a pretty large passive heatsink.
- rasz_pl 10y agoalso for reference ~10W is Pentium ~100MHz territory. Afair 133MHz was the first one that required fan on top of the heatsink.
- lacksconfidence 10y agoFwiw my 486dx2-66 had a fan, and when that fan died we noticed because of corrupted memory crashing the system. Compared to modern CPU coolers it was downright tiny, but required.
- velox_io 10y agoThis is a GPU rather the a DSP (the tricky part is rendering not filtering). Core count is a bit of a pointless metric unless you know how big/ powerful a core is. I like the idea of a self contained VR headset with no wires, but I don't think going to see on for ab a decade, there's just too much processing power needed for a realistic experience. I hope I'm wrong! Edit: It appears this is in fact DSP (digital signal processor). That's a huge amount of power for dedicated signal processing, I'm intrigued to know what can be done with it.
- sbierwagen 10y agoThis is a GPU rather the a DSP No? It's a bunch of Tensilica DSP cores on a chip, for image processing stuff. It doesn't do any rendering: http://ip.cadence.com/vision http://ip.cadence.com/vision The Cherry Trail SoC has the GPU hardware.
- unexistance 10y agothe 'too much' will come in 18 months :D
- ulber 10y agoI think you're wrong about this being a GPU. The first bullet point in the slide tilted "HPU: Architecture" says "Sensor aggregator with environment and gesture processing". I don't see any indication of the HPU doing rendering tasks.
- jng 10y agoI'd guess they do the rendering on the DSPs, they're not GPUs but they'd be a lot better at that than an Atom CPU...
- T-A 10y ago> self contained VR headset with no wires http://www.theverge.com/2016/8/16/12503948/intel-project-allow-vr-headset-cordless-mixed-reality http://www.theverge.com/2016/8/16/12503948/intel-project-all...
- rasz_pl 10y ago
- cloudjacker 10y agoI love to see when vendors actually modify a chip instead of using just off the shelf parts. But even then, low bar....
- ioquatix 10y agoTLDR; It's running Windows 10 :)
- ericseppanen 10y agoFor those who haven't had the pleasure: developing on Tensilica Xtensa cores generally means living within 128-256KB of directly-accessible memory; a windowed register file that makes writing your own exception handlers "interesting"; a 6-year-old GCC bolted to a proprietary backend; per-seat licensing fees to use the compiler; and a corporate owner that's only halfway interested in the ecosystem they now control. So yeah, kind of wishing it would just die and let ARM take over the embedded space.
- ashitlerferad 10y agoThat sounds like a violation of the GPL or at least some sort of trickery.
- aylons 10y ago> a 6-year-old GCC bolted to a proprietary backend Not only a violation of the GPL, but for a code owned by the FSF and even Stallman himself. That's a bold move. And very douchey.
- ericseppanen 10y agoFrom what I can tell, they have structured it in a way that it's a violation in spirit, but not in letter. They don't link to GCC code directly; instead they output the intermediate representation and execute a standalone binary to finish processing the IR. They offer source code for all their GCC modifications, but carefully work around the GPL to accomplish the same thing.
- frozenport 10y agoYeah they tried to obfuscate their internals precisely so this wouldn't happen. Notice how it didn't stop anybody, but instead led to poor user experience.
- the_mitsuhiko 10y agoTypically those are creative workarounds for the GPL not actual violations.
- markingram 10y agoHoloLens 3D miniMap of House with Synced Ladies +1 https://www.youtube.com/watch?v=T-JvTZjbwNs https://www.youtube.com/watch?v=T-JvTZjbwNs GG:)
- rblatz 10y agoInteresting how he walked around the 3D characters instead of taking the easy way though them.
- a_t48 10y agoIt's kind of unpleasant to have your head stuck into a virtual object. People generally avoid objects in VR, too.
- markingram 10y agoyes, because walking through them sometimes cause minor turbulence :)
- rasz_pl 10y agoThat cracked me up. Is that blue circle a gaze tracker? because it keeps pointing at virtual boobs whenever he looks at the models :)) not to mention sending avatar to the kitchen, priceless :D
- shiftpgdn 10y agoIt is! I feel like gaze tracking is one thing that's missing from VR headsets that's in the hololens. Social VR won't work until your eye movement and facial expressions can be translated into the application.
- monk_e_boy 10y agoI'm sure gaze tracking would make rendering more efficient, just render the stuff the user is looking at in high def, the rest in low polygon and low rez textures.
- frik 10y agoSeveral independent review article about Hololens mention the limited view (relatively small view angle), underwhelming and off putting. (You can immediately spot a sponsored article if it doesn't even mention that issue at all.) iPhone 6 and high end Android smartphones already have augment reality apps with SLAM technology that is far more impressive than what MS PR is suggestion Hololens might one day be able to offer - the E3 presentation in 2015 and 2016 were faked as we know in the meantime.
- woah 10y agoHave you even tried one? The small field of view is disappointing, but only because what is in it is so impressive. I'm not sure what you're talking about with the smartphones, but I'm pretty sure they can't show objects in a room with natural looking distances and depth of field like the Hololens can.
- josephpmay 10y agoThe Hololens dose not have an accommodative display (natural looking depth of field). I believe their display is collimated, so that everything is always in focus.
- monk_e_boy 10y agoThey have gaze tracking so this should be fixable.
- josephpmay 10y agogaze tracking is not eye tracking
- blahi 10y agoYeah. That [1] is like pokemon go. Or faked. [1] https://www.youtube.com/watch?v=T-JvTZjbwNs https://www.youtube.com/watch?v=T-JvTZjbwNs posted elsewhere in the comments.
- 10y ago
- ex3ndr 10y agoWell this explains a lot. In my university (Russia, Mathematic-Mechanic faculty of SPBu) there are a lot of investments in computer vision stuff and it it almost impossible to build fast and with low TDP software for CV without SDP/FPGA/etc stuff.
- markingram 10y agoThis article gave a more in-depth look at optics (waveguides) and other parts of Hololens: http://www.tomshardware.com/news/microsoft-hololens-components-hpu-28nm,32546.html http://www.tomshardware.com/news/microsoft-hololens-componen...