7 ms·

This 3D Desktop Was Born at Microsoft

64 points by Libertatea 14y ago

p3d4nt 14y ago
Microsoft not hearing of Gorilla Arm - ironic. Wired grovelling - pass the sickbag.
arocks 14y ago
> People are used to gently flicking computer mice and grazing keyboards and tablet screens; do they really have the stamina to reach into their computers and flail their arms around? That's not the key question. Traditional interfaces have some sort of physical contact. Moving from physical keys to touch screen removed most of the tactile feedback. But still many phones simulate it via haptic vibrations. Imagine trying to grasp an object in the air purely out of visual feedback. I would imagine it to be extremely strenuous especially in 3D space. Unless there is a breakthrough sci-fi skin which can simulate this tactile feedback, we would be, quite literally, hand-waving in the air.
- quarterto 14y ago
  http://masseffect.wikia.com/wiki/Codex/Technology#Computers:_Haptic_Adaptive_Interface http://masseffect.wikia.com/wiki/Codex/Technology#Computers:...
  - ferongr 14y ago
    Mass Effect "lore" tries to justify graphical rule of cool (displays projected into thin air with a few shades of orange as their only colors etc.). I wouldn't take it seriously as something exploring the practicality of such things.
- hdra 14y ago
  I personally feel that the problem is not that of having a tactile feedback. While I don't think it can replace mouse/keyboard, I don't see too much of a problem of it replacing touchscreens. The problem with most of the existing gesture based interfaces is that they are interacting with the interface in an indirect way, as in the gestures are used control something like a pointer on the screen, and users have no way to see how they are positioned relative to the interface they are interacting with. If they are used in ways similar to a touch screens, I think it would be pretty natural, without too much hand-waving in the air.
- stephengillie 14y ago
  Star Trek touchscreens have the same problem.
- jpalomaki 14y ago
  Maybe it would be possible to fake the tactile feedback by using some visual aid? There have been experiments where people "feel" things that are happening to a fake hand ([1] and [2]). If you have an augmented reality setup maybe some visual effects applied to your hand would be translated into feelings. [1] http://bodyodd.nbcnews.com/_news/2011/02/24/6118219-need-a-hand-scientists-convince-people-they-have-3-arms?lite http://bodyodd.nbcnews.com/_news/2011/02/24/6118219-need-a-h... [2] http://www2.psych.ubc.ca/~ennslab/Vision_Lab/Publications_files/80_Austen_etal_CABN04.pdf http://www2.psych.ubc.ca/~ennslab/Vision_Lab/Publications_fi...
- vellum 14y ago
  There have been some experiments with using ultrasound to provide haptic feedback. http://gizmodo.com/5044624/ultrasound-haptic-devices-can-project-tactile-shapes-into-thin-air http://gizmodo.com/5044624/ultrasound-haptic-devices-can-pro...
podperson 14y ago
Amazing? Eh. Useful? Meh.
ebbv 14y ago
First I want to get off my chest; are we still using the phrase "computer genius" in 2013? Something about it just rings 1980's to me. I dunno. It seems quaint and silly. 3D interfaces may eventually be the future, but I don't think this version is it. First of all, it's gonna eat up your desktop space like crazy, for what advantage really? Wow factor? That wears out quickly. Which brings me to my next point; there has been technology to do a variety of different types of 3D desktops for a while now, but it hasn't been pursued because so far going 3D doesn't provide much in the way of practical advantages for most applications. If you're browsing the web and reading email, 3D is doing nothing for you except "Oh man isn't this cool?" The one place where it makes sense is where you are dealing with actual 3D objects, like 3D printing and CAD/CAM software.
- fuzzix 14y ago
  'First I want to get off my chest; are we still using the phrase "computer genius" in 2013? Something about it just rings 1980's to me. I dunno. It seems quaint and silly' Hey, at least they didn't say "whizz-kid".
  - wwwhizz 14y ago
    What's wrong with that? ;)
    - mediacrisis 14y ago
      Awkwaaaard.
  - alphadogg 14y ago
    Or worse: "Internet Tycoon"
- pyre 14y ago
  The interface (using cameras to track hand / eye movement) just reminds me of the interface described in Snow Crash. It's possible that this full solution won't be the future, but maybe it will parts of it will push the 3-D desktop forward.
simonholroyd 14y ago
I'm not sure why press, media, futurists insist on pushing the idea that the future of user interaction is via replicating 3-dimensional physical interaction. There's a reason the keyboard, mouse and (small) touch screen are so successful as input devices. They each create an huge degree of meaning with a high level of precision, for relatively tiny physical movements. While a 3-d interface that mimics reality might be more intuitive to a first-time user, it is vastly less efficient for an experienced user trying to get work done. A 3-d desktop is an accurate mental model but it doesn't need to exist on a screen if it exists in my head. I can move through the z axis of my desktop faster with alt-tab than I could ever move, shuffle, re-order a stack of documents with my hands in a 3-d environment. * EDIT: tv and movies (eg. minority report) probably have a clear incentive to favor input devices where the audience can more easily see what's happening. 3-d works better there.
- mixedbit 14y ago
  I agree and I think it is impossible to create an interface that is optimal for both inexperienced and power users. For users that are using a piece of software frequently, say few hours a day, intuitiveness of an interface should not matter, just efficiency. I'm often surprised when I see professional programmers turning on some fancy 3D destop effects. Sure, these look nice, but they are only distracting and counter productive when you are using an environment several hours a day. tl;dr xmonad rocks
  - chimeracoder 14y ago
    For power users, the word 'intuitive' doesn't even make sense. There's nothing 'intuitive' about hjkl, but when I tried switching from wmii to i3, the very first thing I did was switch their navigation keybindings back from jkl; to hjkl. Yes, the reasons for hjkl are historic and arbitrary, but I don't care - I don't want to relearn yet another set of muscle memories when I don't have to. Even vim on the whole is completely 'unintuitive'. That's the whole reason vimtutor exists. But, the couple of hours I've spent (combined) over the last few years learning how to use it has paid off in full and with dividends in terms of my productivity - and I hopefully have many more decades of life to recoup that investment many more times over. > I'm often surprised when I see professional programmers turning on some fancy 3D destop effects. Sure, these look nice, but they are only distracting and counter productive when you are using an environment several hours a day. I couldn't find these more annoying. Web interfaces tend to be the worst (because they're highly uncustomizeable). I don't want a slick, 3 second animation where the tab wiggles and slides every time I want to change the page (I'm looking at you, AmEx). I know where I want to go, and I just want to get there immediately. Every second that I'm delayed by flashy animations in something that I need to use several times a day just makes me despise the product a little bit more each time. Outside of very specific applications/domains (gaming, simulations, etc.), I don't want anything to replace my keyboard. As GP said, nothing (including the mouse) can beat the keyboard for allowing maximum control and precision with minimal movement.
JasonFruit 14y ago
So the writer's idea of progress runs: command line → WIMP → touch → 3D. I think at least the first three are in order of effectiveness — descending. I suspect that the fourth, at least by my understanding of this idea, will continue the trend.
- corin_ 14y ago
  Certainly there are areas where command line beats a GUI, and where keyboard/mouse beat touchscreen, but that doesn't mean they are less effective. Would computers be doing as much as they do for many, many people if they were still 100% command line?
  - JasonFruit 14y ago
    You know, I bet they would. I think what's made so many people able to use a computer is not the new modes of interaction but something more powerful: time. As people have more exposure to computers, and as they become more aware of the interface conventions that develop over time, they get better at using them no matter what the interface style is. I wonder if people might be more useful to people if they had preserved a higher ratio of critical to non-essential visual data — which I think terminal applications do.
yourcelf 14y ago
This idea goes back further than you might think. The "Stereoscopic Workspace", from the Architecture Machine group at MIT in 1983, was a prototype of this idea, surprisingly close to this implementation 30 years later. http://media.mit.edu/speech/videos/?video=Stereoscopic%20Workspace http://media.mit.edu/speech/videos/?video=Stereoscopic%20Wor... http://media.mit.edu/speech/papers/1983/schmandt_SIGGRAPH83_spatial_input-display_correspondence.pdf http://media.mit.edu/speech/papers/1983/schmandt_SIGGRAPH83_...
adventured 14y ago
The best 3D interface isn't going to look like a computer interface at all. It's going to be a small patch on your skull that lets you look at a link and think : click / go. Your eyes and focus will make your hands almost entirely pointless for using a computer. Within 10 to 20 years it'll be possible to wear said small patch, look at a url bar, think google.com, then think: go or enter, and it'll all just work. I think it'll be available in 10, but larger consumer adoption within 20. That same patch will enable true interfacing with your physical environment. Walk into your house and think kitchen lights on (plus a safe confirmation word or phrase to make sure you mean it), and they'll come on based on the associated phrase. By the time companies like Microsoft get around to trying to perfect a ridiculous hand > 3D interface, this concept will already have destroyed it. Minority Report interfaces also will never become mass consumer adopted because of this.
- abrichr 14y ago
  I appreciate your optimism, but I think perhaps you're underestimating the difficulty associated with decoding brain waves. Right now, the bleeding edge is just coming to grips with decoding signals in the somatic nervous system (the one that's responsible for voluntary control of skeletal muscles). I think we're still a long way off from being able to read the contents of thoughts from an EEG.
- kristopolous 14y ago
  > Minority Report interfaces also will never become mass consumer adopted because of this. it's actually been built, by the same guy that designed the thing for the movie, but many express reservations: http://oblong.com/ http://oblong.com/ There's also kinect and leap in this space in the "stuff I can buy today" column. So I wouldn't discount this as "never" quite yet.
  - georgemcbay 14y ago
    I don't think the point is that Minority Report type interfaces can't or won't be invented, but rather that they are more cool looking than they are practical for doing real work and thus won't generally be used outside of very niche areas.
leetrout 14y ago
Here's a link to a demo video http://vimeo.com/59231550 http://vimeo.com/59231550
j_s 14y ago
The effort put into backwards compatibility at Microsoft is always interesting... here is a presentation on research into supporting legacy 2D custom controls on arbitrary 3D surfaces: http://blogs.msdn.com/b/wpf3d/archive/2006/12/12/interacting-with-2d-on-3d-in-wpf.aspx http://blogs.msdn.com/b/wpf3d/archive/2006/12/12/interacting... video: http://channel9.msdn.com/Blogs/Charles/Daniel-Lehenbauer-and-Kurt-Berglund-Interactive-2D-controls-on-WPF-3D-Surfaces http://channel9.msdn.com/Blogs/Charles/Daniel-Lehenbauer-and...
dclowd9901 14y ago
Am I the only one here amazed that is seems to "know" through what perspective you're looking at it? How is it doing that?
ww520 14y ago
This actually is pretty amazing. It looks like it can become practical.

64 points by Libertatea 14y ago

Microsoft not hearing of Gorilla Arm - ironic. Wired grovelling - pass the sickbag.

> People are used to gently flicking computer mice and grazing keyboards and tablet screens; do they really have the stamina to reach into their computers and flail their arms around? That's not the key question. Traditional interfaces have some sort of physical contact. Moving from physical keys to touch screen removed most of the tactile feedback. But still many phones simulate it via haptic vibrations. Imagine trying to grasp an object in the air purely out of visual feedback. I would imagine it to be extremely strenuous especially in 3D space. Unless there is a breakthrough sci-fi skin which can simulate this tactile feedback, we would be, quite literally, hand-waving in the air.

quarterto 14y ago
http://masseffect.wikia.com/wiki/Codex/Technology#Computers:_Haptic_Adaptive_Interface http://masseffect.wikia.com/wiki/Codex/Technology#Computers:...
- ferongr 14y ago
  Mass Effect "lore" tries to justify graphical rule of cool (displays projected into thin air with a few shades of orange as their only colors etc.). I wouldn't take it seriously as something exploring the practicality of such things.
hdra 14y ago
I personally feel that the problem is not that of having a tactile feedback. While I don't think it can replace mouse/keyboard, I don't see too much of a problem of it replacing touchscreens. The problem with most of the existing gesture based interfaces is that they are interacting with the interface in an indirect way, as in the gestures are used control something like a pointer on the screen, and users have no way to see how they are positioned relative to the interface they are interacting with. If they are used in ways similar to a touch screens, I think it would be pretty natural, without too much hand-waving in the air.
stephengillie 14y ago
Star Trek touchscreens have the same problem.
jpalomaki 14y ago
Maybe it would be possible to fake the tactile feedback by using some visual aid? There have been experiments where people "feel" things that are happening to a fake hand ([1] and [2]). If you have an augmented reality setup maybe some visual effects applied to your hand would be translated into feelings. [1] http://bodyodd.nbcnews.com/_news/2011/02/24/6118219-need-a-hand-scientists-convince-people-they-have-3-arms?lite http://bodyodd.nbcnews.com/_news/2011/02/24/6118219-need-a-h... [2] http://www2.psych.ubc.ca/~ennslab/Vision_Lab/Publications_files/80_Austen_etal_CABN04.pdf http://www2.psych.ubc.ca/~ennslab/Vision_Lab/Publications_fi...
vellum 14y ago
There have been some experiments with using ultrasound to provide haptic feedback. http://gizmodo.com/5044624/ultrasound-haptic-devices-can-project-tactile-shapes-into-thin-air http://gizmodo.com/5044624/ultrasound-haptic-devices-can-pro...

podperson 14y ago

Amazing? Eh. Useful? Meh.

ebbv 14y ago

First I want to get off my chest; are we still using the phrase "computer genius" in 2013? Something about it just rings 1980's to me. I dunno. It seems quaint and silly. 3D interfaces may eventually be the future, but I don't think this version is it. First of all, it's gonna eat up your desktop space like crazy, for what advantage really? Wow factor? That wears out quickly. Which brings me to my next point; there has been technology to do a variety of different types of 3D desktops for a while now, but it hasn't been pursued because so far going 3D doesn't provide much in the way of practical advantages for most applications. If you're browsing the web and reading email, 3D is doing nothing for you except "Oh man isn't this cool?" The one place where it makes sense is where you are dealing with actual 3D objects, like 3D printing and CAD/CAM software.

fuzzix 14y ago
'First I want to get off my chest; are we still using the phrase "computer genius" in 2013? Something about it just rings 1980's to me. I dunno. It seems quaint and silly' Hey, at least they didn't say "whizz-kid".
- wwwhizz 14y ago
  What's wrong with that? ;)
  - mediacrisis 14y ago
    Awkwaaaard.
- alphadogg 14y ago
  Or worse: "Internet Tycoon"
pyre 14y ago
The interface (using cameras to track hand / eye movement) just reminds me of the interface described in Snow Crash. It's possible that this full solution won't be the future, but maybe it will parts of it will push the 3-D desktop forward.

simonholroyd 14y ago

I'm not sure why press, media, futurists insist on pushing the idea that the future of user interaction is via replicating 3-dimensional physical interaction. There's a reason the keyboard, mouse and (small) touch screen are so successful as input devices. They each create an huge degree of meaning with a high level of precision, for relatively tiny physical movements. While a 3-d interface that mimics reality might be more intuitive to a first-time user, it is vastly less efficient for an experienced user trying to get work done. A 3-d desktop is an accurate mental model but it doesn't need to exist on a screen if it exists in my head. I can move through the z axis of my desktop faster with alt-tab than I could ever move, shuffle, re-order a stack of documents with my hands in a 3-d environment. * EDIT: tv and movies (eg. minority report) probably have a clear incentive to favor input devices where the audience can more easily see what's happening. 3-d works better there.

mixedbit 14y ago
I agree and I think it is impossible to create an interface that is optimal for both inexperienced and power users. For users that are using a piece of software frequently, say few hours a day, intuitiveness of an interface should not matter, just efficiency. I'm often surprised when I see professional programmers turning on some fancy 3D destop effects. Sure, these look nice, but they are only distracting and counter productive when you are using an environment several hours a day. tl;dr xmonad rocks
- chimeracoder 14y ago
  For power users, the word 'intuitive' doesn't even make sense. There's nothing 'intuitive' about hjkl, but when I tried switching from wmii to i3, the very first thing I did was switch their navigation keybindings back from jkl; to hjkl. Yes, the reasons for hjkl are historic and arbitrary, but I don't care - I don't want to relearn yet another set of muscle memories when I don't have to. Even vim on the whole is completely 'unintuitive'. That's the whole reason vimtutor exists. But, the couple of hours I've spent (combined) over the last few years learning how to use it has paid off in full and with dividends in terms of my productivity - and I hopefully have many more decades of life to recoup that investment many more times over. > I'm often surprised when I see professional programmers turning on some fancy 3D destop effects. Sure, these look nice, but they are only distracting and counter productive when you are using an environment several hours a day. I couldn't find these more annoying. Web interfaces tend to be the worst (because they're highly uncustomizeable). I don't want a slick, 3 second animation where the tab wiggles and slides every time I want to change the page (I'm looking at you, AmEx). I know where I want to go, and I just want to get there immediately. Every second that I'm delayed by flashy animations in something that I need to use several times a day just makes me despise the product a little bit more each time. Outside of very specific applications/domains (gaming, simulations, etc.), I don't want anything to replace my keyboard. As GP said, nothing (including the mouse) can beat the keyboard for allowing maximum control and precision with minimal movement.

JasonFruit 14y ago

So the writer's idea of progress runs: command line → WIMP → touch → 3D. I think at least the first three are in order of effectiveness — descending. I suspect that the fourth, at least by my understanding of this idea, will continue the trend.

corin_ 14y ago
Certainly there are areas where command line beats a GUI, and where keyboard/mouse beat touchscreen, but that doesn't mean they are less effective. Would computers be doing as much as they do for many, many people if they were still 100% command line?
- JasonFruit 14y ago
  You know, I bet they would. I think what's made so many people able to use a computer is not the new modes of interaction but something more powerful: time. As people have more exposure to computers, and as they become more aware of the interface conventions that develop over time, they get better at using them no matter what the interface style is. I wonder if people might be more useful to people if they had preserved a higher ratio of critical to non-essential visual data — which I think terminal applications do.

yourcelf 14y ago

This idea goes back further than you might think. The "Stereoscopic Workspace", from the Architecture Machine group at MIT in 1983, was a prototype of this idea, surprisingly close to this implementation 30 years later. http://media.mit.edu/speech/videos/?video=Stereoscopic%20Workspace http://media.mit.edu/speech/videos/?video=Stereoscopic%20Wor... http://media.mit.edu/speech/papers/1983/schmandt_SIGGRAPH83_spatial_input-display_correspondence.pdf http://media.mit.edu/speech/papers/1983/schmandt_SIGGRAPH83_...

adventured 14y ago

The best 3D interface isn't going to look like a computer interface at all. It's going to be a small patch on your skull that lets you look at a link and think : click / go. Your eyes and focus will make your hands almost entirely pointless for using a computer. Within 10 to 20 years it'll be possible to wear said small patch, look at a url bar, think google.com, then think: go or enter, and it'll all just work. I think it'll be available in 10, but larger consumer adoption within 20. That same patch will enable true interfacing with your physical environment. Walk into your house and think kitchen lights on (plus a safe confirmation word or phrase to make sure you mean it), and they'll come on based on the associated phrase. By the time companies like Microsoft get around to trying to perfect a ridiculous hand > 3D interface, this concept will already have destroyed it. Minority Report interfaces also will never become mass consumer adopted because of this.

abrichr 14y ago
I appreciate your optimism, but I think perhaps you're underestimating the difficulty associated with decoding brain waves. Right now, the bleeding edge is just coming to grips with decoding signals in the somatic nervous system (the one that's responsible for voluntary control of skeletal muscles). I think we're still a long way off from being able to read the contents of thoughts from an EEG.
kristopolous 14y ago
> Minority Report interfaces also will never become mass consumer adopted because of this. it's actually been built, by the same guy that designed the thing for the movie, but many express reservations: http://oblong.com/ http://oblong.com/ There's also kinect and leap in this space in the "stuff I can buy today" column. So I wouldn't discount this as "never" quite yet.
- georgemcbay 14y ago
  I don't think the point is that Minority Report type interfaces can't or won't be invented, but rather that they are more cool looking than they are practical for doing real work and thus won't generally be used outside of very niche areas.

leetrout 14y ago

Here's a link to a demo video http://vimeo.com/59231550 http://vimeo.com/59231550

j_s 14y ago

The effort put into backwards compatibility at Microsoft is always interesting... here is a presentation on research into supporting legacy 2D custom controls on arbitrary 3D surfaces: http://blogs.msdn.com/b/wpf3d/archive/2006/12/12/interacting-with-2d-on-3d-in-wpf.aspx http://blogs.msdn.com/b/wpf3d/archive/2006/12/12/interacting... video: http://channel9.msdn.com/Blogs/Charles/Daniel-Lehenbauer-and-Kurt-Berglund-Interactive-2D-controls-on-WPF-3D-Surfaces http://channel9.msdn.com/Blogs/Charles/Daniel-Lehenbauer-and...

dclowd9901 14y ago

Am I the only one here amazed that is seems to "know" through what perspective you're looking at it? How is it doing that?

ww520 14y ago

This actually is pretty amazing. It looks like it can become practical.