8 ms·

Minority Report Interface Is Real, Hitting Mainstream Soon

35 points by jamesbritt 16y ago

petervandijck 16y ago
Oh the one where you have to wear funky gloves and wave your arms around a lot? The wide arm movements are a dead give-away. It's MADE FOR MOVIES. Not for actually using it.
- hugh3 16y ago
  Oh, I don't know. When I'm using my computer I often think "Man, I wish I were standing up and making huge arm movements right now so that my arms would get tired within twenty minutes" I'm tempted to say "At least it would be good for games", but frankly the Wii has failed to revolutionize gaming as much as folks thought it would -- after waving your arms around for a bit the novelty wears off and you go back to wanting to mash buttons.
  - jamesbritt 16y ago
    " ...after waving your arms around for a bit the novelty wears off and you go back to wanting to mash buttons." Not me. Button-mashing is tedious. I like standing up, pointing and waving and clicking and twirling. It's engrossing when done well; for example, Wii bowling or Wii table tennis. Or any game with a sword fight. :)
  - wtallis 16y ago
    It sure seems to me that the Wii has revolutionized gaming as a mainstream social activity, even if it hasn't really affected "hardcore" gaming.
    - hugh3 16y ago
      Has it, though? Or was it a brief fad which lasted until everybody got bored of Wii tennis?
- Semiapies 16y ago
  It actually kinda made sense as a system meant to be used quickly, but only for brief periods of time, by completely able-bodied people. For any extended period time, or for anyone with even minor physical disabilities, it's wretched.
  - hugh3 16y ago
    Even for that application, did it have any advantage over, say, a mouse and a keyboard and a couple of 30-inch screens?
    - Semiapies 16y ago
      It looked faster, but impossible to say. Probably easier to teach new cops on the squad.
- mortenjorck 16y ago
  I do think G-Speak has potential applications in dealing specifically with complex three-dimensional data. It's not going to replace what we use every day on the desktop, because most of what we do on the desktop works in two dimensions, but for actually manipulating visual interfaces that are, of necessity, presented in three dimensions, there's nothing like actually moving in 3D. If common desktop applications do, at some point, evolve to meaningfully take advantage of 3D visual interfaces, it could trickle down to a system that assumes a more natural posture (try resting your arms on your desk and gesturing above your keyboard: I think that could work). We've had hardware-accelerated 3D on the desktop for a decade, though, and have yet to see anything substantial come of it for traditionally two-dimensional UIs apart from sometimes-helpful transitional animations. Things could change, but I think applying gestures and multi-touch to 2D (a la 10/GUI, naturally) shows the most promising way forward.
- dpritchett 16y ago
  The LuminAR project (also at MIT Media Lab) [1] is much more down to earth. It projects onto a 2d surface and then you interact with the surface via gestures with your bare hands. [1] http://news.ycombinator.com/item?id=1820748 http://news.ycombinator.com/item?id=1820748
- sprout 16y ago
  I especially like how moving a video from one monitor to another requires moving a bulky plastic (glass?) plate from one terminal to another, a plate sufficiently large to hold an entire computer, touchscreen, and information storage for a hundred videos.
- levesque 16y ago
  You don't necessarily have to wave your arms around. I don't know how the g-speak works, but if it uses the orientation of the hand to do raycasting (as it should), you can leave your hand comfortable by your side and not be bothered by having to raise your arms at shoulder level. The real pain comes when you try to apply 3D interfaces like g-speak to problems that are not inherently 3D. You get awkward interaction and a lot of wasted time. Real applications of 3DUIs include medical operation planning, virtual reality, scientific data visualization and architecture, just to name a few. Also, in most cases, gestural interfaces are not the most efficient way to do it.
  - warfangle 16y ago
    I could see this kind of interface being extremely useful for folks who do any sort of 3D art. Being able to rotate and manipulate without arcane keystrokes would be a boon for quickly sculpting 3D objects.
- crs 16y ago
  Actually there are more ways to use the system than what you may traditionally think of when using a computer. For example, Boeing has patents related to controlling a swarm of UAV's via a g-speak based system. There are more applications where that came from but this was the only example that is public. Its collaborative features are a huge draw. Would you ever use photoshop via gestures, probably not, but you would use it for command and control systems with more than one concurrent operator. I recommend that you go to oblongs website and watch some of the demo videos. A computers interface can be anything from manipulating a physical object on a table to "wearing funky gloves and waving your arms around". Its not limited to just a keyboard, mouse or touch pad.
- nsfmc 16y ago
  So, i had the opportunity to use gspeak for the better part of a year at risd and what's probably not explicit from looking at demo videos is that you actually don't need to move around that much. The system is remarkably good at tracking motion and (more importantly) projecting your finger/gestures onto a much larger surface. Stop and think about that last point. If you're five or ten feet from a wall in your office and say you place a vector that runs perpendicular to your palm. It takes a surprisingly small amount of hand movement to get that vector to hit any portion of the wall. By the time i had gotten used to the system, i could use it usually by performing completely minimal gestures so much so that people didn't even realize i was engaging it explicitly. This is similar to how you can see people playing wii tennis on their couch without standing up. [1] If you read that sculley interview[2] you hear him talk about commoditizing the performance to create consumer products that react the way you expect them to. For me, gspeak convinced me that this was basically the case with a gestural interface, the hard part was figuring out something to do with it. I suspect this is a hard issue for many people because the majority of the complaints i see are basically "so what, i can't see myself using it" and fair enough, answering that question is really hard. But the underlying problem of how to provide that interface isn't your problem anymore, just like you don't have to figure out how to track touches or mouse movement anymore, you just build on top of that as a given. that seems like a net win to me. A large part of gspeak for end users and designers ends up being the development of a descriptive and non-conflicting gestural language for the task at hand. The folks at usc have a good example of what i mean [3]. Say what you will about wild gesticulating or how lame it seems, the system (as any other) is only as cool as what you do with it. [1]: http://www.penny-arcade.com/comic/2006/11/13/ http://www.penny-arcade.com/comic/2006/11/13/ [2]: http://www.cultofmac.com/john-sculley-on-steve-jobs-the-full-interview-transcript/63295 http://www.cultofmac.com/john-sculley-on-steve-jobs-the-full... [3]: http://www.vimeo.com/8083372 http://www.vimeo.com/8083372
wildmXranat 16y ago
So, did this have any movement since ... February ?
- jamesbritt 16y ago
  Um, yes, and this might have been the better link to submit: http://singularityhub.com/2010/10/22/new-video-of-mits-minority-report-interface-g-speak/ http://singularityhub.com/2010/10/22/new-video-of-mits-minor...
frankdenbow 16y ago
What is equally interesting is what can be done by scanning neural activity and essentially letting people control games/physical objects without any physical input. Check out the Emotiv headset for an example http://www.emotiv.com http://www.emotiv.com. Their headset is $300 and is shipping now.
Groxx 16y ago
The second comment covers the interface quite nicely, by linking to this: http://okcancel.com/comic/3.html http://okcancel.com/comic/3.html
ThomPete 16y ago
Here is the thing. There is currently no advantages to Minority Report interfaces that isn't better done by the computer itself. A Minority Report type interface requires algorithms that multiply your actions way more than any system currently available. It's kind of like doing a combo in Tekken or some other figthing game. It's valuable if you can control real time with additional powers but we are not even close to getting there. The kind of advantages that 3D interfaces gives cannot yet be harvested in 99% of the cases. Perhaps controlling a spaceship or a satellite but in most cases it's not better it's just different.
swah 16y ago
And I can't even find a good wireless mouse.
swah 16y ago
This reminded me of Andy Wingo (http://wingolog.org http://wingolog.org), a Guile hacker who works with/for Underkoffler (Oblong).
bokonist 16y ago
I'm surprised the article wasn't talking about the Microsoft Kinect interface. Kinect is hitting the mass market in a month and the UI is completely navigable just by waving your arms. It doesn't have enough precision to do something like move files around, but it's still really cool.
dshankar 16y ago
Define "mainstream." This gives the false idea that such an interface would take over the way humans normally compute and interact with machines. This interface is nowhere close to the level of replacing the traditional keyboard/mouse or the emerging multitouch tablet. People simply cannot interact with textual data quickly enough and would easily get tired with that interface. I can code at my keyboard/mouse for 10-30 hours straight without much fatigue. I wouldn't last 30 minutes on that thing. Nevertheless, it's really cool and I can't wait to see future interface designs. Just don't make the lofty claim that it will be "mainstream." (just a personal opinion)
- hugh3 16y ago
  You can code for 30 hours straight without much fatigue?
  - dshankar 16y ago
    Coffee + 10 minute breaks every 20 minutes of code. Weekly/monthly hackathons help condition your body for this ;-) I obviously crash immediately after. I'm usually delirious and black-out (after TechCrunch Disrupt for example).
zoomzoom 16y ago
Apple's strategic purchase?
corysama 16y ago
Nintendo's PowerGlove taught me an important lesson 20 years ago: Even if hand tracking becomes perfect, it doesn't matter. What matters is that holding your arms up is exhausting. After a minute, you are resting your elbow on something. After two, you are resting your wrist on something. A minute later, you really wish you had a mouse.
bradleyland 16y ago
This same series of videos and content has been making the rounds on new aggregator websites for quite some time now. One of the videos was published over two years ago: http://vimeo.com/2229299 http://vimeo.com/2229299 This tech is cool, but it's not going anywhere fast because its use isn't practical.
konad 16y ago
http://www.catb.org/jargon/html/G/gorilla-arm.html http://www.catb.org/jargon/html/G/gorilla-arm.html The side-effect that destroyed touch-screens as a mainstream input technology despite a promising start in the early 1980s.

35 points by jamesbritt 16y ago

Oh the one where you have to wear funky gloves and wave your arms around a lot? The wide arm movements are a dead give-away. It's MADE FOR MOVIES. Not for actually using it.

hugh3 16y ago
Oh, I don't know. When I'm using my computer I often think "Man, I wish I were standing up and making huge arm movements right now so that my arms would get tired within twenty minutes" I'm tempted to say "At least it would be good for games", but frankly the Wii has failed to revolutionize gaming as much as folks thought it would -- after waving your arms around for a bit the novelty wears off and you go back to wanting to mash buttons.
- jamesbritt 16y ago
  " ...after waving your arms around for a bit the novelty wears off and you go back to wanting to mash buttons." Not me. Button-mashing is tedious. I like standing up, pointing and waving and clicking and twirling. It's engrossing when done well; for example, Wii bowling or Wii table tennis. Or any game with a sword fight. :)
- wtallis 16y ago
  It sure seems to me that the Wii has revolutionized gaming as a mainstream social activity, even if it hasn't really affected "hardcore" gaming.
  - hugh3 16y ago
    Has it, though? Or was it a brief fad which lasted until everybody got bored of Wii tennis?
Semiapies 16y ago
It actually kinda made sense as a system meant to be used quickly, but only for brief periods of time, by completely able-bodied people. For any extended period time, or for anyone with even minor physical disabilities, it's wretched.
- hugh3 16y ago
  Even for that application, did it have any advantage over, say, a mouse and a keyboard and a couple of 30-inch screens?
  - Semiapies 16y ago
    It looked faster, but impossible to say. Probably easier to teach new cops on the squad.
mortenjorck 16y ago
I do think G-Speak has potential applications in dealing specifically with complex three-dimensional data. It's not going to replace what we use every day on the desktop, because most of what we do on the desktop works in two dimensions, but for actually manipulating visual interfaces that are, of necessity, presented in three dimensions, there's nothing like actually moving in 3D. If common desktop applications do, at some point, evolve to meaningfully take advantage of 3D visual interfaces, it could trickle down to a system that assumes a more natural posture (try resting your arms on your desk and gesturing above your keyboard: I think that could work). We've had hardware-accelerated 3D on the desktop for a decade, though, and have yet to see anything substantial come of it for traditionally two-dimensional UIs apart from sometimes-helpful transitional animations. Things could change, but I think applying gestures and multi-touch to 2D (a la 10/GUI, naturally) shows the most promising way forward.
dpritchett 16y ago
The LuminAR project (also at MIT Media Lab) [1] is much more down to earth. It projects onto a 2d surface and then you interact with the surface via gestures with your bare hands. [1] http://news.ycombinator.com/item?id=1820748 http://news.ycombinator.com/item?id=1820748
sprout 16y ago
I especially like how moving a video from one monitor to another requires moving a bulky plastic (glass?) plate from one terminal to another, a plate sufficiently large to hold an entire computer, touchscreen, and information storage for a hundred videos.
levesque 16y ago
You don't necessarily have to wave your arms around. I don't know how the g-speak works, but if it uses the orientation of the hand to do raycasting (as it should), you can leave your hand comfortable by your side and not be bothered by having to raise your arms at shoulder level. The real pain comes when you try to apply 3D interfaces like g-speak to problems that are not inherently 3D. You get awkward interaction and a lot of wasted time. Real applications of 3DUIs include medical operation planning, virtual reality, scientific data visualization and architecture, just to name a few. Also, in most cases, gestural interfaces are not the most efficient way to do it.
- warfangle 16y ago
  I could see this kind of interface being extremely useful for folks who do any sort of 3D art. Being able to rotate and manipulate without arcane keystrokes would be a boon for quickly sculpting 3D objects.
crs 16y ago
Actually there are more ways to use the system than what you may traditionally think of when using a computer. For example, Boeing has patents related to controlling a swarm of UAV's via a g-speak based system. There are more applications where that came from but this was the only example that is public. Its collaborative features are a huge draw. Would you ever use photoshop via gestures, probably not, but you would use it for command and control systems with more than one concurrent operator. I recommend that you go to oblongs website and watch some of the demo videos. A computers interface can be anything from manipulating a physical object on a table to "wearing funky gloves and waving your arms around". Its not limited to just a keyboard, mouse or touch pad.
nsfmc 16y ago
So, i had the opportunity to use gspeak for the better part of a year at risd and what's probably not explicit from looking at demo videos is that you actually don't need to move around that much. The system is remarkably good at tracking motion and (more importantly) projecting your finger/gestures onto a much larger surface. Stop and think about that last point. If you're five or ten feet from a wall in your office and say you place a vector that runs perpendicular to your palm. It takes a surprisingly small amount of hand movement to get that vector to hit any portion of the wall. By the time i had gotten used to the system, i could use it usually by performing completely minimal gestures so much so that people didn't even realize i was engaging it explicitly. This is similar to how you can see people playing wii tennis on their couch without standing up. [1] If you read that sculley interview[2] you hear him talk about commoditizing the performance to create consumer products that react the way you expect them to. For me, gspeak convinced me that this was basically the case with a gestural interface, the hard part was figuring out something to do with it. I suspect this is a hard issue for many people because the majority of the complaints i see are basically "so what, i can't see myself using it" and fair enough, answering that question is really hard. But the underlying problem of how to provide that interface isn't your problem anymore, just like you don't have to figure out how to track touches or mouse movement anymore, you just build on top of that as a given. that seems like a net win to me. A large part of gspeak for end users and designers ends up being the development of a descriptive and non-conflicting gestural language for the task at hand. The folks at usc have a good example of what i mean [3]. Say what you will about wild gesticulating or how lame it seems, the system (as any other) is only as cool as what you do with it. [1]: http://www.penny-arcade.com/comic/2006/11/13/ http://www.penny-arcade.com/comic/2006/11/13/ [2]: http://www.cultofmac.com/john-sculley-on-steve-jobs-the-full-interview-transcript/63295 http://www.cultofmac.com/john-sculley-on-steve-jobs-the-full... [3]: http://www.vimeo.com/8083372 http://www.vimeo.com/8083372

wildmXranat 16y ago

So, did this have any movement since ... February ?

jamesbritt 16y ago
Um, yes, and this might have been the better link to submit: http://singularityhub.com/2010/10/22/new-video-of-mits-minority-report-interface-g-speak/ http://singularityhub.com/2010/10/22/new-video-of-mits-minor...

frankdenbow 16y ago

What is equally interesting is what can be done by scanning neural activity and essentially letting people control games/physical objects without any physical input. Check out the Emotiv headset for an example http://www.emotiv.com http://www.emotiv.com. Their headset is $300 and is shipping now.

Groxx 16y ago

The second comment covers the interface quite nicely, by linking to this: http://okcancel.com/comic/3.html http://okcancel.com/comic/3.html

ThomPete 16y ago

Here is the thing. There is currently no advantages to Minority Report interfaces that isn't better done by the computer itself. A Minority Report type interface requires algorithms that multiply your actions way more than any system currently available. It's kind of like doing a combo in Tekken or some other figthing game. It's valuable if you can control real time with additional powers but we are not even close to getting there. The kind of advantages that 3D interfaces gives cannot yet be harvested in 99% of the cases. Perhaps controlling a spaceship or a satellite but in most cases it's not better it's just different.

swah 16y ago

And I can't even find a good wireless mouse.

swah 16y ago

This reminded me of Andy Wingo (http://wingolog.org http://wingolog.org), a Guile hacker who works with/for Underkoffler (Oblong).

bokonist 16y ago

I'm surprised the article wasn't talking about the Microsoft Kinect interface. Kinect is hitting the mass market in a month and the UI is completely navigable just by waving your arms. It doesn't have enough precision to do something like move files around, but it's still really cool.

dshankar 16y ago

Define "mainstream." This gives the false idea that such an interface would take over the way humans normally compute and interact with machines. This interface is nowhere close to the level of replacing the traditional keyboard/mouse or the emerging multitouch tablet. People simply cannot interact with textual data quickly enough and would easily get tired with that interface. I can code at my keyboard/mouse for 10-30 hours straight without much fatigue. I wouldn't last 30 minutes on that thing. Nevertheless, it's really cool and I can't wait to see future interface designs. Just don't make the lofty claim that it will be "mainstream." (just a personal opinion)

hugh3 16y ago
You can code for 30 hours straight without much fatigue?
- dshankar 16y ago
  Coffee + 10 minute breaks every 20 minutes of code. Weekly/monthly hackathons help condition your body for this ;-) I obviously crash immediately after. I'm usually delirious and black-out (after TechCrunch Disrupt for example).

zoomzoom 16y ago

Apple's strategic purchase?

corysama 16y ago

Nintendo's PowerGlove taught me an important lesson 20 years ago: Even if hand tracking becomes perfect, it doesn't matter. What matters is that holding your arms up is exhausting. After a minute, you are resting your elbow on something. After two, you are resting your wrist on something. A minute later, you really wish you had a mouse.

bradleyland 16y ago

This same series of videos and content has been making the rounds on new aggregator websites for quite some time now. One of the videos was published over two years ago: http://vimeo.com/2229299 http://vimeo.com/2229299 This tech is cool, but it's not going anywhere fast because its use isn't practical.

konad 16y ago

http://www.catb.org/jargon/html/G/gorilla-arm.html http://www.catb.org/jargon/html/G/gorilla-arm.html The side-effect that destroyed touch-screens as a mainstream input technology despite a promising start in the early 1980s.