5 ms·

Genius hack by PhD candidate (with source code): Turn any monitor into a 3D display using a Wiimote

142 points by thorax 19y ago

ArcticCelt 19y ago
I sincerely feel like standing up and giving a long standing ovation. This is the future of gaming, at lest for 3D shooters, nothing less.
- raju 19y ago
  I was close to tears... Sniff...
- greendestiny 19y ago
  I'm really not sure about that, and for the sake of all the other VR and AR researchers who have gone before Johnny Lee - he really didn't invent rendering based on head tracking. The reason I'm not sure about this tech for 3D shooters is that its like looking through a virtual window, you can't fully reorient yourself in 3D - I'm not sure what combining this and full mouse-look control would be like. Also on a PC based setup where you're close to the screen the lack of stereoscopic vision would be more obvious. I think its more likely the future of different sort of game - maybe those old cabal style scrolling shooters for instance. Here's a video of some guys doing the same thing with face tracking (ie no LEDs needed) - http://www.youtube.com/watch?v=iUdZk4KUDTI http://www.youtube.com/watch?v=iUdZk4KUDTI
  - danteembermage 19y ago
    My idea... an old west shoot out. You're in the saloon, peering out the window with a rifle. You can duck out of sight behind the television just like you could with a real window. You could also limit your exposure to different baddies by changing you angle of attack. The directional pad or keyboard could switch between windows, doors, newly created holes, etc. It's not immediately obvious how this will revolutionize all shooters, but for the shooting gallery type this should be included in all of them starting now.
  - BrandonM 19y ago
    Make those IR-LED glasses into stereoscopic vision glasses and the entire 3D effect is realized.
amichail 19y ago
Is there a research contribution here?
- greendestiny 19y ago
  I guess taking this kind of head tracking to the wider world is useful. Tracking with cameras and LEDs is definitely not new, but source code and cheap commodity hardware makes it useful. Either way I doubt that this is actually the guys thesis project in its entirety.
- ashu 19y ago
  Nobody said there is. It's just a "genius hack". I don't see anything wrong with that unless he claims that as a contribution towards his thesis! As a side note, it's not immediately clear from comments like these whether the question is genuine or just a snide remark! Doesn't take much for somebody to take offense easily.
- thaumaturgy 19y ago
  As gently as I can manage ... At the end of the video is a URL for his website. It is http://johnnylee.net/ http://johnnylee.net/ His website is pretty neat. It includes more of his Wii remote projects -- including his interactive whiteboard which I think is even more interesting -- and it includes a link to his academic pursuits. Following the "Academics" link, you'll find that his thesis project is "Projector-Based Location Discovery and Tracking", which takes his whiteboard hobby a couple of steps further.
  - ivankirigin 19y ago
    I just tipped him at http://tipjoy.com http://tipjoy.com for this url: http://www.cs.cmu.edu/~johnny/projects/wii/ http://www.cs.cmu.edu/~johnny/projects/wii/
    - deleted 19y ago
      [deleted]
- whacked_new 19y ago
PStamatiou 19y ago
wow
gojomo 19y ago
What's awesome about this is how some low-tech (but only now common with the Wii) pieces have been assembled to give a more immersive/convincing 3D viewpoint than ever before. It's still not stereoscopic vision, but I believe there's an obvious and still fairly cheap extension that could reach that next level: place a pixel-aligned vertical lenticular lens¹ on the monitor. This can make it so that at predictable distances from the screen, each eye sees just even or odd columns of the display, which would each show just the left-eye or right-eye frames. Without eye-tracking, being just a little off optimal position could ruin the effect. With cheap eye-tracking, software can see which eye is in which zone, always maintaining the proper left-right mapping. (There might be flickering in transitions; I suspect people would adapt or software could fall back to monoscopic when eyes are misaligned. Thus you always have at least as strong an illusion of 3D as in this demo, but if your eyes fall into the proper lens bands -- boom! -- true stereo vision.) ¹ http://en.wikipedia.org/wiki/Lenticular_lens http://en.wikipedia.org/wiki/Lenticular_lens
ivankirigin 19y ago
At iRobot I had the luxury of having lots of people trying to sell us 3D headsets and such. It would make a huge difference for controlling a robot remotely. It's clear that this tech provides much better impression of 3D than any others I've seen. It would work for a small display as well, letting you look around through a tiny portal. Also, am I the only one who things "cake" when someone says "portal"?
- deleted 19y ago
  [deleted]
- LogicHoleFlaw 19y ago
  No..... and I'm surely not the only one who thinks "lie" when someone says "cake".
iamelgringo 19y ago
Not only is he working on some really cool stuff; I think that one of the most powerful things about Jonny Lee, is that he's such a great communicator. I'm sure the guy has a pretty big brain between his ears if he's getting his PhD in CS at CMU. But, he really does have a gift for being able to break down a subject and be able to communicate it well. He also understands the video medium and uses it to his advantage.
- mynameishere 19y ago
  Yeah, and did you see him bustin' moves toward the end? Awesome. How exactly do people interface into hardware like the Wii remote? What he's doing doesn't actually seem computationally difficult--the hard part seems to be...you know, getting the xyz coords from the device.
  - ks 19y ago
    There are drivers for it, but I don't know how good they are. http://www.wiili.org/Wiimote_driver http://www.wiili.org/Wiimote_driver
tyohn 19y ago
I wish I had a Wii developers kit. It would be so cool to create games using this technique.
pfedor 19y ago
Not directly related to the article, but let me mention a patch for Quake II my brother wrote a while ago, which creates another kind of 3D illusion using red-blue glasses: http://www.jfedor.org/red-blue-quake2/ http://www.jfedor.org/red-blue-quake2/
chrisconley 19y ago
this is one of the coolest things I've seen. ever.
kajecounterhack 19y ago
This is sort of old but yeah its cool right? There are also people using candles as the IR source. I'm not sure how that works...
redorb 19y ago
Good presentation! Johnny Lee has been evolutionary in wii hacks, a genuine hacker
allenbrunson 19y ago
from his blog, at http://procrastineering.blogspot.com/ http://procrastineering.blogspot.com/ -- "Louis Castle announced yesterday at GDC2008 that EA's upcoming title 'Boom Blox' will ship with an easter egg that allows head-tracking using the Wii remote! [...] I'm proud. If this pans out, it'll be only 5 months between the initial research prototype to integration into a major product release. Sweet! Happy to see my stuff being used. [...] Just in case you are wondering: No, I don't get any royalties or benefits for the use of this technique in games. Personally, I'm much happier impacting the state of technology on such a large scale in such a short period of time rather than struggling to transform it into personal financial gain." sigh. if we were just a LITTLE bit greedier, he could be so, so rich.
pius 19y ago
Get the get-well cards ... dude's project is sick.
dkokelley 19y ago
I used his whiteboard project to turn my laptop into a touch-capable laptop. The only thing is that my IR source doesn't work very well. I replaced a keychain LED with an IR LED but the switch is a simple squeeze and it is incredibly hard to turn it on and get a good signal, so when I'm trying to calibrate it, I usually accidentally click twice and have to redo it. Does anyone have a good idea for an IR pen?

142 points by thorax 19y ago

I sincerely feel like standing up and giving a long standing ovation. This is the future of gaming, at lest for 3D shooters, nothing less.

raju 19y ago
I was close to tears... Sniff...
greendestiny 19y ago
I'm really not sure about that, and for the sake of all the other VR and AR researchers who have gone before Johnny Lee - he really didn't invent rendering based on head tracking. The reason I'm not sure about this tech for 3D shooters is that its like looking through a virtual window, you can't fully reorient yourself in 3D - I'm not sure what combining this and full mouse-look control would be like. Also on a PC based setup where you're close to the screen the lack of stereoscopic vision would be more obvious. I think its more likely the future of different sort of game - maybe those old cabal style scrolling shooters for instance. Here's a video of some guys doing the same thing with face tracking (ie no LEDs needed) - http://www.youtube.com/watch?v=iUdZk4KUDTI http://www.youtube.com/watch?v=iUdZk4KUDTI
- danteembermage 19y ago
  My idea... an old west shoot out. You're in the saloon, peering out the window with a rifle. You can duck out of sight behind the television just like you could with a real window. You could also limit your exposure to different baddies by changing you angle of attack. The directional pad or keyboard could switch between windows, doors, newly created holes, etc. It's not immediately obvious how this will revolutionize all shooters, but for the shooting gallery type this should be included in all of them starting now.
- BrandonM 19y ago
  Make those IR-LED glasses into stereoscopic vision glasses and the entire 3D effect is realized.

amichail 19y ago

Is there a research contribution here?

greendestiny 19y ago
I guess taking this kind of head tracking to the wider world is useful. Tracking with cameras and LEDs is definitely not new, but source code and cheap commodity hardware makes it useful. Either way I doubt that this is actually the guys thesis project in its entirety.
ashu 19y ago
Nobody said there is. It's just a "genius hack". I don't see anything wrong with that unless he claims that as a contribution towards his thesis! As a side note, it's not immediately clear from comments like these whether the question is genuine or just a snide remark! Doesn't take much for somebody to take offense easily.
thaumaturgy 19y ago
As gently as I can manage ... At the end of the video is a URL for his website. It is http://johnnylee.net/ http://johnnylee.net/ His website is pretty neat. It includes more of his Wii remote projects -- including his interactive whiteboard which I think is even more interesting -- and it includes a link to his academic pursuits. Following the "Academics" link, you'll find that his thesis project is "Projector-Based Location Discovery and Tracking", which takes his whiteboard hobby a couple of steps further.
- ivankirigin 19y ago
  I just tipped him at http://tipjoy.com http://tipjoy.com for this url: http://www.cs.cmu.edu/~johnny/projects/wii/ http://www.cs.cmu.edu/~johnny/projects/wii/
  - deleted 19y ago
    [deleted]
whacked_new 19y ago

PStamatiou 19y ago

wow

gojomo 19y ago

What's awesome about this is how some low-tech (but only now common with the Wii) pieces have been assembled to give a more immersive/convincing 3D viewpoint than ever before. It's still not stereoscopic vision, but I believe there's an obvious and still fairly cheap extension that could reach that next level: place a pixel-aligned vertical lenticular lens¹ on the monitor. This can make it so that at predictable distances from the screen, each eye sees just even or odd columns of the display, which would each show just the left-eye or right-eye frames. Without eye-tracking, being just a little off optimal position could ruin the effect. With cheap eye-tracking, software can see which eye is in which zone, always maintaining the proper left-right mapping. (There might be flickering in transitions; I suspect people would adapt or software could fall back to monoscopic when eyes are misaligned. Thus you always have at least as strong an illusion of 3D as in this demo, but if your eyes fall into the proper lens bands -- boom! -- true stereo vision.) ¹ http://en.wikipedia.org/wiki/Lenticular_lens http://en.wikipedia.org/wiki/Lenticular_lens

ivankirigin 19y ago

At iRobot I had the luxury of having lots of people trying to sell us 3D headsets and such. It would make a huge difference for controlling a robot remotely. It's clear that this tech provides much better impression of 3D than any others I've seen. It would work for a small display as well, letting you look around through a tiny portal. Also, am I the only one who things "cake" when someone says "portal"?

deleted 19y ago
[deleted]
LogicHoleFlaw 19y ago
No..... and I'm surely not the only one who thinks "lie" when someone says "cake".

iamelgringo 19y ago

Not only is he working on some really cool stuff; I think that one of the most powerful things about Jonny Lee, is that he's such a great communicator. I'm sure the guy has a pretty big brain between his ears if he's getting his PhD in CS at CMU. But, he really does have a gift for being able to break down a subject and be able to communicate it well. He also understands the video medium and uses it to his advantage.

mynameishere 19y ago
Yeah, and did you see him bustin' moves toward the end? Awesome. How exactly do people interface into hardware like the Wii remote? What he's doing doesn't actually seem computationally difficult--the hard part seems to be...you know, getting the xyz coords from the device.
- ks 19y ago
  There are drivers for it, but I don't know how good they are. http://www.wiili.org/Wiimote_driver http://www.wiili.org/Wiimote_driver

tyohn 19y ago

I wish I had a Wii developers kit. It would be so cool to create games using this technique.

pfedor 19y ago

Not directly related to the article, but let me mention a patch for Quake II my brother wrote a while ago, which creates another kind of 3D illusion using red-blue glasses: http://www.jfedor.org/red-blue-quake2/ http://www.jfedor.org/red-blue-quake2/

chrisconley 19y ago

this is one of the coolest things I've seen. ever.

kajecounterhack 19y ago

This is sort of old but yeah its cool right? There are also people using candles as the IR source. I'm not sure how that works...

redorb 19y ago

Good presentation! Johnny Lee has been evolutionary in wii hacks, a genuine hacker

allenbrunson 19y ago

from his blog, at http://procrastineering.blogspot.com/ http://procrastineering.blogspot.com/ -- "Louis Castle announced yesterday at GDC2008 that EA's upcoming title 'Boom Blox' will ship with an easter egg that allows head-tracking using the Wii remote! [...] I'm proud. If this pans out, it'll be only 5 months between the initial research prototype to integration into a major product release. Sweet! Happy to see my stuff being used. [...] Just in case you are wondering: No, I don't get any royalties or benefits for the use of this technique in games. Personally, I'm much happier impacting the state of technology on such a large scale in such a short period of time rather than struggling to transform it into personal financial gain." sigh. if we were just a LITTLE bit greedier, he could be so, so rich.

pius 19y ago

Get the get-well cards ... dude's project is sick.

dkokelley 19y ago

I used his whiteboard project to turn my laptop into a touch-capable laptop. The only thing is that my IR source doesn't work very well. I replaced a keychain LED with an IR LED but the switch is a simple squeeze and it is incredibly hard to turn it on and get a good signal, so when I'm trying to calibrate it, I usually accidentally click twice and have to redo it. Does anyone have a good idea for an IR pen?