7 ms·
FPV with Oculus Rift and a Quadcopter
- SkyDrone 12y agoGreat to see more projects using the Oculus Rift for FPV. We are using it in out Sky Drone FPV system.
- ohadron 12y agoYoutube here: https://www.youtube.com/watch?v=ANSjwWomIJ8 https://www.youtube.com/watch?v=ANSjwWomIJ8 Very cool idea. I could think of some indoor applications for it. For outdoors, since the distance between the cameras and the object they're looking at is so far, and the distance between the two eyes is so close the difference between the two images is almost meaningless. I'd put the two cameras further apart and see what happens.
- bigiain 12y agoI remember looking through this thing: http://eyestilts.com/intro.html http://eyestilts.com/intro.html at Burningman many years ago - it makes things look very un-real - much like tilt-shift photos. I suspect it might become pretty disorienting to fly using such odd optics...
- omegant 12y agoThe lens in this device are pretty close to the looking subject. If you have cameras with a wider separation on a drone they'll be farther asay from the subjects and the effect will be less extreme, they are also very difficult to mount with 3 meters of separation.
- baddox 12y agoUse two GPS guided quadcopters, each with their own camera. Problem solved, and now you can dynamically vary the intensity of the depth perception.
- bigiain 12y ago"Problem solved", says - I suspect - the guy who's never tried to make even _one_ quad copter go where he wanted, never mind a pair of them controlled to optically-accurate positions! ;-)
- baddox 12y agoThat device is awesome! I would love to try it. It would be fairly simple to simulate the effect in any game with VR goggles. A really weird effect I've tried is to reverse the images each eye sees. I did this (first unintentionally, then deliberately) with my 3D television. It inverts the depth perception, so a small object in the foreground looks like it's sunken deep into the background. But mostly it's just confusing and stressful on the eyes, since there's presumably no evolutionary preparation to make sense of such visual input. Someone should make a physical optical device to offer this effect in the real world (it probably already exists).
- thisjepisje 12y agoReminds me of this xkcd: http://xkcd.com/941/ http://xkcd.com/941/
- maaaats 12y agoOur drone couldn't really fit anything wider, so we went with the average human IPD according to the rift SDK. You're right, at distance it's of no use. Closeup, as we tried to illustrate with the pole, it helps. Aligning the cheap CMOS cameras were hard, though.
- elliottkember 12y agoI'd love to know how well this works. I always assumed the pan and tilt setup would be too slow for the Rift.
- catshirt 12y agoi assumed it must use a 360 camera but i was wrong. i don't know much about quadcopters but it seems like a decent idea to me
- maaaats 12y agoThe video latency is low enough that you don't notice. We tried with moving our hand in front of it, it feels weird too see yourself real-time! The rotations of the cameras are a bit too slow though, but mainly because of our setup, so this can be a lot better. Our custom protocol couldn't really handle dropped packets, so if we filled the sender's or receiver's buffers, the servos would start to spasm. To counter that we just didn't send head position as often as we could. Of course, by fixing that you still get the round-trip time of turning your head -> moving the cameras -> getting updated image back. But we think it's feasible.
- elliottkember 12y agoI do a bit of FPV myself so I know what you mean about the real-time! The round-trip is what I meant - especially with all the work to get the latency down on the Rift. I've always thought fixed, super-wide-angle lenses and software would be the ideal way to go - the groundstation does all the work.
- maaaats 12y agoYeah, it's been suggested to us a few times after people have seen the project. Someone should definitely explore that way as well.
- baddox 12y agoAny reason you didn't just transmit 3 normal RC PWM channels? You could do all the custom circuitry on the ground. Plug the Oculus into a microprocessor, then output to the trainer port of an RC transmitter or perhaps use one of the standard RC transmitter modules like JR or Futaba.
- mambodog 12y agoAs it appears to be wireless I would imagine the latency between head movement to camera movement and then video capture to video output would likely be enough to make users feel ill.
- deleted 12y ago[deleted]
- gojomo 12y agoPerhaps, but it won't be long before omnidirectional arrays of cameras have all perspectives available even before you turn your head.
- ohadron 12y agoWireless 2.4ghz radio latency is measured in milliseconds which is quick enough not to notice. Latency also depends on how quick the gimbal servos are but that too may be quick enough.
- cvbncvbncgbc 12y agoSince the movement and change of speed of the copter isn't very fast you can apply time warp to this; or just make the camera film in 120 fps.
- baddox 12y agoThe camera movement latency is probably quite noticeable. With a brushless gimbal this could be greatly reduced. The 5.8 GHz video transmission setup is widely used for FPV flying, and its latency is negligible. Video showing the reaction speed of brushless gimbals: http://youtu.be/Dr5CnIA52zY http://youtu.be/Dr5CnIA52zY
- matsemann 12y agoWow, cool that someone is sharing our project! It's for a course at my university called "experts in teams" where they combine master students from many programs in teams and give them tasks. A video with more shoots is available[1]. We got UAVs for the Norwegian oil industry as a task, and explored how they can be flown better in the future. Other teams with the same task made other cool stuff as well, for instance a custom drone for geological mapping. [2] [1]: https://www.youtube.com/watch?v=ANSjwWomIJ8 https://www.youtube.com/watch?v=ANSjwWomIJ8 [2]: https://www.youtube.com/watch?v=5di01L1mot8 https://www.youtube.com/watch?v=5di01L1mot8
- makmanalp 12y agoCould you tell us more about the "experts in teams" program? That sounds like a very interesting and unique approach.
- maaaats 12y agoThe link[1] has a description. Basically, different professors create different "villages" one may apply to. Everyone taking a master's has to take this course, e.g. all 4th year students. Our university has everything from engineering to human sciences, so the diversity of the groups are great. This gives new influences to the field of the village, and good practice in teamwork for us students. [1]: http://www.ntnu.edu/eit/formal-documents http://www.ntnu.edu/eit/formal-documents
- baddox 12y agoDoes it look weird to have different little bits of video static in each eye? Also, a cool idea would be an OSD that supports 3D overlays.
- maaaats 12y agoYeah, it is weird. You kinda want to rub your eyes or something. It was even weirder when we adjusted the cameras. I looked through the Oculus Rift, while another person adjusted and I would tell him when they were properly aligned. Having a person "turn" your eye, drop it etc. was crazy!
- baddox 12y agoThis idea has been buzzing around the FPV community since the Oculus Rift was first announced. There have been a few barebones setups, but this is the first fully-fledged implementation I've seen. It's a fun usage for VR goggles and a great build project, but it's honestly not super useful, even for hobby flying. Any quadcopter you're flying with a head tracker is going to be so far away from other objects that the 3D effect will be minimal. The 3D might be great for zipping quickly through trees/obstacles like a pod racer [0], but for that you don't dare use head tracking. [0] http://youtu.be/xlKrabm5Exg http://youtu.be/xlKrabm5Exg
- pedrocr 12y agoThey're not controlling the quad with head tracking, just the camera orientation. So for "pod racing" this should work making it more realistic both because of the 3D effect and the fact that you can turn your head like you would if you were sitting inside it.
- baddox 12y agoI know. It would still be disastrous to be able to turn your head when doing fast moves like this. To make it work there would need to be something in the frame of the camera, or overlaid on the video, to indicate the center of the frame, because orientation is extremely important.
- pedrocr 12y agoYeah, having a HUD in a fixed orientation relative to the quad would be awesome.
- pmarin 12y agoI think it is not for the 3D. It is for precise orientation of the camera using the head tracking.
- higherpurpose 12y agoLooks like the cameras could use some good OIS.
- baddox 12y agoGood old fashioned vibration dampening would go a long way. That's a big area of concern for all FPV pilots, and it's fairly easy to get good results. The ultimate solution would be to also add a brushless gimbal for stabilization as well as low reaction latency: http://youtu.be/Dr5CnIA52zY http://youtu.be/Dr5CnIA52zY
- montag 12y agoAwesome project. Have you thought about using a spherical camera, so that the pan/tilt can be done entirely in software? Make it a bandwidth problem, instead of a mechanical problem.
- baddox 12y agoOr, for a more challenging (though more dangerous and probably less useful) project, make the head tracking control the pitch, roll, and yaw of the quadcopter directly. With altitude hold functionality (a lot of flight controllers implement this with a barometer, though not the DJI Phantom) you could have completely hands free flight.
- sopooneo 12y agoI had thought of this, then I wondered, why not have head movement do both? That is, have the camera pan/tilt/roll in sync with your head and also shift the plane on the same axis. That way you could know which way was centered (looking out over the nose of the craft), and immediately see where you'd be headed when the turn completed.
- andygates 12y agoThere's a chap who got a Parrot AR Drone flying like this - they're quite simple to control with arbitrary devices via node.js and nodecopter. Fun proof of concept.
- ris 12y agoThink about the geometry of that problem harder - particularly when it comes to the stereo vision aspect.
- sopooneo 12y agoI think one major issue there is that all of the available video transmitters for FPV transmit in standard definition. You'd have a hard time spreading that over a sphere and being able to coble anything useful together. Most of the FPV videos you see on youtube are not showing what the pilot saw in real time. They're showing you the HD that was recording on board the plane and retrieved after landing.
- spiritplumber 12y agohttp://spiritplumber.deviantart.com/art/dancers-in-the-dark-390629203 http://spiritplumber.deviantart.com/art/dancers-in-the-dark-... Here, I wrote you a little story.
- spiritplumber 12y agoGeez, why the hate? It's a story about FPV quadcopters dancing.
- leoc 12y agoApparently telepresence like this was the original application of HMDs (like the Philco Headsight); the concept was only generalised into VR (and AR) by Ivan Sutherland a few years later. Somewhat related: https://www.youtube.com/watch?v=SQ2tCMXOd_w https://www.youtube.com/watch?v=SQ2tCMXOd_w https://share.oculusvr.com/app/hiyoshi-jump https://share.oculusvr.com/app/hiyoshi-jump
- DanHulton 12y agoThis is going to be a job someday. People are going to train for this. Very likely police, field agents, military, but there are absolutely going to be people whose sole jobs will be piloting drones like this. (Until those people are replaced by AI algorithms, but still.)
- gcr 12y agoI want to turn this into a sport. Imagine this with some augmented reality thing where multiple contestants could shoot each others' copters with an IR laser or something. Heck yes.
- spullara 12y agoThis game exists for the Parrot AR drones. http://singularityhub.com/2010/06/17/parrots-ar-drone-gets-cool-new-augmented-reality-games-video/ http://singularityhub.com/2010/06/17/parrots-ar-drone-gets-c...
- deleted 12y ago[deleted]