5 ms·

Surround 360 is now open source

156 points by samber 10y ago

banderon 10y ago
How much would it cost to make one?
- Cshelton 10y ago
  Just by breezing through the hardware specs, using the stuff they recommend, I'd estimate around $35k -$40k... So let's call it $50k, built and ready to use =) those cameras are $1.5k each...17 of them. Edit: Actually, those lenses are around $700/each as well. And you would definitely want a professional putting it together for you with that kind of money. So $100k probably. Definitely made for industry/production use, not for someone trying to film a weekend trip in the mountains haha.
  - swei 10y ago
    Retail cost for the Point Grey camera is about 1.3k and Sunex is around $130. Total cost is running about 30-40k, including computer hardware.
  - criley2 10y ago
    >not for someone trying to film a weekend trip in the mountains haha. The GoPro Omni and the NCTECH IRIS360 may be a little bit closer to the indie scene rather than the professional scene, but those solutions are still $2000 - $5000... and are only 4-6 camera solutions.
    - redtuesday 10y ago
      There is also 360heros [1][2] which can use up to 14 GoPro cameras for under 10.000. [1] http://www.360heros.com/2014/01/worlds-first-fully-spherical-3d-360-video-and-photo-gear/ http://www.360heros.com/2014/01/worlds-first-fully-spherical... [2] http://shop.360heros.com/3DPRO-360-VIDEO-p/3dpro.htm http://shop.360heros.com/3DPRO-360-VIDEO-p/3dpro.htm
  - corysama 10y ago
    Direct source for the cameras requires registration to see the price https://www.ptgrey.com/grasshopper3-41-mp-color-usb3-vision-cmosis-cmv4000-2-camera https://www.ptgrey.com/grasshopper3-41-mp-color-usb3-vision-... Non-direct source has price listing http://www.edmundoptics.com/cameras/usb-cameras/point-grey-grasshopper-3-high-performance-usb-3-0-cameras/88514/ http://www.edmundoptics.com/cameras/usb-cameras/point-grey-g...
shahbazac 10y ago
Can a rig like this be used to extract a depth map/point cloud, instead of just visual 3d images? What will be the accuracy of such a point cloud?
- deleted 10y ago
  [deleted]
- luciens 10y ago
  Yes but not very well. The quality of depth generation output from stereo pair images is pretty tightly linked to sensor size and lens distortion. Camera arrays like this tend to use many low res cameras with wide angle lenses so, again, they are kinda bad for depth generation. I found a blog full of tools for photogrammetry & depth map creation a while back that are great for getting a better handle on these limitations if you are interested: http://3dstereophoto.blogspot.com/p/software.html?m=1 http://3dstereophoto.blogspot.com/p/software.html?m=1
- qdot76367 10y ago
  You want LIDAR for making the point clouds, and a camera for colorizing them. Pure photogrammetry is doable (I think that's what Earthmine did for a while?), but it isn't as pretty or accurate.
  - snovv_crash 10y ago
    For single-point-of-view, yes. Otherwise, if you can take images from multiple viewpoints, there are a bunch of solutions for pure photogrammetry, eg. Pix4d, Photoscan, Capturing Reality. Theoretically, there is enough data in the images to build photorealistic models. Algorithmically, we aren't quite there yet.
- Keyframe 10y ago
  Considering you can extract a point cloud from regular footage with parallax in it, even from stills, I see why not. It wouldn't be very accurate though.
- Cshelton 10y ago
intrasight 10y ago
How about using a dozen Rasperry Pis and camera modules? Perhaps not the same image quality as a $50K system, but is there any technical reason it couldn't work with Surround 360?
Veratyr 10y ago
Hardware looks very similar to https://vr.google.com/jump/ https://vr.google.com/jump/ and https://gopro.com/odyssey https://gopro.com/odyssey but with cameras top and bottom. Really interesting to head about the math and software that went into it though and great that they made it open source!
- greenknight 10y ago
  The GoPro cameras are quite different to the cameras that facebook are using... GoPro cameras have rolling shutter on the lens, whilst the facebook cameras are using higher quality cameras which have no rolling shutter. The google jump is also using go pro cameras.
dharma1 10y ago
Nice. Has anyone had a look at the stitching source code - how tightly coupled is it to this particular camera/lens combo? You can pick up Xiaomi Yi's for about $70/a piece. Of course RAW footage from the point gray will give tons more dynamic range but still would be a relatively decent setup for 3D360.
- greenknight 10y ago
  Pretty sure its not just the raw footage but the face that the action cameras have rolling shutter... I would assume that the rolling shutter along with the cameras not being in sync would mean that doing optical flow wouldn't work correctly in terms of creating a clean image.
dkarapetyan 10y ago
This is just nerd candy right? I mean facebook is in the same business as google when it comes to what the business model is based on. Get people to pull levers in a virtual casino and sell the resulting statistics to advertisers. The surprising thing is that I think it actually works. New graduates will be lining up to work on projects like this and then end up debugging shitty PHP.
- ibash 10y ago
  If you get enough nerd candy you can evolve them.
- mgraczyk 10y ago
  I work on similar projects at Google, so I can say with certainty that this is not just about candy. YouTube makes up a sizable portion of Google's business. Providing 360 capture devices like this (ours is called Jump) allows people to create new content for YouTube, which can then be monetized. Building cool VR hardware and software is not as distant from Google's and Facebook's core businesses as you believe. Watching 360 videos with 3D audio isn't necessarily the height of human experience, but it's a far cry from "pulling levers in a virtual casino".
  - dkarapetyan 10y ago
    And how many people work on that project? 10? 100? 200 people?
    - mgraczyk 10y ago
      Many, and many more on similarly interesting projects. All I can say is that ~nobody writes PHP :)
      - dkarapetyan 10y ago
        Lucky then.
- zamalek 10y ago
travelhead 10y ago
Lol guys just buy a gear360, light weight and fits in the palm of your hand. Been using it for months, $350-$400 on Amazon works great!
- luciens 10y ago
  Gear 360, Kodak SP360 and Ricoh Theta, GoPro Omni are all low resolution monoscopic 360 capture. Surround 360, Nokia OZO, Project Beyond, GoPro Odyssey are high resolution stereo 360.
sigvef 10y ago
What are the differences between this and Samsung's Project Beyond camera? http://thinktankteam.info/beyond_m/ http://thinktankteam.info/beyond_m/
ashitlerferad 10y ago
More open source cameras: http://www.elphel.com/ http://www.elphel.com/
justinclift 10y ago
As an interesting data point, it looks like they designed the hardware using Solidworks 2016: https://github.com/facebook/Surround360/blob/master/surround360_design/3d_models/STEP_files/160523_3.STEP#L9-L10 (.STEP files - good for CAD data interchange - often include the name of the creation program)

156 points by samber 10y ago

banderon 10y ago

How much would it cost to make one?

Cshelton 10y ago
Just by breezing through the hardware specs, using the stuff they recommend, I'd estimate around $35k -$40k... So let's call it $50k, built and ready to use =) those cameras are $1.5k each...17 of them. Edit: Actually, those lenses are around $700/each as well. And you would definitely want a professional putting it together for you with that kind of money. So $100k probably. Definitely made for industry/production use, not for someone trying to film a weekend trip in the mountains haha.
- swei 10y ago
  Retail cost for the Point Grey camera is about 1.3k and Sunex is around $130. Total cost is running about 30-40k, including computer hardware.
- criley2 10y ago
  >not for someone trying to film a weekend trip in the mountains haha. The GoPro Omni and the NCTECH IRIS360 may be a little bit closer to the indie scene rather than the professional scene, but those solutions are still $2000 - $5000... and are only 4-6 camera solutions.
  - redtuesday 10y ago
    There is also 360heros [1][2] which can use up to 14 GoPro cameras for under 10.000. [1] http://www.360heros.com/2014/01/worlds-first-fully-spherical-3d-360-video-and-photo-gear/ http://www.360heros.com/2014/01/worlds-first-fully-spherical... [2] http://shop.360heros.com/3DPRO-360-VIDEO-p/3dpro.htm http://shop.360heros.com/3DPRO-360-VIDEO-p/3dpro.htm
- corysama 10y ago
  Direct source for the cameras requires registration to see the price https://www.ptgrey.com/grasshopper3-41-mp-color-usb3-vision-cmosis-cmv4000-2-camera https://www.ptgrey.com/grasshopper3-41-mp-color-usb3-vision-... Non-direct source has price listing http://www.edmundoptics.com/cameras/usb-cameras/point-grey-grasshopper-3-high-performance-usb-3-0-cameras/88514/ http://www.edmundoptics.com/cameras/usb-cameras/point-grey-g...

shahbazac 10y ago

Can a rig like this be used to extract a depth map/point cloud, instead of just visual 3d images? What will be the accuracy of such a point cloud?

deleted 10y ago
[deleted]
luciens 10y ago
Yes but not very well. The quality of depth generation output from stereo pair images is pretty tightly linked to sensor size and lens distortion. Camera arrays like this tend to use many low res cameras with wide angle lenses so, again, they are kinda bad for depth generation. I found a blog full of tools for photogrammetry & depth map creation a while back that are great for getting a better handle on these limitations if you are interested: http://3dstereophoto.blogspot.com/p/software.html?m=1 http://3dstereophoto.blogspot.com/p/software.html?m=1
qdot76367 10y ago
You want LIDAR for making the point clouds, and a camera for colorizing them. Pure photogrammetry is doable (I think that's what Earthmine did for a while?), but it isn't as pretty or accurate.
- snovv_crash 10y ago
  For single-point-of-view, yes. Otherwise, if you can take images from multiple viewpoints, there are a bunch of solutions for pure photogrammetry, eg. Pix4d, Photoscan, Capturing Reality. Theoretically, there is enough data in the images to build photorealistic models. Algorithmically, we aren't quite there yet.
Keyframe 10y ago
Considering you can extract a point cloud from regular footage with parallax in it, even from stills, I see why not. It wouldn't be very accurate though.
Cshelton 10y ago

intrasight 10y ago

How about using a dozen Rasperry Pis and camera modules? Perhaps not the same image quality as a $50K system, but is there any technical reason it couldn't work with Surround 360?

Veratyr 10y ago

Hardware looks very similar to https://vr.google.com/jump/ https://vr.google.com/jump/ and https://gopro.com/odyssey https://gopro.com/odyssey but with cameras top and bottom. Really interesting to head about the math and software that went into it though and great that they made it open source!

greenknight 10y ago
The GoPro cameras are quite different to the cameras that facebook are using... GoPro cameras have rolling shutter on the lens, whilst the facebook cameras are using higher quality cameras which have no rolling shutter. The google jump is also using go pro cameras.

dharma1 10y ago

Nice. Has anyone had a look at the stitching source code - how tightly coupled is it to this particular camera/lens combo? You can pick up Xiaomi Yi's for about $70/a piece. Of course RAW footage from the point gray will give tons more dynamic range but still would be a relatively decent setup for 3D360.

greenknight 10y ago
Pretty sure its not just the raw footage but the face that the action cameras have rolling shutter... I would assume that the rolling shutter along with the cameras not being in sync would mean that doing optical flow wouldn't work correctly in terms of creating a clean image.

dkarapetyan 10y ago

This is just nerd candy right? I mean facebook is in the same business as google when it comes to what the business model is based on. Get people to pull levers in a virtual casino and sell the resulting statistics to advertisers. The surprising thing is that I think it actually works. New graduates will be lining up to work on projects like this and then end up debugging shitty PHP.

ibash 10y ago
If you get enough nerd candy you can evolve them.
mgraczyk 10y ago
I work on similar projects at Google, so I can say with certainty that this is not just about candy. YouTube makes up a sizable portion of Google's business. Providing 360 capture devices like this (ours is called Jump) allows people to create new content for YouTube, which can then be monetized. Building cool VR hardware and software is not as distant from Google's and Facebook's core businesses as you believe. Watching 360 videos with 3D audio isn't necessarily the height of human experience, but it's a far cry from "pulling levers in a virtual casino".
- dkarapetyan 10y ago
  And how many people work on that project? 10? 100? 200 people?
  - mgraczyk 10y ago
    Many, and many more on similarly interesting projects. All I can say is that ~nobody writes PHP :)
    - dkarapetyan 10y ago
      Lucky then.
zamalek 10y ago

travelhead 10y ago

Lol guys just buy a gear360, light weight and fits in the palm of your hand. Been using it for months, $350-$400 on Amazon works great!

luciens 10y ago
Gear 360, Kodak SP360 and Ricoh Theta, GoPro Omni are all low resolution monoscopic 360 capture. Surround 360, Nokia OZO, Project Beyond, GoPro Odyssey are high resolution stereo 360.

sigvef 10y ago

What are the differences between this and Samsung's Project Beyond camera? http://thinktankteam.info/beyond_m/ http://thinktankteam.info/beyond_m/

ashitlerferad 10y ago

More open source cameras: http://www.elphel.com/ http://www.elphel.com/

justinclift 10y ago

As an interesting data point, it looks like they designed the hardware using Solidworks 2016: https://github.com/facebook/Surround360/blob/master/surround360_design/3d_models/STEP_files/160523_3.STEP#L9-L10 (.STEP files - good for CAD data interchange - often include the name of the creation program)