15 ms·

Show HN: A web app to turn photos into 3D AR models.

191 points by schappim 5y ago

schappim 5y ago
Hi I'm Marcus (one of the creators). Feel free to hit me up if you have any questions :-) We are "very alpha" right now. Edit: We have an API over at: https://usdz.app/api https://usdz.app/api
- genpfault 5y ago
  What photogrammetry pipeline are you using?
  - schappim 5y ago
    It is a stupidly long AWS based pipeline w/ the photogrammetry part is in swift.
  - deleted 5y ago
    [deleted]
- porkbrain 5y ago
  If it's not a secret, I'd love to read some design document about the backend behind this app. What's the blackbox that takes set of images as an input and produces a 3D model as an output?
  - schappim 5y ago
    We've very much built on the shoulders of giants: - Frontend & API (Rails + jumpstartrails.com) - Images --> USDZ (Swift Code on a Cloud hosted Mac) - USDZ --> GLTF (Swift + Custom Ruby) - CDN (AWS Cloudfront) Lots of Glue: - AWS S3 / Cloudfront - AWS SQS
    - deleted 5y ago
      [deleted]
  - nobbis 5y ago
    It uses the Object Capture API introduced in the latest version of macOS. The API allows you to create 3D models from images with just a few lines of code. https://developer.apple.com/augmented-reality/object-capture https://developer.apple.com/augmented-reality/object-capture
    - schappim 5y ago
      If we're talking Swift frameworks, ModelIO (underrated) and ObjectCapture are certainly part of things.
      - nobbis 5y ago
        He asked for the “blackbox that takes set of images as an input and produces a 3D model as an output.” That black box is Object Capture API, which takes a folder of images and outputs an USDZ (or OBJ) file. Model I/O isn’t necessary. Kudos to Apple for releasing a photogrammetry pipeline that allows web and app developers to build tools like this, with zero knowledge of 3D reconstruction required.
        schappim 5y ago
        You're not wrong (althought we do use ModelIO a lot). I just saw that you're from https://www.aboundlabs.com https://www.aboundlabs.com and metascan.ai Your scans look great! So how does your black box work?
        secretsatan 5y ago
        I’ve just been trying out the photogrammetry session, it’s super simple to use. I think Apple’s photogrammetry software isn’t as advanced as other’s, but it is free and works well small scale. I work in the same field, making a capture app for photogrammetry software
        schappim 5y ago
        Yeah what Apple's has is a clear lead on the time required to build objects. I think that the photogrammetry side of things, the "give me an object now" will shift towards mobile device camera apps. It is only serverside now due to power requirements.
- armagon 5y ago
  Hi Marcus. It would be nice to have some idea going in for what the requirements are. - I have to admit, I don't understand why I can't preview the 3D experiences right on my computer. - Likewise, when I try the QR code on my iPhone 6 ... well, I'm hardly surprised to not see anything in 3D, but it wouldn't hurt to have a little more detail than "Scan this QR code on your iPhone or iPad." I suppose I could mention that I've done some game development and a little bit of VR/AR, but have never heard of the USDZ file format before.
  - schappim 5y ago
    Hi there! USDZ is the format that Apple is backing for VR/AR delivery (developed by Pixar)[1]. We're betting that it is what is going to be used in any Apple headset. Right now you can't easily view USDZs in the browser (the format is a moving target), however Apple is making strides to make it available with the proposed Model element[2]. We were doing some USDZ --> GLTF converstion (GLTF being a format that will work in the browser), but it is jankey (technical term). I have not tried an iPhone 6 yet. I will have to pry it from my children's hands and remove a layer of cheese and excretions before I do. I've made a video on how to do this via the browser here[3]. TBH, the experience a heap better in the iOS App (currently in testflight) [4]. [1] https://graphics.pixar.com/usd/release/wp_usdz.html https://graphics.pixar.com/usd/release/wp_usdz.html (seem to have cert issues) [2] https://github.com/WebKit/explainers/tree/main/model https://github.com/WebKit/explainers/tree/main/model [3] https://usdz.app/screencasts/getting-started https://usdz.app/screencasts/getting-started [4] https://testflight.apple.com/join/VteIxFd7 https://testflight.apple.com/join/VteIxFd7
    - schappim 5y ago
      Oh, I should also mention, HEICs are your friend (as they have depth, and accelerometer data embedded in the file format).
- nemacol 5y ago
  Anyway to take a model that was created and get it into CAD / modeling software? I don't know much about VR/AR but I know it is a pain in the bum to replicate an object / create a model for 3d printing. Thank you.
  - schappim 5y ago
    We allow you to download as USDZs (and sometimes do conversions to GLTFs). You can import both of these into Blender and export as Obj (and I'm pretty sure STLs). You may want to use our API to do the Images ---> USDZ conversion[1]. [1] https://usdz.app/api https://usdz.app/api
    - nemacol 5y ago
      Excellent. Thank you for the reply.
- tluyben2 5y ago
  This is really great, thanks for doing this!
- ianbutler 5y ago
  Hey very cool stuff, one piece of feedback it would be good to be able to delete models on the beta iPhone app, or if we can it’s not particularly obvious how to do so.
  - schappim 5y ago
    You're right! Right now it is not possible (via the phone). It should be possible via the API or https://usdz.app https://usdz.app
- thih9 5y ago
  Pretty cool! How are you handling spam or other kinds of content moderation? Is it easier because users don’t upload a regular text/image and need to create something custom?
  - schappim 5y ago
    There hasn't been anything humans can't handle. If it gets out of hand, we'll look at something similar to Akismet, or an ML based solution. Right now we're hand picking experiences to make it to the front page.
    - onara 5y ago
      At early stage you are not going to get spam anyway. Once it grows (and i hope it does) you will get fare share of spam. As you said at that point you could check solutions like Akismet, OOPSpam.
    - thih9 5y ago
      Thanks for answering! Hand picking sounds like a really good approach, simple and effective.
      - schappim 5y ago
        So funny you should mention this, yesterday we had our "first porn" moment. Thankfully if one throws a bunch of random images at the API and trys to make a model, the algorithm just fails. However we're thinking it might be fun to do something w/ NSFW image detection and display a silly model...
- hpen 5y ago
  Hey have you guys considered licensing an API?
  - schappim 5y ago
    We don’t think is a good long term business model (with the obvious path of functionality moving to the client).
    - toomuchtodo 5y ago
      Smart. Looking forward to this being an on device capability.
granto 5y ago
This looks great, congrats! I’ve tried a number of photogrammetry software and they typically had challenges. One question: are you able to determine the size of the object?
coolmitch 5y ago
testflight app doesn't seem to be able to access my camera (iPhone SE2) despite having permissions
- schappim 5y ago
  Thanks for the report Mitch. We'll have to look into this.
  - rckt 5y ago
    Having the same issue, using the SE 2nd too.
edoceo 5y ago
Would love to see some examples of how to go from this into something like A-frame - or even a guide on how it might work?
- schappim 5y ago
  Your wish is my command. - This is one I'm building for mobile (work in progress) [1] - This is one for desktop [2] [1] https://files.littlebird.com.au/mobile-how-to-wip-BDikJ9.m4v https://files.littlebird.com.au/mobile-how-to-wip-BDikJ9.m4v [2] https://usdz.app/screencasts/getting-started https://usdz.app/screencasts/getting-started
somishere 5y ago
Very Interesting project! esp. when compared to the swathe of junky or overly complex photogrammetry tools out there from a few years back (but mostly still polluting the app store) I've only recently been trying out poly.cam which is probably the best example of a user friendly workflow ... but it's knowingly priced. Have you guys got a pricing model in mind? Also do you have any documentation around best practices/results, etc? Anyway sounds like things are going to pick up again quickly with apple's new APIs. Good luck - exciting times ahead!
- schappim 5y ago
  Thanks for the tip! I'm making a "how-to" video now. I've yet to do the voiceover, but here are the raw slides: https://files.littlebird.com.au/take-great-shots-cXH2CM.m4v https://files.littlebird.com.au/take-great-shots-cXH2CM.m4v The TL;DR is: - Avoid thin objects - Avoid reflective objects - Avoid transparent objects - Avoid objects that deform - Hardshadows are not good, use soft lighting - Your photos should have a crisp focus and be well lit (across the object) - Things work best when the background has a solid colour - Make the object take up the full frame (more data for the algorithm) - Make sure to get shots from all sides
- nobbis 5y ago
  As far as best practices, Apple has some documentation on how to use their photogrammetry pipeline: https://developer.apple.com/documentation/realitykit/capturing_photographs_for_realitykit_object_capture https://developer.apple.com/documentation/realitykit/capturi...
  - schappim 5y ago
    Hey Nobbis, I've just requested an invite to your SDK competitor to Apple and others. What you've done looks great and I have a few questions: - Does your solution have the same photogrammetry requirements as the Apple/others? - Is there a reason you're only supporting iOS at this time? Is that because you're using an iOS framework, or because you're relying on a file format (say HEIC)? - How long until you guys open up the API? Keep up the good work!
zuhayeer 5y ago
Very cool, reminds me a bit of the photogrammetry software Autodesk has. Always thought it could be more "consumerized" so that a series of photos can easily be made into 3D models. This was also a nice throwback for me to some drone photographs I've taken a while back at Berkeley stitched together into a 3d model: https://twitter.com/zuhayeer/status/933752635726880768?s=20 https://twitter.com/zuhayeer/status/933752635726880768?s=20
- schappim 5y ago
  Thanks for your kind words! Ultimately I think this is going to be part of the camera apps or some built-in app. It is just a function of power.
etaioinshrdlu 5y ago
How many photos does it take? I think an API for this would be ridiculously cool.
- schappim 5y ago
  You need at least 20-ish, 70-200 make amazing objects. There is a free API here: https://usdz.app/api https://usdz.app/api
aero-glide2 5y ago
Does this have android support?
- schappim 5y ago
  It will. We've been building the mobile app using a cross platform framework (Capacitor) with the view of making it Android compatible. Capacitor does the job, but we've had to do some native code to fill holes. If you're a dev, you can also use the API: https://usdz.app/api https://usdz.app/api You could probably whip up a quick script that runs on Android. My first go of doing mobile (for enlisting my co-founder) was to use Apple Shortcuts w/ the API.
darkteflon 5y ago
Looks Excellent! Going to try this out tonight in Final Cut Pro with the mTracker 3D plugin.
- schappim 5y ago
  Nice! One tip, if you have Motion (which there is a good chance being a FCP guy, you do), you can import the USDZ file into Motion. You can download the high quality version of a model here[1]. [1] https://files.littlebird.com.au/Shared-Image-2022-01-02-18-15-42-yUKMSQ.png https://files.littlebird.com.au/Shared-Image-2022-01-02-18-1...
  - darkteflon 5y ago
    I do - thank you very much! Had no idea when I woke up this morning that I would be scanning my four year-old’s dinosaurs into it.
    - schappim 5y ago
      Nice! I have been "borrowing" the kid's toys too. With enough photos you can get some great shots (such as this one)[1]. Looks better as a USDZ, than a GLTF. [1] https://usdz.app/experiences/284-dragon https://usdz.app/experiences/284-dragon
tomdekan 5y ago
This looks great. Unfortunately the app is not working for me on an iPhone SE 2020. I am getting an ‘Error accessing your camera. Please check your permissions’. The permission to allow Hologrid to access my camera is correct in settings.
GistNoesis 5y ago
If you don't have an iphone, you can view he model by downloading them in the "options" drop-down menu to obtain the usdz file, which you can then use either with Pixar open-source suite https://github.com/PixarAnimationStudios/USD https://github.com/PixarAnimationStudios/USD or other software that support the format. Or you can use an online file converter like https://products.aspose.app/3d/en/conversion/usd-to-obj https://products.aspose.app/3d/en/conversion/usd-to-obj and see the result in Meshlab. Direct in-browser visualization is possible but https://products.aspose.app/3d/fr/viewer https://products.aspose.app/3d/fr/viewer worked but the scale was wrong and prevented to zoom enough. You can also view some high quality scan in the Apple's demo https://developer.apple.com/augmented-reality/quick-look/ https://developer.apple.com/augmented-reality/quick-look/ (Edit: The ones with the "Made with object Capture" label which this service is a front-end for https://developer.apple.com/augmented-reality/object-capture/ https://developer.apple.com/augmented-reality/object-capture... ).
- schappim 5y ago
  Yup, good tip. We gotta make it nicer for Android. We have a USDZ to GLTF (Chrome/Android) converter too. We should make the gltf more readily available.
wruza 5y ago
The dog is oriented in 3d somehow in a “_/_“ way, instead of lying on the floor. Side question: how the site has access to my camera, when it’s set to disabled in my ios-safari settings? Settings - Safari - Camera - Deny.
- schappim 5y ago
  Could be the difference between file upload (via camera) and media device access?
  - wruza 5y ago
    Testing it… I’m not sure. I tap on the 3d cube logo in the picture of an object, and then “camera screen” pops up, but it’s not the camera app, it’s safari, I see that in the apps list (by double clicking the home button). Nothing in downloads either. Very strange.
fareesh 5y ago
Is there something like neutral lighting where a texture can be photographed and then the engine's artificial lighting makes it look realistic post lighting?
- machiaweliczny 5y ago
  yes
  - fareesh 5y ago
    Where can I read more about this? What is the terminology for it?
    - machiaweliczny 5y ago
      I think I've seen it at top 10 vision papers 2021[0] [0]https://www.youtube.com/watch?v=8bHzytU1SeM https://www.youtube.com/watch?v=8bHzytU1SeM
    - glaslong 5y ago
      Generally for photogrammetry you want to aim for more diffuse lighting. Avoid direct light sources as they will cast shadows onto your subject that get baked into the texture. Cloudy days or light diffuser umbrellas are helpful. https://sketchfab.com/blogs/community/lighting-in-photogrammetry/ https://sketchfab.com/blogs/community/lighting-in-photogramm... There are methods for de-lighting, but it’s better to just capture without if possible. Or if you want to get fancy you can build a capture rig to create intentional shadows and process them into texture normals with something like Substance. https://store.substance3d.com/blog/your-smartphone-material-scanner https://store.substance3d.com/blog/your-smartphone-material-...
      - schappim 5y ago
        Further to this, I've made a video on how to get the best shots: https://usdz.app/screencasts/how-to-take-great-ar-object-captures-with-the-usdz-app https://usdz.app/screencasts/how-to-take-great-ar-object-cap...
abraxas 5y ago
I can downvote comments but not stories. Is there a karma threshold for downvoting stories? EDIT:Ugh, sorry about this comment. Looks like I somehow posted it under the wrong story. No clue how this happened. Looks like I'm unable to delete it as the deletion time window has expired.
- toomuchtodo 5y ago
  Stories can’t be downvoted.
- sokoloff 5y ago
  If you think it isn’t appropriate for the site, there’s a flag link. (I can’t imagine that applies here.)
hansoolo 5y ago
Thank you for making this! This is exactly the easy solution I was looking for a few days ago for a side project.

191 points by schappim 5y ago

Hi I'm Marcus (one of the creators). Feel free to hit me up if you have any questions :-) We are "very alpha" right now. Edit: We have an API over at: https://usdz.app/api https://usdz.app/api

genpfault 5y ago
What photogrammetry pipeline are you using?
- schappim 5y ago
  It is a stupidly long AWS based pipeline w/ the photogrammetry part is in swift.
- deleted 5y ago
  [deleted]
porkbrain 5y ago
If it's not a secret, I'd love to read some design document about the backend behind this app. What's the blackbox that takes set of images as an input and produces a 3D model as an output?
- schappim 5y ago
  We've very much built on the shoulders of giants: - Frontend & API (Rails + jumpstartrails.com) - Images --> USDZ (Swift Code on a Cloud hosted Mac) - USDZ --> GLTF (Swift + Custom Ruby) - CDN (AWS Cloudfront) Lots of Glue: - AWS S3 / Cloudfront - AWS SQS
  - deleted 5y ago
    [deleted]
- nobbis 5y ago
  It uses the Object Capture API introduced in the latest version of macOS. The API allows you to create 3D models from images with just a few lines of code. https://developer.apple.com/augmented-reality/object-capture https://developer.apple.com/augmented-reality/object-capture
  - schappim 5y ago
    If we're talking Swift frameworks, ModelIO (underrated) and ObjectCapture are certainly part of things.
    - nobbis 5y ago
      He asked for the “blackbox that takes set of images as an input and produces a 3D model as an output.” That black box is Object Capture API, which takes a folder of images and outputs an USDZ (or OBJ) file. Model I/O isn’t necessary. Kudos to Apple for releasing a photogrammetry pipeline that allows web and app developers to build tools like this, with zero knowledge of 3D reconstruction required.
      - schappim 5y ago
        You're not wrong (althought we do use ModelIO a lot). I just saw that you're from https://www.aboundlabs.com https://www.aboundlabs.com and metascan.ai Your scans look great! So how does your black box work?
      - secretsatan 5y ago
        I’ve just been trying out the photogrammetry session, it’s super simple to use. I think Apple’s photogrammetry software isn’t as advanced as other’s, but it is free and works well small scale. I work in the same field, making a capture app for photogrammetry software
        schappim 5y ago
        Yeah what Apple's has is a clear lead on the time required to build objects. I think that the photogrammetry side of things, the "give me an object now" will shift towards mobile device camera apps. It is only serverside now due to power requirements.
armagon 5y ago
Hi Marcus. It would be nice to have some idea going in for what the requirements are. - I have to admit, I don't understand why I can't preview the 3D experiences right on my computer. - Likewise, when I try the QR code on my iPhone 6 ... well, I'm hardly surprised to not see anything in 3D, but it wouldn't hurt to have a little more detail than "Scan this QR code on your iPhone or iPad." I suppose I could mention that I've done some game development and a little bit of VR/AR, but have never heard of the USDZ file format before.
- schappim 5y ago
  Hi there! USDZ is the format that Apple is backing for VR/AR delivery (developed by Pixar)[1]. We're betting that it is what is going to be used in any Apple headset. Right now you can't easily view USDZs in the browser (the format is a moving target), however Apple is making strides to make it available with the proposed Model element[2]. We were doing some USDZ --> GLTF converstion (GLTF being a format that will work in the browser), but it is jankey (technical term). I have not tried an iPhone 6 yet. I will have to pry it from my children's hands and remove a layer of cheese and excretions before I do. I've made a video on how to do this via the browser here[3]. TBH, the experience a heap better in the iOS App (currently in testflight) [4]. [1] https://graphics.pixar.com/usd/release/wp_usdz.html https://graphics.pixar.com/usd/release/wp_usdz.html (seem to have cert issues) [2] https://github.com/WebKit/explainers/tree/main/model https://github.com/WebKit/explainers/tree/main/model [3] https://usdz.app/screencasts/getting-started https://usdz.app/screencasts/getting-started [4] https://testflight.apple.com/join/VteIxFd7 https://testflight.apple.com/join/VteIxFd7
  - schappim 5y ago
    Oh, I should also mention, HEICs are your friend (as they have depth, and accelerometer data embedded in the file format).
nemacol 5y ago
Anyway to take a model that was created and get it into CAD / modeling software? I don't know much about VR/AR but I know it is a pain in the bum to replicate an object / create a model for 3d printing. Thank you.
- schappim 5y ago
  We allow you to download as USDZs (and sometimes do conversions to GLTFs). You can import both of these into Blender and export as Obj (and I'm pretty sure STLs). You may want to use our API to do the Images ---> USDZ conversion[1]. [1] https://usdz.app/api https://usdz.app/api
  - nemacol 5y ago
    Excellent. Thank you for the reply.
tluyben2 5y ago
This is really great, thanks for doing this!
ianbutler 5y ago
Hey very cool stuff, one piece of feedback it would be good to be able to delete models on the beta iPhone app, or if we can it’s not particularly obvious how to do so.
- schappim 5y ago
  You're right! Right now it is not possible (via the phone). It should be possible via the API or https://usdz.app https://usdz.app
thih9 5y ago
Pretty cool! How are you handling spam or other kinds of content moderation? Is it easier because users don’t upload a regular text/image and need to create something custom?
- schappim 5y ago
  There hasn't been anything humans can't handle. If it gets out of hand, we'll look at something similar to Akismet, or an ML based solution. Right now we're hand picking experiences to make it to the front page.
  - onara 5y ago
    At early stage you are not going to get spam anyway. Once it grows (and i hope it does) you will get fare share of spam. As you said at that point you could check solutions like Akismet, OOPSpam.
  - thih9 5y ago
    Thanks for answering! Hand picking sounds like a really good approach, simple and effective.
    - schappim 5y ago
      So funny you should mention this, yesterday we had our "first porn" moment. Thankfully if one throws a bunch of random images at the API and trys to make a model, the algorithm just fails. However we're thinking it might be fun to do something w/ NSFW image detection and display a silly model...
hpen 5y ago
Hey have you guys considered licensing an API?
- schappim 5y ago
  We don’t think is a good long term business model (with the obvious path of functionality moving to the client).
  - toomuchtodo 5y ago
    Smart. Looking forward to this being an on device capability.

granto 5y ago

This looks great, congrats! I’ve tried a number of photogrammetry software and they typically had challenges. One question: are you able to determine the size of the object?

coolmitch 5y ago

testflight app doesn't seem to be able to access my camera (iPhone SE2) despite having permissions

schappim 5y ago
Thanks for the report Mitch. We'll have to look into this.
- rckt 5y ago
  Having the same issue, using the SE 2nd too.

edoceo 5y ago

Would love to see some examples of how to go from this into something like A-frame - or even a guide on how it might work?

schappim 5y ago
Your wish is my command. - This is one I'm building for mobile (work in progress) [1] - This is one for desktop [2] [1] https://files.littlebird.com.au/mobile-how-to-wip-BDikJ9.m4v https://files.littlebird.com.au/mobile-how-to-wip-BDikJ9.m4v [2] https://usdz.app/screencasts/getting-started https://usdz.app/screencasts/getting-started

somishere 5y ago

Very Interesting project! esp. when compared to the swathe of junky or overly complex photogrammetry tools out there from a few years back (but mostly still polluting the app store) I've only recently been trying out poly.cam which is probably the best example of a user friendly workflow ... but it's knowingly priced. Have you guys got a pricing model in mind? Also do you have any documentation around best practices/results, etc? Anyway sounds like things are going to pick up again quickly with apple's new APIs. Good luck - exciting times ahead!

schappim 5y ago
Thanks for the tip! I'm making a "how-to" video now. I've yet to do the voiceover, but here are the raw slides: https://files.littlebird.com.au/take-great-shots-cXH2CM.m4v https://files.littlebird.com.au/take-great-shots-cXH2CM.m4v The TL;DR is: - Avoid thin objects - Avoid reflective objects - Avoid transparent objects - Avoid objects that deform - Hardshadows are not good, use soft lighting - Your photos should have a crisp focus and be well lit (across the object) - Things work best when the background has a solid colour - Make the object take up the full frame (more data for the algorithm) - Make sure to get shots from all sides
nobbis 5y ago
As far as best practices, Apple has some documentation on how to use their photogrammetry pipeline: https://developer.apple.com/documentation/realitykit/capturing_photographs_for_realitykit_object_capture https://developer.apple.com/documentation/realitykit/capturi...
- schappim 5y ago
  Hey Nobbis, I've just requested an invite to your SDK competitor to Apple and others. What you've done looks great and I have a few questions: - Does your solution have the same photogrammetry requirements as the Apple/others? - Is there a reason you're only supporting iOS at this time? Is that because you're using an iOS framework, or because you're relying on a file format (say HEIC)? - How long until you guys open up the API? Keep up the good work!

zuhayeer 5y ago

Very cool, reminds me a bit of the photogrammetry software Autodesk has. Always thought it could be more "consumerized" so that a series of photos can easily be made into 3D models. This was also a nice throwback for me to some drone photographs I've taken a while back at Berkeley stitched together into a 3d model: https://twitter.com/zuhayeer/status/933752635726880768?s=20 https://twitter.com/zuhayeer/status/933752635726880768?s=20

schappim 5y ago
Thanks for your kind words! Ultimately I think this is going to be part of the camera apps or some built-in app. It is just a function of power.

etaioinshrdlu 5y ago

How many photos does it take? I think an API for this would be ridiculously cool.

schappim 5y ago
You need at least 20-ish, 70-200 make amazing objects. There is a free API here: https://usdz.app/api https://usdz.app/api

aero-glide2 5y ago

Does this have android support?

schappim 5y ago
It will. We've been building the mobile app using a cross platform framework (Capacitor) with the view of making it Android compatible. Capacitor does the job, but we've had to do some native code to fill holes. If you're a dev, you can also use the API: https://usdz.app/api https://usdz.app/api You could probably whip up a quick script that runs on Android. My first go of doing mobile (for enlisting my co-founder) was to use Apple Shortcuts w/ the API.

darkteflon 5y ago

Looks Excellent! Going to try this out tonight in Final Cut Pro with the mTracker 3D plugin.

schappim 5y ago
Nice! One tip, if you have Motion (which there is a good chance being a FCP guy, you do), you can import the USDZ file into Motion. You can download the high quality version of a model here[1]. [1] https://files.littlebird.com.au/Shared-Image-2022-01-02-18-15-42-yUKMSQ.png https://files.littlebird.com.au/Shared-Image-2022-01-02-18-1...
- darkteflon 5y ago
  I do - thank you very much! Had no idea when I woke up this morning that I would be scanning my four year-old’s dinosaurs into it.
  - schappim 5y ago
    Nice! I have been "borrowing" the kid's toys too. With enough photos you can get some great shots (such as this one)[1]. Looks better as a USDZ, than a GLTF. [1] https://usdz.app/experiences/284-dragon https://usdz.app/experiences/284-dragon

tomdekan 5y ago

This looks great. Unfortunately the app is not working for me on an iPhone SE 2020. I am getting an ‘Error accessing your camera. Please check your permissions’. The permission to allow Hologrid to access my camera is correct in settings.

GistNoesis 5y ago

If you don't have an iphone, you can view he model by downloading them in the "options" drop-down menu to obtain the usdz file, which you can then use either with Pixar open-source suite https://github.com/PixarAnimationStudios/USD https://github.com/PixarAnimationStudios/USD or other software that support the format. Or you can use an online file converter like https://products.aspose.app/3d/en/conversion/usd-to-obj https://products.aspose.app/3d/en/conversion/usd-to-obj and see the result in Meshlab. Direct in-browser visualization is possible but https://products.aspose.app/3d/fr/viewer https://products.aspose.app/3d/fr/viewer worked but the scale was wrong and prevented to zoom enough. You can also view some high quality scan in the Apple's demo https://developer.apple.com/augmented-reality/quick-look/ https://developer.apple.com/augmented-reality/quick-look/ (Edit: The ones with the "Made with object Capture" label which this service is a front-end for https://developer.apple.com/augmented-reality/object-capture/ https://developer.apple.com/augmented-reality/object-capture... ).

schappim 5y ago
Yup, good tip. We gotta make it nicer for Android. We have a USDZ to GLTF (Chrome/Android) converter too. We should make the gltf more readily available.

wruza 5y ago

The dog is oriented in 3d somehow in a “_/_“ way, instead of lying on the floor. Side question: how the site has access to my camera, when it’s set to disabled in my ios-safari settings? Settings - Safari - Camera - Deny.

schappim 5y ago
Could be the difference between file upload (via camera) and media device access?
- wruza 5y ago
  Testing it… I’m not sure. I tap on the 3d cube logo in the picture of an object, and then “camera screen” pops up, but it’s not the camera app, it’s safari, I see that in the apps list (by double clicking the home button). Nothing in downloads either. Very strange.

fareesh 5y ago

Is there something like neutral lighting where a texture can be photographed and then the engine's artificial lighting makes it look realistic post lighting?

machiaweliczny 5y ago
yes
- fareesh 5y ago
  Where can I read more about this? What is the terminology for it?
  - machiaweliczny 5y ago
    I think I've seen it at top 10 vision papers 2021[0] [0]https://www.youtube.com/watch?v=8bHzytU1SeM https://www.youtube.com/watch?v=8bHzytU1SeM
  - glaslong 5y ago
    Generally for photogrammetry you want to aim for more diffuse lighting. Avoid direct light sources as they will cast shadows onto your subject that get baked into the texture. Cloudy days or light diffuser umbrellas are helpful. https://sketchfab.com/blogs/community/lighting-in-photogrammetry/ https://sketchfab.com/blogs/community/lighting-in-photogramm... There are methods for de-lighting, but it’s better to just capture without if possible. Or if you want to get fancy you can build a capture rig to create intentional shadows and process them into texture normals with something like Substance. https://store.substance3d.com/blog/your-smartphone-material-scanner https://store.substance3d.com/blog/your-smartphone-material-...
    - schappim 5y ago
      Further to this, I've made a video on how to get the best shots: https://usdz.app/screencasts/how-to-take-great-ar-object-captures-with-the-usdz-app https://usdz.app/screencasts/how-to-take-great-ar-object-cap...

abraxas 5y ago

I can downvote comments but not stories. Is there a karma threshold for downvoting stories? EDIT:Ugh, sorry about this comment. Looks like I somehow posted it under the wrong story. No clue how this happened. Looks like I'm unable to delete it as the deletion time window has expired.

toomuchtodo 5y ago
Stories can’t be downvoted.
sokoloff 5y ago
If you think it isn’t appropriate for the site, there’s a flag link. (I can’t imagine that applies here.)

hansoolo 5y ago

Thank you for making this! This is exactly the easy solution I was looking for a few days ago for a side project.