8 ms·

Turn Photos into 3D Models

100 points by omarfarooq 5y ago

waynecochran 5y ago
3D model flavor? Point cloud, triangle/quad mesh? Texture map + normal map?
- kingludite 5y ago
  The way the story ends with "then you export it" has me somewhat confused.
CharlesW 5y ago
Based on the language used, my guess is that this app is a front-end for Apple's macOS Object Capture API: https://developer.apple.com/augmented-reality/object-capture/ https://developer.apple.com/augmented-reality/object-capture... In any case, this article has links to sample code in case you want to create a similar app: https://taeeun-kim.medium.com/how-to-create-3d-models-with-object-capture-9b3d1226bd10 https://taeeun-kim.medium.com/how-to-create-3d-models-with-o...
thot_experiment 5y ago
If you can, don't fuck with this shit and use/support open source software. https://alicevision.org/#meshroom https://alicevision.org/#meshroom
- KarlKemp 5y ago
  Yes, but the step-by-step guide has eight steps. And the first step cites five papers. But at no point (at whatever depth) does it tell me what to do.
  - thot_experiment 5y ago
    I guess you're not in the "If you can" category then, seems basic reading comprehension is a fading art. I for one appreciate that the tutorial attempts to help you understand how the process works and why. Perhaps it's just my enormous brain at work but I found that when I opened the program up the "drag your images here" space and "start" button were pretty self-explanatory. (please read this with dripping sarcasm and imagine me rolling my eyes very hard) On a more serious note I really think meshroom has done a great job encapsulating a complicated process in a way that is simple to execute on the outset and yet affords the end user a great deal of control over what's going on. I really like the way they visualize the process with the node graph and that they cache the results at every step so you only ever have to recompute changes downstream of the node you've edited. I love tools that let me solve problems the toolmakers didn't specifically envision.
    - KarlKemp 5y ago
      I'll take the attempted insult. But at no point do you approach anything that would fall under the definition of "sarcasm", unless you're trying to agree with me. You're so far away from even mere irony that there will probably be rain or your wedding day (don't remember the song). Call me a cynic (it's in the dictionary), but I used to be insulted by smarter people. I shall have to think about that. Anyway... I've actually done photogrammetry (with the Apple API– wish I could share my excellent rubber ducky, but Wikimedia Commons doesn't quite like my file format). I was mostly just pointing out that I had different expectations from a "step-by-step guide", and that OP's wish of OSS adoption might be more effectively supported by making them want to do so, rather than shaming them with slogans from the mid-90s. Sometimes, a more powerful product is more complicated. But the same people who will eagerly post "correlation is not causation" on every science story as if that weren't the first thing any scientist learnt will happily pretend that something being difficult to use is evidence of and necessary for a more "professional" product.
      - thot_experiment 5y ago
        You're right that I'd definitely be doing more good if I wasn't being an dick about it, sorry. Though probably I shouldn't have engaged anyway. However, I am still drinking so despite my better judgement let's get into it on the facts of the matter at hand. (dang please forgive me for degrading the quality of HN) > mostly just pointing out that I had different expectations from a "step-by-step guide" Let's not pretend you were doing your best to communicate this in good faith with your original comment, but I shouldn't have been insulting, again, sorry. However you absolutely touched a nerve because as someone who's made it a point the last few years to try to switch to using only OSS Meshroom really does a comparatively bangup job of usability. > Yes, but the step-by-step guide has eight steps. Sure the guide could be written better, but it does a good job of explaining the process and a great deal of related minutia. > And the first step cites five papers. Come on now, if this isn't meant to paint it as some sort of inapproachable academic process (which is not the case at all) why mention it. > But at no point (at whatever depth) does it tell me what to do. Step Three absolutely clearly(ish) tells you what to do, and that step alone is enough to get a decent result. Yeah the formatting could be better, and yes there could be a quick start guide/TLDR that says "take a bunch of photos from a lot of angles without moving the object, drag them in and hit start" but come on, if you read through it it definitely does tell you what to do, hence the reading comprehension comment. Regarding sarcasm, certainly at least the bit about "my enormous brain" fits the definition.
  - Etheryte 5y ago
    I think the disconnect here is that you seem to think the step-by-step is a tutorial on how to use AliceVision, it isn't. It's a high level explanation of how the photogrammetry pipeline works. If it helps, you can think of it like a pop-sci article covering scientific research. If you're looking for a tutorial on how to use the tooling built around this pipeline, you can start with [0], found by clicking Meshroom in the menu and then the tutorial link. [0] https://meshroom-manual.readthedocs.io/en/latest/tutorials/sketchfab/sketchfab.html https://meshroom-manual.readthedocs.io/en/latest/tutorials/s...
  - plorg 5y ago
    I tried installing Meshroom and it felt pretty rough, but it turns out I had simply failed to restart my Linux box after installing the CUDA toolbox. The software seems stable, though I'm still messing around trying to better understand it. My initial attempts with the default settings and a handful of photos of my mouse could use some work, and it's pretty slow on my box (an aging Xeon and about the oldest graphics card that's compatible with CUDA for this software). But it's interesting stuff, and better polished than a lot of research software I've worked with.
- sixothree 5y ago
  The thing that immediately turned me off about it is the fact that it is a phone app. Mobile apps all just feel like creepy spyware to me lately. "Send us photos of the things you are working on". Creepy weirdos is all I can think.
  - amelius 5y ago
    I tried to install Acrobat reader on Android. It asked access to: - In app purchases (ok, I can pay, but only once please; not a fan of subscriptionware) - Identity (what for?) - Contacts (what for?) - Photos/Media/Files (Ok, I can understand it, but can I please allow only certain folders?) - Camera (again, what for???) - Wi-Fi connection information (what for?) Since it is all-or-nothing, I had to decline. And now I'm looking for another pdf reader without ads. This whole mobile shitshow is driving me nuts.
    - Geonode 5y ago
      My favorite is creative cloud and Autodesk desktop app both having several processes open all the time which reopen if you close them.
      - amelius 5y ago
        Recently found out that my Moto G4 phone is running an Android version of 4 years ago. It didn't run any security updates since, and without warning. And I can't upgrade. Phone hardware is perfectly fine otherwise. Google should be ashamed of themselves!
        fooblaster 5y ago
        This isn't really directly Google's fault. hardware vendors refuse to support bsps for older hardware, much of which is closed source binaries that Google cannot maintain. You might notice the support window for pixel hardware that doesn't use Qualcomm socs is significantly longer.
        deleted 5y ago
        [deleted]
- moffkalast 5y ago
  Yeah was about to say "so.. meshroom but restricted to iphones only?". No thanks.
- kmlx 5y ago
  all for open source, but these are two different products. from their website: > To fully utilize Meshroom, a NVIDIA CUDA-enabled GPU is recommended. The binaries are built with CUDA-10 and are compatible with compute capability 3.0 to 7.5. Without a supported NVIDIA GPU, only "Draft Meshing" can be used for 3D reconstruction. versus an easy to use iphone app.
  - Aardwolf 5y ago
    Somewhat off topic from the article, but I'm curious about GPU vs CPU computing here: So it looks like meshroom disables a feature (or features) when not having a Nvidia CUDA-enabled GPU, rather than e.g. still have the feature, but have it be slower. Any idea, if it were computed (or emulated) on CPU, how much slower it would be? (like 10x slower, 100x slower, 1000x slower, even more?!) And also, ignoring speed but instead looking at implementation effort, if it's feasible to emulate on CPU at all, or is GPU really so different for this that it's too much work to implement the same thing for a general purpose processor? At least matrix multiplication libraries (the main thing GPUs can do well afaik) exist for CPU too...
    - dr_zoidberg 5y ago
      From my experience with deep learning I can eyeball it: one or two orders of magnitude. The fine details may be different on this particular problem. In DL in general, if you have sort of equivalent CPU/GPUs (think a 5600 vs GeForce 3070) you'll find yourself in the range of 10 to 30x faster training with GPU. If you have substantially different platforms, you can start looking at 100x speedups or more. For example, my mobile 4600H CPU trains about 100x slower than a desktop GeForce 2060. And close to 300x faster than the old Intel 7200u I use as a "slow system" to test things. Which also has a GeForce 940MX, that trains about 30x slower than the 2060 (if you manage to make your model small enough to fit in 2GB of VRAM).
causality0 5y ago
I want one of these 3D photo apps but geared for object replication rather than art projects and videogames. Like let me scan an object then add measurements to it that let me more perfectly recreate it, such as identifying curves for me to input a measured radius, asking whether this section is supposed to be a straight edge or a flat plain or a perfect cylinder.
- SV_BubbleTime 5y ago
  My experience with professional solutions a few years ago was that it worked well for large areas but the tolerances weren’t anywhere near good enough. A “trick” seems to be that it looks great with the photos overlayed as textures, but the actual 3D data without that isn’t nearly as impressive.
  - causality0 5y ago
    Exactly. I feel like AI could do a great job with this by starting with a rough scan and then asking the user intelligent questions about what certain shapes and measurements are.
schappim 5y ago
Looks like the same tech we used at https://usdz.app https://usdz.app . It will be very interesting to see what new APIs Apple brings out this WWDC. Hopefully the photogrammetry API makes its way from macOS to iOS. In the mean time, if someone wanted to make their own app, they could use: https://usdz.app/api https://usdz.app/api
totalview 5y ago
How many of these apps do we need before people stop thinking this is a new and novel idea?
- flor1s 5y ago
  You also have a pretty heavyweight competitor with Epic Games (RealityScan).
weird-eye-issue 5y ago
Next I want something to turn photos into music
darylteo 5y ago
Great. I look forward to the 100 more queries about "can we add this to our app".
madduci 5y ago
Seems like it's a hot topic and there are a lot of clones lately in the App Store. And probably Nvidia after its latest demo is disrupting the market again. The only one that I think offers a cool demo/use case is Holograms.
- madduci 5y ago
  *Volograms
gandalfian 5y ago
Take photos in a circle around object sounds suspiciously like object vr. Usefull but old. But it also suggests it may be using an iPhones lidar face sensors to scan the object which would be quite cool. There are no obvious examples though? and I don't have an iPhone with face recognition so who knows. Rather vague on prices to export models?
boriskourt 5y ago
I've been using 3D Scanner App [0] for almost two years now. And it has been free and very feature rich this whole time. So many very helpful export options, as well as a nice frontend. I've also made use of getting the raw data here and there. Its hard for me to tell what this adds over that. And like another comment points out, this looks like a thin wrapper over the tools already provided by Apple. [0]: https://apps.apple.com/us/app/3d-scanner-app/id1419913995 https://apps.apple.com/us/app/3d-scanner-app/id1419913995
- lorenzhs 5y ago
  The App Store page says "Simply take photos of an object from all angles and upload them to our service, and we will create and send you a 3D model ready for 3D printing, Augmented Reality and our web app r3Dent." so not it appears to use some kind of server-side processing. So it's not even on-device.
- ant6n 5y ago
  Too bad it’s iPhone 12 and iPad Pro only.
mdrzn 5y ago
This is DEFINITELY not free but there's no info on the whole website about prices. I'm not gonna download and figure out.

100 points by omarfarooq 5y ago

waynecochran 5y ago

3D model flavor? Point cloud, triangle/quad mesh? Texture map + normal map?

kingludite 5y ago
The way the story ends with "then you export it" has me somewhat confused.

CharlesW 5y ago

Based on the language used, my guess is that this app is a front-end for Apple's macOS Object Capture API: https://developer.apple.com/augmented-reality/object-capture/ https://developer.apple.com/augmented-reality/object-capture... In any case, this article has links to sample code in case you want to create a similar app: https://taeeun-kim.medium.com/how-to-create-3d-models-with-object-capture-9b3d1226bd10 https://taeeun-kim.medium.com/how-to-create-3d-models-with-o...

thot_experiment 5y ago

If you can, don't fuck with this shit and use/support open source software. https://alicevision.org/#meshroom https://alicevision.org/#meshroom

KarlKemp 5y ago
Yes, but the step-by-step guide has eight steps. And the first step cites five papers. But at no point (at whatever depth) does it tell me what to do.
- thot_experiment 5y ago
  I guess you're not in the "If you can" category then, seems basic reading comprehension is a fading art. I for one appreciate that the tutorial attempts to help you understand how the process works and why. Perhaps it's just my enormous brain at work but I found that when I opened the program up the "drag your images here" space and "start" button were pretty self-explanatory. (please read this with dripping sarcasm and imagine me rolling my eyes very hard) On a more serious note I really think meshroom has done a great job encapsulating a complicated process in a way that is simple to execute on the outset and yet affords the end user a great deal of control over what's going on. I really like the way they visualize the process with the node graph and that they cache the results at every step so you only ever have to recompute changes downstream of the node you've edited. I love tools that let me solve problems the toolmakers didn't specifically envision.
  - KarlKemp 5y ago
    I'll take the attempted insult. But at no point do you approach anything that would fall under the definition of "sarcasm", unless you're trying to agree with me. You're so far away from even mere irony that there will probably be rain or your wedding day (don't remember the song). Call me a cynic (it's in the dictionary), but I used to be insulted by smarter people. I shall have to think about that. Anyway... I've actually done photogrammetry (with the Apple API– wish I could share my excellent rubber ducky, but Wikimedia Commons doesn't quite like my file format). I was mostly just pointing out that I had different expectations from a "step-by-step guide", and that OP's wish of OSS adoption might be more effectively supported by making them want to do so, rather than shaming them with slogans from the mid-90s. Sometimes, a more powerful product is more complicated. But the same people who will eagerly post "correlation is not causation" on every science story as if that weren't the first thing any scientist learnt will happily pretend that something being difficult to use is evidence of and necessary for a more "professional" product.
    - thot_experiment 5y ago
      You're right that I'd definitely be doing more good if I wasn't being an dick about it, sorry. Though probably I shouldn't have engaged anyway. However, I am still drinking so despite my better judgement let's get into it on the facts of the matter at hand. (dang please forgive me for degrading the quality of HN) > mostly just pointing out that I had different expectations from a "step-by-step guide" Let's not pretend you were doing your best to communicate this in good faith with your original comment, but I shouldn't have been insulting, again, sorry. However you absolutely touched a nerve because as someone who's made it a point the last few years to try to switch to using only OSS Meshroom really does a comparatively bangup job of usability. > Yes, but the step-by-step guide has eight steps. Sure the guide could be written better, but it does a good job of explaining the process and a great deal of related minutia. > And the first step cites five papers. Come on now, if this isn't meant to paint it as some sort of inapproachable academic process (which is not the case at all) why mention it. > But at no point (at whatever depth) does it tell me what to do. Step Three absolutely clearly(ish) tells you what to do, and that step alone is enough to get a decent result. Yeah the formatting could be better, and yes there could be a quick start guide/TLDR that says "take a bunch of photos from a lot of angles without moving the object, drag them in and hit start" but come on, if you read through it it definitely does tell you what to do, hence the reading comprehension comment. Regarding sarcasm, certainly at least the bit about "my enormous brain" fits the definition.
- Etheryte 5y ago
  I think the disconnect here is that you seem to think the step-by-step is a tutorial on how to use AliceVision, it isn't. It's a high level explanation of how the photogrammetry pipeline works. If it helps, you can think of it like a pop-sci article covering scientific research. If you're looking for a tutorial on how to use the tooling built around this pipeline, you can start with [0], found by clicking Meshroom in the menu and then the tutorial link. [0] https://meshroom-manual.readthedocs.io/en/latest/tutorials/sketchfab/sketchfab.html https://meshroom-manual.readthedocs.io/en/latest/tutorials/s...
- plorg 5y ago
  I tried installing Meshroom and it felt pretty rough, but it turns out I had simply failed to restart my Linux box after installing the CUDA toolbox. The software seems stable, though I'm still messing around trying to better understand it. My initial attempts with the default settings and a handful of photos of my mouse could use some work, and it's pretty slow on my box (an aging Xeon and about the oldest graphics card that's compatible with CUDA for this software). But it's interesting stuff, and better polished than a lot of research software I've worked with.
sixothree 5y ago
The thing that immediately turned me off about it is the fact that it is a phone app. Mobile apps all just feel like creepy spyware to me lately. "Send us photos of the things you are working on". Creepy weirdos is all I can think.
- amelius 5y ago
  I tried to install Acrobat reader on Android. It asked access to: - In app purchases (ok, I can pay, but only once please; not a fan of subscriptionware) - Identity (what for?) - Contacts (what for?) - Photos/Media/Files (Ok, I can understand it, but can I please allow only certain folders?) - Camera (again, what for???) - Wi-Fi connection information (what for?) Since it is all-or-nothing, I had to decline. And now I'm looking for another pdf reader without ads. This whole mobile shitshow is driving me nuts.
  - Geonode 5y ago
    My favorite is creative cloud and Autodesk desktop app both having several processes open all the time which reopen if you close them.
    - amelius 5y ago
      Recently found out that my Moto G4 phone is running an Android version of 4 years ago. It didn't run any security updates since, and without warning. And I can't upgrade. Phone hardware is perfectly fine otherwise. Google should be ashamed of themselves!
      - fooblaster 5y ago
        This isn't really directly Google's fault. hardware vendors refuse to support bsps for older hardware, much of which is closed source binaries that Google cannot maintain. You might notice the support window for pixel hardware that doesn't use Qualcomm socs is significantly longer.
      - deleted 5y ago
        [deleted]
moffkalast 5y ago
Yeah was about to say "so.. meshroom but restricted to iphones only?". No thanks.
kmlx 5y ago
all for open source, but these are two different products. from their website: > To fully utilize Meshroom, a NVIDIA CUDA-enabled GPU is recommended. The binaries are built with CUDA-10 and are compatible with compute capability 3.0 to 7.5. Without a supported NVIDIA GPU, only "Draft Meshing" can be used for 3D reconstruction. versus an easy to use iphone app.
- Aardwolf 5y ago
  Somewhat off topic from the article, but I'm curious about GPU vs CPU computing here: So it looks like meshroom disables a feature (or features) when not having a Nvidia CUDA-enabled GPU, rather than e.g. still have the feature, but have it be slower. Any idea, if it were computed (or emulated) on CPU, how much slower it would be? (like 10x slower, 100x slower, 1000x slower, even more?!) And also, ignoring speed but instead looking at implementation effort, if it's feasible to emulate on CPU at all, or is GPU really so different for this that it's too much work to implement the same thing for a general purpose processor? At least matrix multiplication libraries (the main thing GPUs can do well afaik) exist for CPU too...
  - dr_zoidberg 5y ago
    From my experience with deep learning I can eyeball it: one or two orders of magnitude. The fine details may be different on this particular problem. In DL in general, if you have sort of equivalent CPU/GPUs (think a 5600 vs GeForce 3070) you'll find yourself in the range of 10 to 30x faster training with GPU. If you have substantially different platforms, you can start looking at 100x speedups or more. For example, my mobile 4600H CPU trains about 100x slower than a desktop GeForce 2060. And close to 300x faster than the old Intel 7200u I use as a "slow system" to test things. Which also has a GeForce 940MX, that trains about 30x slower than the 2060 (if you manage to make your model small enough to fit in 2GB of VRAM).

causality0 5y ago

I want one of these 3D photo apps but geared for object replication rather than art projects and videogames. Like let me scan an object then add measurements to it that let me more perfectly recreate it, such as identifying curves for me to input a measured radius, asking whether this section is supposed to be a straight edge or a flat plain or a perfect cylinder.

SV_BubbleTime 5y ago
My experience with professional solutions a few years ago was that it worked well for large areas but the tolerances weren’t anywhere near good enough. A “trick” seems to be that it looks great with the photos overlayed as textures, but the actual 3D data without that isn’t nearly as impressive.
- causality0 5y ago
  Exactly. I feel like AI could do a great job with this by starting with a rough scan and then asking the user intelligent questions about what certain shapes and measurements are.

schappim 5y ago

Looks like the same tech we used at https://usdz.app https://usdz.app . It will be very interesting to see what new APIs Apple brings out this WWDC. Hopefully the photogrammetry API makes its way from macOS to iOS. In the mean time, if someone wanted to make their own app, they could use: https://usdz.app/api https://usdz.app/api

totalview 5y ago

How many of these apps do we need before people stop thinking this is a new and novel idea?

flor1s 5y ago
You also have a pretty heavyweight competitor with Epic Games (RealityScan).

weird-eye-issue 5y ago

Next I want something to turn photos into music

darylteo 5y ago

Great. I look forward to the 100 more queries about "can we add this to our app".

madduci 5y ago

Seems like it's a hot topic and there are a lot of clones lately in the App Store. And probably Nvidia after its latest demo is disrupting the market again. The only one that I think offers a cool demo/use case is Holograms.

madduci 5y ago
*Volograms

gandalfian 5y ago

Take photos in a circle around object sounds suspiciously like object vr. Usefull but old. But it also suggests it may be using an iPhones lidar face sensors to scan the object which would be quite cool. There are no obvious examples though? and I don't have an iPhone with face recognition so who knows. Rather vague on prices to export models?

boriskourt 5y ago

I've been using 3D Scanner App [0] for almost two years now. And it has been free and very feature rich this whole time. So many very helpful export options, as well as a nice frontend. I've also made use of getting the raw data here and there. Its hard for me to tell what this adds over that. And like another comment points out, this looks like a thin wrapper over the tools already provided by Apple. [0]: https://apps.apple.com/us/app/3d-scanner-app/id1419913995 https://apps.apple.com/us/app/3d-scanner-app/id1419913995

lorenzhs 5y ago
The App Store page says "Simply take photos of an object from all angles and upload them to our service, and we will create and send you a 3D model ready for 3D printing, Augmented Reality and our web app r3Dent." so not it appears to use some kind of server-side processing. So it's not even on-device.
ant6n 5y ago
Too bad it’s iPhone 12 and iPad Pro only.

mdrzn 5y ago

This is DEFINITELY not free but there's no info on the whole website about prices. I'm not gonna download and figure out.