4 ms·
Certainly a lot of scope for this kind of thing .. people who do lidar scans or photogrammetry of buildings tend to end up with very large meshes or very large
by jgord 2y ago
Certainly a lot of scope for this kind of thing .. people who do lidar scans or photogrammetry of buildings tend to end up with very large meshes or very large point clouds.. which means they need souped up PCs and expensive software to wrangle into some usable CAD format.
Its an area where things can be improved a lot imho - I did some work a while back fitting flat planes to pointclouds, and ended up with mesh model anything from 40x to 100x smaller data than the ptcloud dataset. see quato.xyz for samples where you can compare the cloud, the mesh produced.. and view the 3D model in recent browsers.
My approach had some similarity to gaussian splats... but using only planar regions .. great for buildings made of flat slabs, less so for smooth curves and foliage.
Applying their MeshAnything algo to fine meshes from photogrammetry scans of buildings would be of great benefit - probably getting those meshes down to a size where they can be shared as 3D webgl/threejs pages.
Even deciding on triangle points to efficiently tesselate / cover a planar region with holes etc, is basically a knapsack problem, which heuristics, monte-carlo and ML can improve upon.
- spookie 2y agoCould you go more in depth with your approach? Sounds really usefull for man made structures!
- jgord 2y agoahh.. looking for investment, so reticent to talk about the internals. Basically does a lot of matmulls and sampling to find good good planar fits... Most of these sample datasets took an hour to compute, using 12 threads on a midrange CPU... should be doable in seconds if ported to a highend GPU.
- librasteve 2y agosuggest a nanopivot, make it work on Apple AI (if you haven’t already) and position as iPhone AI mesh
- spookie 2y agoUnderstandable! Thanks for what you've shared. I'm doing academic work on something that could leverage a digital twin, hence my interest. There are many uses for this tech, particularly in less techy crowds that still make significant use of traditional photogrammetry. Your solution, if local, could give a significant advantage over other products such as Polycam. Again, if local, you could allow for much bigger scans (wink for those doing architecture, particularly those in the restoration field). Anyhow, hope you get that funding!
- fsloth 2y agoIf you want to show photogrammetric pointclouds of buildings potree db&algorithm is pretty good and if you don’t like the library for some reason it’s pretty easy to reimplement (potree.org). You just dump the pointcloud to a hierarchical octree and at viewer end just download the nodes in your frusttum and voila. There are other approaches but this wins hands down on usability/simplicity.
- jgord 2y agoIm quite familiar with potree and a big fan .. having hacked some of the internals and added features to my own custom version - so people can share annotations, measurements, save to cloud or export linework .. without writing code/custom html. Also added code to import e57 cube panoramas .. Still, I think if one can use ML to simplify a pointcloud or fine mesh .. then the data is much smaller and cleaner, easier to import to existing CAD tools etc.
- toxik 2y agoSurely you heard about surfels, sounds like a very similar approach.
- jgord 2y agoLooked up surfels .. I think they are different from fitting local planes - I was interested in the geometry, not the lighting. But surfels seems like a clever idea, pre-dating view dependent NeRFs.