3 ms·
Apple's Cubify Anything: Scaling Indoor 3D Object Detection
- desertmonad 2y agoLooks promising but the license, Attribution-NonCommercial-NoDerivatives is pretty limiting..
- syntaxing 2y agoSurprised this isn’t in coreML. Seems useful for the Vision Pro or something
- hokumguru 2y agoMight see it at WWDC this year?
- fidotron 2y agoThe accuracy of the results don't seem that great. For example, looking at the pictures on the wall in their sample, or the beams in the ceiling. It's possible it's some artifact of the processing resolution, but I think most people that have worked with NNs for AR input will be surprised that this is not considered disappointing.
- ellisv 2y ago> The accuracy of the results don't seem that great. For example, looking at the pictures on the wall in their sample, or the beams in the ceiling. Do you mean the accuracy of the classification or the precision of the lidar scans? In my experience the lidar precision on the iPhones is decent but not great, so the texture mapping can look a bit off at times. I'd love to have these bounding boxes on my scans though.
- fidotron 2y agoI mean the accuracy with which it's locating the bounds. What is extra curious is it obviously supports rotated cubes, yet it often doesn't use them when it should, leading to overstating the bounds, as if it's over enthusiastically trying to put things aligned to some inferred axis. This is obviously an attempt at the general case to apply cubes to anything, but what is disappointing is the performance on boxy objects is lower than I've seen on private NNs used for AR and CV for years (ironically enough on iPads), using just rgb and no depth. I half think the exercise here was to establish if transformers were the way to go for this, and on the strength of that the answer would be probably not.
- pablogancharov 2y agoIn case anyone is interested in rendering USDZ scans in Three.js, I created a demo: https://usdz-threejs-viewer.vercel.app/ https://usdz-threejs-viewer.vercel.app/
- mhuffman 2y agoVery nice and smooth! Do you have source for your demo?
- callumprentice 2y agoThere is one in the Three.JS example suite with source: https://threejs.org/examples/webgl_loader_usdz.html https://threejs.org/examples/webgl_loader_usdz.html
- fooblaster 2y agoFyi, looks mighty black and white on a pixel 6a.
- dev_john15 2y ago[flagged]
- tech_jane18 2y ago[flagged]
- deleted 2y ago[deleted]
- callumprentice 2y agoI keep meaning to get back to my suite of equirectangular image functions - viewers, editors, authoring etc. and this reminded me to resurrect the Viewer. https://equinaut.surge.sh/?eqr=https://raw.githubusercontent.com/apple/ml-cubifyanything/refs/heads/main/teaser.jpg https://equinaut.surge.sh/?eqr=https://raw.githubusercontent... Not quite right I think because the source image issn't 2x1 aspect ratio. They can look really nice: both in the real world - https://equinaut.surge.sh/?eqr=https://upload.wikimedia.org/wikipedia/commons/thumb/0/08/Laon_Cathedral_Interior_360x180%2C_Picardy%2C_France_-_Diliff.jpg/2560px-Laon_Cathedral_Interior_360x180%2C_Picardy%2C_France_-_Diliff.jpg https://equinaut.surge.sh/?eqr=https://upload.wikimedia.org/... or the virtual world: https://equinaut.surge.sh/?eqr=https://live.staticflickr.com/65535/51954915010_4c652acde8_o_d.jpg https://equinaut.surge.sh/?eqr=https://live.staticflickr.com...
- Svip 2y agoWill it work on a picture of a Power Mac G4 Cube[0]? Whenever I see "cube" and "apple" together (which, in fairness, is rare), I think of the Cube. [0] https://en.wikipedia.org/wiki/Power_Mac_G4_Cube https://en.wikipedia.org/wiki/Power_Mac_G4_Cube
- pzo 2y agoThey overcomplicate by using 3-4 different (sub) license in one project: in README: Licenses - The sample code is released under Apple Sample Code License. - The data is released under CC-by-NC-ND. - The models are released under Apple ML Research Model Terms of Use. Acknowledgements - We use and acknowledge contributions from multiple open-source projects in ACKNOWLEDGEMENTS." then having in github license button "Copyright (C) 2025 Apple Inc. All Rights Reserved." in repo file LICENSE LICENSE_MODEL why making it so confusing and elaborate? Its so useless to even use by 3rd party devs for making apps and releasing on their platform. So then just make it one license with the most strict restrictions you can make AGPL and/or CC-by-NC-ND .
- brookst 2y agoThey could have transformed it from insane to sublime by slapping a highly restrictive license on the readme itself. Seriously missed opportunity.
- guipsp 2y agoIt complicated, but it's not overcomplicated. CC is not adequate for code and I belive that none of the code is GPL so your suggestion regarding AGPL is strange.
- generalizations 2y agoWhy isn't CC-by-NC-ND adequate for code? Kinda makes sense IMO and the summary looks useful? > CC-BY-NC-ND is a type of Creative Commons license that allows others to use a work non-commercially, but they cannot modify it or create derivative works. This means the original work can be shared, but it must remain unchanged and cannot be used for commercial purposes. Notwithstanding it's only applied to the data in this case, it sure looks like a useful license for code.
- tpmoney 2y ago> Why isn't CC-by-NC-ND adequate for code? Kinda makes sense IMO and the summary looks useful? Because the Creative Commons folks themselves say it’s not because it doesn’t cover a number of software specific edge cases.
- totetsu 2y agoIs this so your smart speaker can better report whats in your house back to apple?
- Carrok 2y agoI really want an app I can scan my whole house with the camera/lidar combo on my phone, and export it into Blender, where I can then rearrange furniture and stuff. Apps like Scaniverse get you pretty close, but everything is one mesh, would be great to be able to slide the couch around the space without having the manually cut it out of the mesh.