7 ms·
> This looks amazing! Congratulations. Thank you! > What was the most challenging aspect of this? Wow, that's hard to say! Our work truly stands on the shoul
by duckworthd 6y ago
> This looks amazing! Congratulations.
Thank you!
> What was the most challenging aspect of this?
Wow, that's hard to say! Our work truly stands on the shoulders of giants (Mildenhall et al, 2020). I can list off a few challenges:
figuring out if an idea "kinda works" or "definitely works" or has a bug,
figuring out how to measure progress,
coordinating a group of 6 researchers living 9 hours apart,
and assembling everything together for a simultaneous paper-website-video release!
> I'm curious to see how you performed edge detection on the transient objects and were able to isolate them so cleanly.
We don't! All of this comes by the magic of machine learning :). We train our model to attentuate the importance of "difficult" pixels that aren't 3D-consistent in the training set. We also partition the scene into "static" and "transient" volumetric radiance fields without explicit supervision. We do so by regularizing the latter to be empty unless necessary, and providing it with access to a learned, image-specific latent embedding. We discard the transient radiance field when rendering these videos, thus removing tourists and other moving objects.
> For some reason, the paper isn't loading, so feel free to say it's explained in detail there.
Well that won't do. Download it here: https://arxiv.org/abs/2008.02268 https://arxiv.org/abs/2008.02268. It's 40 MB, so your download bar may indicate it's almost done, but it actually has a good bit left to go.
> Do you plan on releasing code?
I hope so! As with most code, what ran on our machines may not run on yours. Migrating the code to open source will be a big effort. I hope what we describe in the paper is sufficient to build something like you see here.
- alacombe 6y ago> Migrating the code to open source will be a big effort. No, it is not. It's 5 shell command at the most. $ git init $ git add . $ git commit -m 'Initial import' $ git remote add origin git://... $ git push origin master
- Jaruzel 6y agoYou are assuming there that all their code is non-proprietary, doesn't belong to someone else, already has a open-licence, and is of good enough quality to be shared with the public.
- alacombe 6y ago> their code is non-proprietary, doesn't belong to someone else, already has a open-licence Then say "we can't share for legal reason", not "we're planning to". This is just a bs corporate answer. > is of good enough quality to be shared with the public. This is a petty excuse. There is plenty of open-source code utterly crappy/barely functional out there.
- Jaruzel 6y ago> There is plenty of open-source code utterly crappy/barely functional out there. Mine included ;)