4 ms·
From my understanding, traditional photogrammetry typically generates 3d point clouds from image pixels by correlating visual features between images with known
by zanecodes 2y ago
From my understanding, traditional photogrammetry typically generates 3d point clouds from image pixels by correlating visual features between images with known camera parameters, allowing the camera pose of each image to be estimated in a shared coordinate space. These point clouds are postprocessed to estimate closed surfaces which can then be converted into textured triangle meshes to be rendered using traditional 3d rasterization techniques.
Gaussian splatting represents a scene a cloud of 3d gaussian ellipsoids, with direction-dependent color components (usually represented using spherical harmonics) to deal with effects like reflections. The "Gaussian" part is important, because gaussian distributions are easy to differentiate, making it possible (and fast) to optimize the positions, sizes, orientations, and colors of a collection of Gaussian splats to minimize the difference between the input photos and the rendered scene. This optimization is usually done by starting with the same 3d point clouds and camera poses estimated using the same or similar tools as traditional photogrammetry (e.g. COLMAP), and using this point cloud to place and color your initial Gaussian splats. One of the key insights in the original Gaussian splatting paper was the use of some heuristics to determine when to split a splat into smaller ones to provide higher detail over a given area, and when to combine splats into larger ones to cover uniform/low detail areas.
The nature of Gaussian splats being essentially fancy point clouds means that they can't currently be easily integrated into existing 3d scene manipulation pipelines, although this is rapidly changing as they gain popularity, and tools to convert them into textured meshes and estimate material properties like albedo, reflectance, and so on do exist.