Techniques for Fast and High-Quality 3D Reconstruction of General Scenes
Doctoral thesis, 2022
This thesis is a collection of techniques used for 3D reconstruction; the creation of 3D models from
real world objects or scenes. Given the increase in accuracy, robustness and speed of modern methods
and algorithms, new and exciting applications of this technology is constantly appearing. Asset creation
for games and movies is a successful example, but there are numerous other applications in architecture,
medicine, communication and more.
The contribution of each paper in this thesis aims to make the use of 3D reconstruction even more
ubiquitous by addressing problems such as performance, memory usage, ease-of-use, robustness
and quality.
Paper I presents a compression technique for volumetric video modeled with voxels. Memory
consumption is an important issue when storing volume data, especially if the data is also varying
with time.
Paper II describes an end-to-end pipeline for recording and rendering volumetric video. A
simple and readily available setup of webcams and a single desktop computer is used to record and render
scenes in real-time.
In Paper III, an interactive tool is developed that aims to help in modeling of real-world
objects. Structured as a simple quad modeling program, the user can construct 3D models on top of a
set of photographs of a chosen object. In the background, or after explicit activation, a multi-view
stereo algorithm helps the user to align the geometry correctly to images in world space. This greatly
simplifies the problem of modeling real world objects accurately, while levering the input from the user
to help with topology and visibility.
Paper IV implements a direct solver for the problem of neural rendering. The reconstruction
is formulated as a non-linear least-squares problem which is solved efficiently with the Gauss Newton
method and the Preconditioned Conjugate Gradient algorithm. This formulation achieves a significant
improvement to reconstruction times compared to previous methods, while also being suitable for
distributed computing due to needing three order of magnitudes fewer iterations until convergence.
Paper V handles the shape-radiance ambiguity in neural rendering. Given infinite spatial
resolution of view-dependent information, almost any shape can satisfy the incoming radiance to each camera, resulting
in errors in the geometry. To address this problem, we propose a solution to separate Lambertian and view-dependent colors
during reconstruction.
real world objects or scenes. Given the increase in accuracy, robustness and speed of modern methods
and algorithms, new and exciting applications of this technology is constantly appearing. Asset creation
for games and movies is a successful example, but there are numerous other applications in architecture,
medicine, communication and more.
The contribution of each paper in this thesis aims to make the use of 3D reconstruction even more
ubiquitous by addressing problems such as performance, memory usage, ease-of-use, robustness
and quality.
Paper I presents a compression technique for volumetric video modeled with voxels. Memory
consumption is an important issue when storing volume data, especially if the data is also varying
with time.
Paper II describes an end-to-end pipeline for recording and rendering volumetric video. A
simple and readily available setup of webcams and a single desktop computer is used to record and render
scenes in real-time.
In Paper III, an interactive tool is developed that aims to help in modeling of real-world
objects. Structured as a simple quad modeling program, the user can construct 3D models on top of a
set of photographs of a chosen object. In the background, or after explicit activation, a multi-view
stereo algorithm helps the user to align the geometry correctly to images in world space. This greatly
simplifies the problem of modeling real world objects accurately, while levering the input from the user
to help with topology and visibility.
Paper IV implements a direct solver for the problem of neural rendering. The reconstruction
is formulated as a non-linear least-squares problem which is solved efficiently with the Gauss Newton
method and the Preconditioned Conjugate Gradient algorithm. This formulation achieves a significant
improvement to reconstruction times compared to previous methods, while also being suitable for
distributed computing due to needing three order of magnitudes fewer iterations until convergence.
Paper V handles the shape-radiance ambiguity in neural rendering. Given infinite spatial
resolution of view-dependent information, almost any shape can satisfy the incoming radiance to each camera, resulting
in errors in the geometry. To address this problem, we propose a solution to separate Lambertian and view-dependent colors
during reconstruction.
voxels
3D reconstruction
computer vision
computer graphics
volumetric video
Author
Sverker Rasmuson
Embedded Electronics Systems and Computer Graphics
A low-cost, practical acquisition and rendering pipeline for real-time free-viewpoint video communication
Visual Computer,; Vol. 37(2021)p. 553-565
Journal article
User-guided 3D reconstruction using multi-view stereo
Proceedings - I3D 2020: ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games,; (2020)
Paper in proceeding
Exploiting coherence in time-varying voxel data
Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games,; (2016)p. 15-21
Paper in proceeding
PERF: Performant, Explicit Radiance Fields
Frontiers in Computer Science,; Vol. 4(2022)
Journal article
Addressing the Shape-Radiance Ambiguity in Radiance Fields - Sverker Rasmuson, Erik Sintorn, Ulf Assarsson
Our lives are increasingly being lived in digital spaces. Our work life, education, social life, and entertainment
are more often than not performed in front of a screen. A bridge between the real world and the digital virtual world can be achieved
with 3D reconstruction. When 3D reconstructing a person, an object, or a room, 3D models are created that can represent this piece
of the real world in a virtual environment.
The most common way of performing such a 3D reconstruction is through a set of photographs or video streams. High-quality camera
equipment, in the form of smartphones, are nowadays available to almost everyone. This means that the available data for creating 3D
models of the real world is larger and more accessible than ever. In combination with the ever more powerful graphics hardware, and new
interaction devices such as virtual reality goggles, new applications for this technology is popping up every day.
In this thesis a wide variety of scenarios for 3D reconstruction are explored, ranging from faces to toys to outdoor scenes. Both
static scenes of individual objects and dynamic scenes of moving persons are considered. A challenging aspect of 3D reconstruction is
the wide variety of types of objects and materials that can occur in a given scene. The work in this thesis show that many types of
general scenes can be handled with modern methods and techniques, while still offering good performance and high quality of
reconstruction.
are more often than not performed in front of a screen. A bridge between the real world and the digital virtual world can be achieved
with 3D reconstruction. When 3D reconstructing a person, an object, or a room, 3D models are created that can represent this piece
of the real world in a virtual environment.
The most common way of performing such a 3D reconstruction is through a set of photographs or video streams. High-quality camera
equipment, in the form of smartphones, are nowadays available to almost everyone. This means that the available data for creating 3D
models of the real world is larger and more accessible than ever. In combination with the ever more powerful graphics hardware, and new
interaction devices such as virtual reality goggles, new applications for this technology is popping up every day.
In this thesis a wide variety of scenarios for 3D reconstruction are explored, ranging from faces to toys to outdoor scenes. Both
static scenes of individual objects and dynamic scenes of moving persons are considered. A challenging aspect of 3D reconstruction is
the wide variety of types of objects and materials that can occur in a given scene. The work in this thesis show that many types of
general scenes can be handled with modern methods and techniques, while still offering good performance and high quality of
reconstruction.
Subject Categories
Computational Mathematics
Computer Science
Computer Vision and Robotics (Autonomous Systems)
ISBN
978-91-7905-629-2
Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie: 5095
Publisher
Chalmers