CMX Student/Postdoc Seminar

At some point, every robot must translate its raw perceptual data (i.e., RGB images, lidar scans) into a mathematical model of the scene geometry. In this talk, I will discuss an emerging scene representation in robotics, Neural Radiance Fields (NeRFs), that can be built using only (posed) RGB images. NeRFs promise a number of advantages over traditional scene representations, including the ability to generate differentiable, high-quality, synthetic views of the scene from unseen poses and their inherent ability to represent uncertainty about the scene's geometry.

In the first part of the talk, I will provide a brief tutorial on neural rendering, deriving the rendering procedure used by NeRFs and surveying the current state-of-the-art. Following the tutorial, I will present some of our recent work using NeRFs for robotic tasks such as visual navigation, including an exciting theoretical result showing that the NeRF density can be rigorously interpreted as a stochastic process. I will conclude with a brief discussion of open research directions in robot perception (i.e., incorporating semantics, uncertainty-awareness) and some ideas about how NeRFs can bridge existing gaps. Overall, I hope to make a case that NeRFs are a promising scene/geometry representation for a variety of robotics applications and to provide a thorough explanation of their basic concepts for robotics practitioners.