An introduction to non-line-of-sight imaging, and how to create images of hidden objects around a corner in under 10 minutes!
Imagine being able to take a three dimensional image of a room without having to enter it. That seems like a rather unbelievable accomplishment, but it is slowing becoming a reality with the technologies being developed in computational imaging, particularly non-line-of-sight imaging. Even though compact, commercialized forms of the technology are still a ways off in the future, it is already possible to build a "camera" to collect the data with today’s technology.
The process of capturing NLOS image data is fairly similar to taking regular image data, and while there are numerous variations of the approach, they all share the same components. First, there is an opening to the scene, like a doorway or window into a room, through which the camera can observe a surface within the scene, like a wall, floor, or ceiling.
A diagram of an example scene with an NLOS camera outside a room and hidden objects within the room.
Then a "flash" is used to illuminate the scene. In the case of NLOS imaging, that flash is a high power laser. That light bounces around the scene, reflects off the observed surface, and eventually enters the camera. The vertical-temporal image below shows the laser "flash" bouncing around the scene and returning to the imaging surface as various wavefronts.
Vertical-temporal view of a light wave.
The camera has lenses and a shutter to collect light only during some particular period of time. Most cameras you have used capture light over 1/60th, or perhaps even 1/300th, of a second duration, but NLOS cameras need to capture light more precisely than that. They collect light over picosecond durations, that’s 1/1000000000000th of a second!
NLOS imagers need such small exposure times because they are actually recording the light patterns that form on a surface within the scene. When the laser illuminates the scene, its light travels throughout the scene and creates patterns of illumination on the observed surface. If you have ever observed patterns of light on the bottom of a swimming pool or sandy lake bottom, it is a similar effect.
You might be thinking that recording strange patterns of light doesn’t seem very useful, but that is precisely the kind of data being collected by a NLOS imager. While images of light wave patterns is nice and all, that doesn’t really live up to the hype of capturing images of hidden objects in a scene. Read on to see how 3D images can be formed from such apparently useless data.
There are many ways to form a 3D image from a sequence of light wave patterns, so we list some of the approaches below along with key highlights of each method:
Comparing the methods, you can see why we selected the fast f-k migration algorithm for implementation. It is accurate, requires less software development effort, and can run quickly on a desktop.
Your browser does not support the video tag.