Dynamic, Pixelwise Image Sensing and Systems

This technology includes a suite of inventions, including a fully-functional coded-exposure-pixel (CEP) camera prototype, a novel programmable CMOS sensor, and a method for adaptive adjustment of exposure of every pixel, based on changes in the brightness of the scene for that pixel to extend dynamic range at native sensor resolution.

Unlike conventional cameras, which record all light incidents onto a pixel, CEP cameras can be programmed to selectively detect only some of that light or sort all of the light, depending on the pixel code. One of our approaches, coded two-bucket (C2B) imaging, is a new operating principle for computational sensors with applications in active 3D shape estimation and coded-exposure imaging. A C2B sensor modulates the light arriving at each pixel by controlling which of the pixel’s two “buckets” should integrate it. C2B sensors output two images per video frame—one per bucket—and allow rapid, fully-programmable, per-pixel control of the exposure. In conjunction with a concurrently coded illumination, this enables a wide range of new coded multi-exposure single-readout-frame imaging capabilities at video rates.

OPPORTUNITY

While the last decade has seen a major race to improve the resolution, optics, and noise properties of mobile cameras, modern computational photography applications such as 3D sensing, gesture analysis, and robotic navigation drive the growing need for programmability, or coding, of the camera exposure at the individual-pixel level. We offer a low-cost, small-form-factor and fast computational camera solution where all computation is done within one chip and one video frame.

The light efficiency and electronic per-pixel coding capabilities of our sensors and methods open up imaging applications beyond what is possible today, including:

Time-of-flight (ToF) 3D imaging in outdoor, industrial robotics applications
Low-power high-dynamic-range (HDR) single frame imaging without motion artifacts
Direct/indirect light imaging in challenging conditions, such as scattering and reflective scenes (e.g. ice on aircraft, blood vessels under skin, etc.)
Compressive acquisition of high-speed video with optimal light efficiency
Simultaneous acquisition of both epipolar-only and non-epipolar video streams

STATUS