Figure 1: Framework for our joint learning of imaging model and reconstruction.
Single Photon Avalanche Photodiodes (SPADs) have recently received a
lot of attention in imaging and vision applications due to their
excellent performance in low-light conditions, as well as their
ultra-high temporal resolution. Unfortunately, like many evolving
sensor technologies, image sensors built around SPAD technology
currently suffer from a low pixel count.
In this work, we investigate a simple, low-cost, and compact optical
coding camera design that supports high resolution image
reconstructions from raw measurements with low pixel counts.
We demonstrate this approach for regular intensity imaging,
depth imaging, as well transient imaging.
Our method uses an end-to-end framework to simultaneously optimize
the optical design and a reconstruction network for obtaining
super-resolved images from raw measurements. The optical design
space is that of an engineered point spread function (implemented
with diffractive optics), which can be considered an optimized
anti-aliasing filter to preserve as much high resolution information
as possible despite imaging with a low pixel count, low fill-factor
SPAD array. We further investigate a deep network for
reconstruction. The effectiveness of this joint design and
reconstruction approach is demonstrated for a range of different
applications, including high speed imaging, and time of flight depth
imaging, as well as transient imaging. While our work specifically
focuses on low-resolution SPAD sensors, similar approaches should
prove effective for other emerging image sensor technologies with
low pixel counts and low fill-factors.