Figure 1: Framework for our joint learning of imaging model and reconstruction.
Single Photon Avalanche Photodiodes (SPADs) have recently received a lot of attention in
imaging and vision applications due to their excellent performance in low-light conditions,
as well as their ultra-high temporal resolution. Unfortunately, like many evolving sensor technologies,
image sensors built around SPAD technology currently suffer from a low pixel count.
In this work, we investigate a simple, low-cost, and compact optical coding camera design that supports
high-resolution image reconstructions from raw measurements with low pixel counts. We demonstrate this
approach for regular intensity imaging, depth imaging, as well transient imaging.
Our method uses an end-to-end framework to simultaneously optimize the optical design and a reconstruction
network for obtaining super-resolved images from raw measurements. The optical design space is that of an
engineered point spread function (implemented with diffractive optics), which can be considered an optimized
anti-aliasing filter to preserve as much high-resolution information as possible despite imaging with a low pixel
count, low fill-factor SPAD array. We further investigate a deep network for reconstruction. The effectiveness
of this joint design and reconstruction approach is demonstrated for a range of different applications,
including high-speed imaging, and time of flight depth imaging, as well as transient imaging. While our work
specifically focuses on low-resolution SPAD sensors, similar approaches should prove effective for other
emerging image sensor technologies with low pixel counts and low fill-factors.