Compact Snapshot Hyperspectral Imaging with Diffracted Rotation

Daniel S. Jeon, Seung-Hwan Baek, Shinyoung Yi, Qiang Fu, Xiong Dun, Wolfgang Heidrich, Min H. Kim
Accepted to ACM Transactions on Graphics (Proc. SIGGRAPH), 2019

Abstract
Paper and Supplement
Citation

We propose a compact, diffraction-based snapshot hyperspectral imaging method with a novel diffractive optical element attached to a conventional, bare image sensor. Our method replaces the common optical elements in hyperspectral imaging (prism, coded mask, relay and imaging lenses) with a single optical element. Our single DOE-based camera is coupled with a data-driven spectral reconstruction method that can restore faithful spectral information from spectrally-varying point spread functions. (a) Our fabricated DOE (inset) and a DSLR camera, installed with the DOE for spectral imaging. (b) Reconstructed hyperspectral image from real input. (c) Spectrally-varying PSFs measured per wavelength. (d) Corresponding captured spectral channels. (e) Spectral plots of two patches from the captured ColorChecker, compared to the ground truth.

Abstract

Traditional snapshot hyperspectral imaging systems include various optical elements: a dispersive optical element (prism), a coded aperture, several relay lenses, and an imaging lens, resulting in an impractically large form factor. We seek an alternative, minimal form factor of snapshot spectral imaging based on recent advances in diffractive optical technology. We thereupon present a compact, diffraction-based snapshot hyperspectral imaging method, using only a novel diffractive optical element (DOE) in front of a conventional, bare image sensor. Our diffractive imaging method replaces the common optical elements in hyperspectral imaging with a single optical element. To this end, we tackle two main challenges: First, the traditional diffractive lenses are not suitable for color imaging under incoherent illumination due to severe chromatic aberration because the size of the point spread function (PSF) changes depending on the wavelength. By leveraging this wavelength-dependent property alternatively for hyperspectral imaging, we introduce a novel DOE design that generates an anisotropic shape of the spectrally-varying PSF. The PSF size remains virtually unchanged, but instead the PSF shape rotates as the wavelength of light changes. Second, since there is no dispersive element and no coded aperture mask, the ill-posedness of spectral reconstruction increases significantly. Thus, we propose an end-to-end network solution based on the unrolled architecture of an optimization procedure with a spatial-spectral prior, specifically designed for deconvolution-based spectral reconstruction. Finally, we demonstrate hyperspectral imaging with a fabricated DOE attached to a conventional DSLR sensor. Results show that our method compares well with other state-of-the-art hyperspectral imaging methods in terms of spectral accuracy and spatial resolution, while our compact, diffraction-based spectral imaging method uses only a single optical element on a bare image sensor.

Paper

paper [Jeon2019Hyperspectral.pdf (~13.0MB)]

supplementary material [Jeon2019Hyperspectral_supp.pdf (~58.0MB)]

Citation

  @Article{HyperspectralDOE:SIG:2019,
  author  = {Daniel S. Jeon and Seung-Hwan Baek and Shinyoung Yi and Qiang Fu 
             and Xiong Dun and Wolfgang Heidrich and Min H. Kim},
  title   = {Compact Snapshot Hyperspectral Imaging with Diffracted Rotation},
  journal = {ACM Transactions on Graphics (Proc. SIGGRAPH 2019)},
  year    = {2019},
  volume  = {38},
  number  = {4},
  pages   = {117:1--13},
  doi     = "10.1145/3306346.3322946",
  url     = "https://doi.org/10.1145/3306346.3322946",
  }