Aberration-Aware Depth-from-Focus.

Xinge Yang, Qiang Fu, Mohamed Elhoseiny, Wolfgang Heidrich
ICCP, IEEE TPAMI, 2022.

---

• Abstract
• Paper

---

Aberration-Aware Training pipeline. (a) all-in-focus RGB images and corresponding depth maps are given as the input. (b) different focus distances are selected to simulate the focal swap process. The PSF network estimates PSF for different object positions and focus distances (orange path). Then the PSF is convolved with the all-in-focus image to get the focal stack (blue path). (c) the DfF network takes the focal stack and focus distances to estimate the depth map (black path).

Abstract

Computer vision methods for depth estimation usually use simple camera models with idealized optics. For modern machine learning approaches, this creates an issue when attempting to train deep networks with simulated data, especially for focus-sensitive tasks like Depth-from-Focus. In this work, we investigate the domain gap caused by off-axis aberrations that will affect the decision of the best-focused frame in a focal stack. We then explore bridging this domain gap through aberration-aware training (AAT). Our approach involves a lightweight network that models lens aberrations at different positions and focus distances, which is then integrated into the conventional network training pipeline. We evaluate the generality of network models on both synthetic and real-world data. The experimental results demonstrate that the proposed AAT scheme can improve depth estimation accuracy without fine-tuning the model for different datasets.

Paper

Paper                  [Yang2023AATDfF.pdf] 

Supplementary          [Yang2023AATDfF_supp.pdf] 

Code                   [https://github.com/vccimaging/Aberration-Aware-Depth-from-Focus]