Curriculum Learning for ab initio Deep Learned Refractive Optics.

Xinge Yang, Qiang Fu, Wolfgang Heidrich
Nature Communications, 2024.

Curriculum learning for automated lens design. a We utilize a differentiable ray-tracing approach to simulate the sensor captured image of an object image. This sensor capture can then be input into a downstream deep network for image reconstruction. During the forward image simulation (black arrows), we track the gradient of each optical parameter. We can subsequently back-propagate (blue arrows) the errors from either the simulated image for classical optical design, or from the network output for end-to-end optical design. The end-to-end optical design jointly optimizes the optical lens and the image reconstruction network. Classical lens design methods often face issues such as local minima and degenerate optical structures, including self-intersections, requiring appropriate starting points and consistent human intervention. We introduce a curriculum learning strategy that encompasses: a curriculum path (gray dashed arrow in a), optical regularization (b), and a re-weighting mask (c). b The optical regularization term presents lens from degenerate structures during the optimization. c The re-weighting mask dynamically directs attention towards problematic regions of simulated images during each epoch, compelling the optimization process to escape local minima. This curriculum learning strategy aims to automate the design of complex optical lenses from scratch, for both classical and computational lenses. d An example of this automated classical lens design using the curriculum learning strategy. The lens design process initiates from a flat structure, gradually elevating the design complexity until it meets the final design specifications. Detailed evaluations can be found in Table 1 and Supplementary Note 4.1. The cat image was photographed by Xinge Yang (CC BY 2.0).

Abstract

Deep optical optimization has recently emerged as a new paradigm for designing computational imaging systems using only the output image as the objective. However, it has been limited to either simple optical systems consisting of a single element such as a diffractive optical element or metalens, or the fine-tuning of compound lenses from good initial designs. Here we present a DeepLens design method based on curriculum learning, which is able to learn optical designs of compound lenses ab initio from randomly initialized surfaces without human intervention, therefore overcoming the need for a good initial design. We demonstrate the effectiveness of our approach by fully automatically designing both classical imaging lenses and a large field-of-view extended depth-of-field computational lens in a cellphone-style form factor, with highly aspheric surfaces and a short back focal length.

Paper

Paper                  [Yang2024DeepLens.pdf (~3MB)] 
Supplementary          [Yang2024DeepLens_supp.pdf (~60MB)] 
Code                   [https://github.com/vccimaging/DeepLens]