Learning Adaptive Tensorial Density Fields for Clean Cryo-EM Reconstruction

Yuanhao Wang, Ramzi Idoughi, Wolfgang Heidrich

NeurIPS (Poster)

Our framework is composed of two steps: During the training step (a) we first update the quadtree structure using downsampled projections. Then, we freeze that structure and update the tensor representation in each node using the original noisy projections. The second step is the volume querying (b), where we uniformly sample the ROI, and use the learned representation to estimate the densities at the selected positions.

Abstract

We present a novel learning-based framework for reconstructing 3D structures from tilt-series cryo-Electron Tomography (cryo-ET) data. Cryo-ET is a powerful imaging technique that can achieve near-atomic resolutions. Still, it suffers from challenges such as missing-wedge acquisition, large data size, and high noise levels. Our framework addresses these challenges by using an adaptive tensorial-based representation for the 3D density field of the scanned sample. First, we optimize a quadtree structure to partition the volume of interest. Then, we learn a vector-matrix factorization of the tensor representing the density field in each node. Moreover, we use a loss function that combines a differentiable tomographic formation model with three regularization terms: total variation, boundary consistency constraint, and an isotropic Fourier prior. Our framework allows us to query the density at any location using the learned representation and obtain a high-quality 3D tomogram. We demonstrate the superiority of our framework over existing methods using synthetic and real data. Thus, our framework boosts the quality of the reconstruction while reducing the computation time and the memory footprint. The code is available at https://github.com/yuanhaowang1213/adaptivetensordf.



Paper

Paper [Wang2023LearningAdaptiveTDF.pdf (~21.8MB)] 
Supplement [Wang2023LearningAdaptiveTDF_supp.pdf (~37.9MB)]

Code and Datasets

Source code shared on [Github] 

Datasets from the [EMPIAR (the Electron Microscopy Public Image Archive)]

Acknowledgement

This work was supported by King Abdullah University of Science and Technology as part of VCC Center Competitive Funding.

Citation

@inproceedings{wang2023learning,
	title={Learning Adaptive Tensorial Density Fields for Clean Cryo-{ET} Reconstruction},
	author={Yuanhao Wang and Ramzi Idoughi and Wolfgang Heidrich},
	booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
	year={2023},
	url={https://openreview.net/forum?id=r7g9nFsulw}
}