Single-molecule localization microscopy (SMLM) overcomes the diffraction limit by precisely localizing individual emitters, such as fluorescent molecules. This enables super-resolution biological imaging and particle tracking at the nanoscale. In PSF engineering, a phase mask in the microscope’s Fourier plane reshapes the point spread function (PSF), i.e., the image of a single emitter, so that it encodes additional information such as depth or emission color. Combined with deep learning, the phase mask and the reconstruction network can be optimized together, end to end. In the first part of this talk, we present two task-specific PSF engineering applications that go beyond localization. First, we jointly learn a wavelength-dependent phase mask and a decoding network to directly measure the 3D pairwise distance between two differently colored emitters in a single shot. Second, we present a lattice-aware PSF designed for state detection in 3D neutral-atom arrays used for quantum computing.
In the second part, we address a persistent challenge in applying deep learning to SMLM: the simulation-to-experiment gap. We introduce PILPEL (Physics-Informed Latent Particles for Emitter Localization), a generative model that learns directly from experimental measurements to produce realistic, fully labeled training data. Current hand-built simulators must be manually refined for each experiment and still fail to fully mimic spatially varying backgrounds, complex noise patterns, emitter spatial distributions, and other artifacts. Trained on unlabeled measurements, PILPEL bridges this gap by learning the background, noise, and emitter statistics, including the unmodeled factors that simulators omit. A physical PSF model in its decoder disentangles the emitters from the learned scene, so the trained decoder generates realistic images with exactly known emitter positions, yielding high-fidelity training data. We validate our approach across diverse 2D and 3D datasets, demonstrating that networks trained on PILPEL-generated data outperform their simulation-trained counterparts.
Supervisors: Prof. Yoav Shechtman