Skip to content (access key 's')
Logo of Technion
Logo of CS Department


Pixel Club: Multicolor Localization Microscopy by Deep Learning
event speaker icon
Eran Hershko (EE, Technion)
event date icon
Tuesday, 6.11.2018, 11:30
event location icon
Room 337 Taub Bld.
Deep learning has become an extremely effective tool for image classification and image restoration problems. Here, we apply deep learning to microscopy, and demonstrate how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera. While existing single-molecule methods for spectral classification require additional optical elements in the emission path, e.g. spectral filters, prisms, or phase masks, our neural net correctly identifies static as well as mobile emitters with high efficiency using a standard, unmodified single-channel configuration. Furthermore, we demonstrate how deep learning can be used to design new phase-modulating elements that, when implemented into the imaging path, result in further improved color differentiation between species, and experimentally demonstrate simultaneous measurement of four species in a single image. Examination of the designed optimal phase pattern shows us that it enables the net to achieve excellent performance even when the SNR is extremely low and emitters’ density is high. Our implementation of the useful architecture for discriminating different PSFs according to color can also be used for analyzing any other effecters on the shape of the PSF (e.g. z-position, molecular orientation, movement dynamics, number of contributing emitters, etc.).

The Research was done under the supervision of Prof. Tomer Michaeli in the faculty of Electrical Engineering and Prof. Yoav Shechtman in the faculty of Biomedical Engineering.
[Back to the index of events]