Cryogenic electron tomography (cryo-ET) visualizes the 3D spatial distribution of macromolecules at nanometer resolution inside native cells. However, automated identification of macromolecules inside cellular tomograms is challenged by noise and reconstruction artifacts, as well as the presence of many molecular species in the crowded volumes. 

To overcome these obstacles, an international team of scientists from France, Spain and Germany, under the leadership of Charles Kervrann, from France BioImaging BioImage Informatics Node, developed a deep learning-based framework to quickly identify multiple classes of macromolecules in cryo-ET volumes. This DeepFinder pro­gram, now published in Nature Methods, builds upon convolutional neural networks that have already proven highly valuable in the microscopy field.

Overview of DeepFinder (from Moebel, E., et al., Nat Methods 18, 1386–1394 (2021).

a) The DeepFinder workflow consists of a training stage (stage I) and an analysis (or inference) stage (stage II). These two stages correspond to five steps (represented by blue boxes) to locate macromolecular complexes within crowded cells.

b) Ribosome localization with DeepFinder in a cryo-electron tomogram of a C. reinhardtii cell.
Tomographic slice with superimposed segmented cell membrane (gray) and ribosomes classified with respect to their binding states: membrane-bound (blue) and cytosolic (yellow).

c) Tomographic slices showing coordinates
of detected ribosomes (colors correspond to b). The positions and classes were determined by analyzing the segmentation map shown in b. This
analysis used 48 tomograms for training, one for validation and eight for testing. Scale bar, 60 nm.

Once trained, the inference stage of DeepFinder is faster than template matching and performs better than other competitive deep learning methods at identifying macromolecules of various sizes in both synthetic and experimental datasets. On cellular cryo-ET data, DeepFinder localized membrane-bound and cytosolic ribosomes (roughly 3.2 MDa), ribulose 1,5-bisphosphate carboxylase–oxygenase (roughly 560 kDa soluble complex) and photosystem II (roughly 550 kDa membrane complex) with an accuracy comparable to expert-supervised ground truth annotations. DeepFinder is therefore a promising algorithm for the semiautomated analysis of a wide range of molecular targets in cellular tomograms. It also serves as a prime example illustrating the importance of developing efficient, customized AI tools to accelerate knowledge generation in the biomedical life sciences.

DeepFinder has been implemented as a free, open-source program with an accessible graphical user interface.

The team is currently working on adapting it to fluorescence microscopy.

Moebel, E., Martinez-Sanchez, A., Lamm, L. et al. Deep learning improves macromolecule identification in 3D cellular cryo-electron tomograms. Nat Methods 18, 1386–1394 (2021). https://doi.org/10.1038/s41592-021-01275-4

Contact

A new version of TrackMate is available now, with major changes that improve its versatility. TrackMate now integrates state-of-the-art segmentation algorithms from machine-learning and deep-learning such as StarDist, Ilastik and Weka.

TrackMate[1] is a Fiji plugin that address cell or organelle tracking in Life-Science microscopy images. Its main goals are to be user-friendly, interoperable and to serve as a platform to accelerate the development of novel tracking algorithms and analysis pipelines.

With this new version we rewrote almost entirely TrackMate so that it can integrate state-of-the-art segmentation algorithms and benefit from their output. For instance, TrackMate can now store, display, save, load and exploit object contours in 2D.

We also made a new application programming interface that will facilitate and accelerate reusing TrackMate in other analysis pipelines and allow 3rd party contributors to add new segmentation algorithms in TrackMate in an easy way. We used this API ourselves to add 7 new segmentation algorithms to TrackMate:

For instance, the StarDist[2] algorithm is integrated as two different detectors. The first one uses the built-in deep-learning model that can segment cell nuclei in fluorescence image in a wide range of situation. The robustness of the StarDist algorithm in turn positively impacts the robustness of tracking and allows for better detection of cell divisions with TrackMate tracking algorithms. This will facilitate cell migration studies.

The TrackMate StarDist integration also allows for specifying and using a custom deep-learning model. For instance, we trained a specific model to detect T-cells imaged in bright-field microscopy and track them over time. Before the emergence of such detection algorithms, the tracking of label-free cells was difficult.

We also integrated the ilastik[3] segmentation software. A TrackMate user can input an ilastik classifier to detect objects then track them. We used them to study the bacterial growth of Neisseria meningitidis clones. The output of this analysis pipeline offers the lineage of each single cell along with its morphology and how it evolves across cell divisions.

The new capabilities of TrackMate can be used to address applications beyond tracking. For instance, it is now possible to use TrackMate to perform the segmentation of 3D objects using a slice-by-slice approach. This approach consists in segmenting objects in each 2D section of a 3D stack, then merging the segmentation results along Z in a subsequent step. This can be done in TrackMate, using the tracking step for merging. We implemented a novel tracking algorithm to foster this application, the overlap tracker. We could use this approach combining the cellpose[4] algorithm in 2D to segment 3D images of Arabidopsis thaliana floral meristem.

There are several other algorithms that are now offered to the TrackMate user, within a user-friendly software meant to interoperate with the key software of bioimage analysis. More importantly, TrackMate is an open-source academic software, and its new API will foster the development of new analysis pipeline with TrackMate and the integration of new algorithms by other developers, increasing the breadth of applications it can address for Life-Science researchers.

This new version of TrackMate is the product of a collaboration between the IAH facility (Institut Pasteur), part of the FBI Bioimage Informatics Node , the Jacquemet lab (Turku Bioscience Centre) , and the Dumenil lab (Institut Pasteur) .

Bringing TrackMate in the era of machine-learning and deep-learningDmitry Ershov, Minh-Son Phan, Joanna W. Pylvänäinen, Stéphane U. Rigaud, Laure Le Blanc, Arthur Charles-Orszag, James R. W. Conway, Romain F. Laine, Nathan H. Roy, Daria Bonazzi, Guillaume Duménil, Guillaume Jacquemet, Jean-Yves Tinevez bioRxiv 2021.09.03.458852; doi: https://doi.org/10.1101/2021.09.03.458852

Contact: Jean-Yves Tinevez

[1] https://imagej.net/plugins/trackmate/

[2] Alejandro F. Frangi, Julia A. Schnabel, Christos Davatzikos, Carlos Alberola-López, and Gabor Fichtinger Uwe Schmidt, Martin Weigert, Coleman Broaddus, and Gene Myers. Cell detection with star-convex polygons. In Alejandro F. Frangi, Julia A. Schnabel, Christos Davatzikos, Carlos Alberola-López, and Gabor Fichtinger, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2018, pages 265–273, Cham, 2018. Springer International Publishing. doi:10.1007/978-3-030-00934-2_30.

[3] Stuart Berg, Dominik Kutra, Thorben Kroeger, Christoph N Straehle, Bernhard X Kausler, Carsten Haubold, Martin Schiegg, Janez Ales, Thorsten Beier, Markus Rudy, Kemal Eren, Jaime I Cervantes, Buote Xu, Fynn Beuttenmueller, Adrian Wolny, Chong Zhang, Ullrich Koethe, Fred A Hamprecht, and Anna Kreshuk. ilastik: interactive machine learning for (bio)image analysis. Nature Methods, 16(12):1226–1232, 2019. ISSN 1548-7105. doi:10.1038/s41592-019-0582-9.

[4] Carsen Stringer, Tim Wang, Michalis Michaelos, and Marius Pachitariu. Cellpose: a generalist algorithm for cellular segmentation. Nature Methods, 18(1):100–106, jan 2021. doi:10.1038/s41592-020-01018-x.