Automatic Analysis of Fungal-Infected Tissue using Deep Learning

Automatic Analysis of Fungal-Infected Tissue using Deep Learning Jan-Philipp Praetorius1, Carl-Magnus Svensson1, Franziska Hoffmann6,7, Ferdinand von Eggeling6,7,8,9, Oliver Kurzai3,4,5 and Marc Thilo Figge1,2,3 1 Applied Systems Biology, Hans KnölI Institute, Jena, Germany. 2 Faculty of Biological Sciences, Friedrich Schiller University, Jena, Germany. 3 Center for Sepsis Control and Care (CSCC), Jena University Hospital, Germany. 4 Fungal Septomics, Leibniz Institute for Natural Product Research and Infection Biology, Hans Knöll Institute (HKI), Jena, Germany. 5 Institute of Hygiene and Microbiology, University of Würzburg, Germany. 6 Institute of Physical Chemistry, Friedrich Schiller University, Jena, Germany. 7 ENT Department, University Hospital Jena, Germany. 8 Leibniz Institute of Photonic Technology (IPHT), Jena, Germany. 9 Jena Center for Soft Matter (JCSM), Friedrich Schiller University Jena, Jena, Germany 1. Experimental setup 5. Conclusions •Deep learning of neural networks allows to identify fungal-infected tissue •Novel approach for the registration of optical tissue image and MALDI image •Machine learning models allow to associate fungal-infected tissue with MS-spectra 3. Interface for optical image – MALDI image This work was financially supported by the Deutsche Forschungsgemeinschaft (DFG) through the excellence graduate school Jena School for Microbial Communication (JSMC), the CRC/TR124 FungiNet. 4. Classification of MALDI-imaging •MALDI image is 2D image with 50 μm resolution (e.g. 190 x 170 MS) •Each MS contains values for m/z (mass-to-charge ratio) •Each value represents an intensity MS for whole 2D image MSij Sample MS: •𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛𝑎𝑙𝑖𝑡𝑦 𝑝𝑒𝑟 𝑀𝑆 = 8000 Dimension reduction by Autoencoder Vector with 16 features for each MS S0 S1 S15 S14 … Classify each MS with Random forest classifier by using probability-map as label Background Normal tissue Fungal-infected tissue Preliminary result •Overlay of stained tissue and MALDI image •Light Green: Background •Dark Green: Annotated fungal infected tissue on stained images •Purple: Tissue •Red: MS classified with fungus Deep Learning of Convolutional Neural Networks (CNN) •Network – metric-value on test images: 𝐷𝑖𝑐𝑒 𝐶𝑜𝑒𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑡(𝐷𝐶) = 2𝑥 𝑇𝑃 2 𝑇𝑃+𝐹𝑃+𝐹𝑁 = 1.0 •Network – loss-value on test images: 𝐿 𝑦, ỹ= 𝑦𝑖𝑗 ∗ 𝑙𝑜𝑔(𝑦 𝑖𝑗 ) ∞ 𝑖=0 +𝐷𝐶 ∞ 𝑗=0 = 0.3067 2. Optical image segmentation Contact: Jan-Philipp Praetorius [email protected] Research Group: Applied Systems Biology (A) Test region of manually annotated fungus-infected tissue (B) Network prediction: white regions correspond to high probability P of fungal-infected tissue Scalebar: P(fungal-infected tissue) (C) Complete tissue sample: green regions represent the annotated fungus (D)Network inference for the entire sample image A B Segmentation results C D Network inference Issue: •We need to correlate MS coordinates with the corresponding position in the optical tissue image •No free or commercial software solution existing Approach: •Approximate assignment of each MS to the original tissue image •Use of an interactive graphical user interface (GUI) Gomori PAS Training 2322 2850 Test 410 503 •Combine optical image analysis with MALDI images to detect fungal infected regions •Learn one model for each staining method: •Gomori (Gömöri trichrome stain used on muscle tissue) •PAS (Periodic acid-Schiff reaction) •Classify mass spectrum (MS) according to respective fungus species Funding: GUI Perform clustering with two classes to distinguish between background and tissue Selection of fixed points, which will be used as anchor points for co-localization x 1 1 2 2 Optical tissue image y-dimension-length: 193 x-dimension-length: 178 Mass spectrum of random coordinate (x1 ,y1) MALDI - image ? 20 μm 2 mm # images