MSE Graduate Student Develops Novel Hybrid Convolution Neural Network
By Samantha Bertolino
MSE graduate student Rajat Sainju
Rajat Sainju, a third-year MSE doctoral student, has quite literally broken the mold of typical semantic segmentation for electron microscopy images, a technique that his research group has since applied in the development of computer vision-based algorithms for microscopy data. The opportunity to work in this new field would not have been possible without the support of his advisor, assistant professor Yuanyuan Zhu, and the MSE department. In fact, Rajat found the resources existing within UConn MSE to be crucial in guiding him toward his career goals, providing him with access to many renowned academics and a dedicated staff, and with opportunities to present and discuss his ongoing research in a highly collaborative, engaging environment. Utilizing these resources, Rajat co-authored one of the top 100 most downloaded materials science papers published in Scientific Reports in 2019.
The paper, entitled, “Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images in Steels,” focuses on the development and application of deep learning-based semantic segmentation algorithms, which can be used to automatically identify and segment nanoscale crystallographic defects in electron micrographs. Semantic segmentation involves the process of assigning each object – a type of defect in this case – to a corresponding class. The algorithm is able to make a prediction of all pixels that represent or belong to an object. Depending on the type of material, the defect type, and the number of defects, labeling each pixel manually in an image can take hours, even for seasoned researcher.
In this particular study, however, Rajat worked with his collaborators to develop a new convolution neural network architecture ‘DefectSegNet,’ which is now able to learn and identify any type of defect (such as dislocation lines, precipitates, and voids in steels) from a set of very small yet high-quality Scanning Transmission Electron Microscopy (STEM) images. When compared to the manual quantification of defect metrics, the prediction of defect-maps by DefectSegNet is significantly faster, and can now be completed reliably within seconds. These automated image analysis capabilities were demonstrated using micrographs acquired on HT-9 martensitic steel.
Rajat’s paper on DefectSegNet has become the foundation of his future research. The lessons learned while demonstrating the feasibility of deep learning-based semantic segmentation for identifying defects that form under a complex-contrast mechanism have also opened exciting avenues for other applications of computer vision to S/TEM-image processing.
His current research focuses on the development of computer vision-based algorithms for automated high throughput analysis of images. Specifically, he seeks to understand material dynamics by combining in-situ environmental S/TEM and deep learning-based analysis. Depending on the experimental conditions and the information that needs to be extracted, a given project may include solving a combination of vision-based challenges such as object defection, object tracking, semantic, and instance segmentation. This can be used to better understand the fundamental processes and reactions within materials, like redox reactions, defect motion, catalysis, and phase transformations to name a few. These scientific tools contribute to reliable extraction of statistically significant, high-quality information from microscopy data. The implementation of deep learning algorithms removes human subjectivity, making the measurements robust, reliable, replicable, and comparable. Being able to take advantage of such algorithms will “save a lot of human hours.”
“The bigger picture is to understand the material dynamics and behavior under various conditions.” Rajat says of his work when applied to the real world, “Our hope is to create a positive impact through fundamental researchon the lives of as many people as possible.” The driving force behind his work is to continue developing such tools, and he aims to make them accessible to people across different fields. Building these image-processing tools is not limited to the materials science domain, but can also be applied to medical images, robotics, and satellite imagery, among other things. With this in mind, Rajat hopes to attain a job as a researcher in academia sometime soon.
He thanks his advisor, Dr. Zhu, for helping him to get this far. “She is a great role model with an infectious passion for materials science,” he says. Her vision for the future of in-situ electron microscopy and the integration of deep learning/computer vision for the advancement of microscopy has deeply influenced his research. “Becoming a scientist requires a broad range of scientific skills, critical thinking, imagination, integrity, independence, and of course, a very supportive mentor. By providing an environment of growth, Professor Zhu has helped me to acquire those skills and build an aptitude for scientific discovery.”
Published: November 30, 2020
Categories: electron microscopy, graduate students, in-situ electron microscopy, news
Available Archives