Key Takeaways
- Automation creates a new age of electron microscopy for materials science, enabling faster data collection, reduced bias, and more complex experiments with little human intervention.
- However, automation results in large data sets that are challenging to move, store, and process.
- Researchers at Berkeley Lab developed a method for automated microscopy, a streaming service for faster data transfer, and a web-based platform called Distiller that brings the two together.
Modern electron microscopes can capture incredibly detailed images of materials down to the atomic level, providing valuable insights for materials science. However, they require a skilled operator and can only focus on very small areas at a time. Researchers at the Department of Energy's Lawrence Berkeley National Laboratory (Berkeley Lab) created an automated workflow that overcomes these limitations by allowing large amounts of data to be collected over wide areas without human intervention - and then quickly transferred to supercomputers for real-time processing.
All of this - the automated data collection and streaming - is made available to scientists through the web-based platform known as Distiller, named for its ability to distill, or extract the most meaningful information out of, complex microscopy data sets.
"We're moving towards getting huge amounts of data processed in an automated way with minimal human intervention."
– Peter Ercius
"We now have a way for people to interact with a detector on one side and a supercomputer on the other side with a simple webpage called Distiller in the middle," said Peter Ercius, interim facility director of the National Center for Electron Microscopy (NCEM) at Berkeley Lab's Molecular Foundry. "We're moving towards getting huge amounts of data processed in an automated way with minimal human intervention."
This effort stems from the 4D Camera Distillery, a DOE-funded program aimed at developing artificial intelligence (AI) and machine learning (ML) methods for electron microscopy experiments.
Advancing Electron Microscopy Through Automation
Despite developments in capabilities that have made electron microscopy an increasingly powerful and versatile tool for materials science, the process of collecting data has not changed significantly since its inception. While fields such as the life sciences have been able to harness the benefits of automated data collection, similar advancements for materials science applications have been hindered by the greater variability and complexity of materials samples. The Molecular Foundry developed a process for automating electron microscopy for heterogeneous materials in a modular way, allowing discrete steps to be pieced together into different workflows that are customized to different experiments.
"We developed an automation client that can string together a set of instructions, like (1) focus the image then (2) take an image then (3) move the stage, and then ask it to do this on repeat," explained Ercius. When strung together, these steps can carry out long running experiments to create a large area map from a grid of images, or combine imaging techniques to learn more from a material. The high-throughput collection of atomic-resolution images through automation is demonstrated in Nanotechnology.
Streaming Big Data into Big Discoveries in Real Time
However, the consequence of automation is that large data sets are taken very quickly. Further exacerbating the problem is that new fast detectors can produce very large amounts of data - 700 gigabytes in 15 seconds - presenting significant challenges in storing and processing information as well as moving it, in this case between NCEM and the National Energy Research Scientific Computing Center (NERSC). Initially, the microscope was limited to about 15 minutes of operation due to insufficient disk space. As described by lead author Sam Welborn, a postdoctoral researcher at NERSC, "by collaborating with NCEM through the NERSC Science Acceleration Program, we developed a method for efficiently streaming data from NCEM to NERSC and skipping the hard disk altogether". The method is analogous to streaming a video to your phone - instead of first downloading the full file, the viewer can watch the video as it transfers. Compared to the conventional file-transfer process, this method is up to 14 times faster and more reliable. Their work was published in Microscopy and Microanalysis and at the ISC High Performance Conference.
The TEAM 1 transmission electron microscope at the Molecular Foundry's National Center for Electron Microscopy. (Credit: Thor Swift/Berkeley Lab)
With Distiller, the user can monitor data collection in real-time, transfer the data to the NERSC supercomputer, and process it during a live experiment. "NCEM's Distiller project and data streaming pipeline is a poster child for the Superfacility concept and DOE's Integrated Research Infrastructure (IRI)," said Bjoern Enders, data science workflows architect in NERSC's Data Science Engagement Group. "It demonstrates seamless, real-time integration of supercomputers into a workflow and utilizes NERSC features specifically designed for interfacing with large data-generating experiments. These capabilities highlight the benefit of close collaboration between scientific facilities."
The automation client and streaming service are available to all researchers, either in-person or remote, through the Molecular Foundry's User Program. Looking ahead, AI/ML will be used to add more complex decision-making algorithms while further development in microscope stage technology may allow for more accurate stage motions. The streaming service also has the potential to be expanded outside of NCEM-specific data formats.
The Molecular Foundry and NERSC are DOE Office of Science user facilities at Berkeley Lab.
The research was supported by the Department of Energy's Office of Science.
