A new publication from Opto-Electronic Advances, 10.29026/oea.2024.230034 discusses how the synergy of traditional techniques and deep learning enables single-frame high-precision fringe pattern analysis.
Optical metrology, as a general-purpose metrology technique that uses light as information carriers for non-contact and non-destructive measurement, is fundamental to manufacturing, basic research, and engineering applications. With the invention of the laser and charge-coupled device (CCD), many optical metrology methods and instruments are employed in state-of-the-art manufacturing processes, precision positioning, and quality assessment because of their advantages in terms of accuracy, sensitivity, repeatability, and speed. For many optical metrology techniques such as interferometry, digital holography, and fringe projection profilometry (FPP), fringe pattern analysis is the primary focus of research for recovering the underlying phase distribution from the recorded fringe patterns. The accuracy and efficiency of phase retrieval from fringe patterns are essential to dynamically reconstruct various desired physical properties of the objects (the profile, distance, strain, etc.).
For structured light 3D imaging based on FPP, to minimize the number of fringes required for a single reconstruction, Prof. Qian Chen and Chao Zuo's research group at Nanjing University of Science and Technology established the theoretical framework for phase shifting profilometry and temporal phase unwrapping and developed a series of composite phase shifting methods for fast 3D measurement, including bi-frequency phase shifting, 2+2 phase shifting, geometric constraints based composite phase shifting, and micro Fourier transform profilometry (μFTP). These composite phase shifting methods reduce the number of fringe patterns required per 3D reconstruction from ~10 to 5, 4, 3, or even 2, achieving high-speed 3D sensing at 10k frames per second. Nevertheless, high-accuracy 3D reconstruction using only one single pattern has been the ultimate goal of structured light 3D imaging in perpetual pursuit. However, the key to the success of FTP is that the high-frequency fringe information modulated by the object surface can be well separated from the background intensity in the frequency domain. As a result, the FTP technique is limited to measuring smooth surfaces with limited height variations.
Recently, with the explosive growth of available data and computing resources, deep learning, as a "data-driven" machine learning technique, has achieved impressive success in numerous fields, such as computer vision and computational imaging. Deep learning, which pervades almost all aspects of optical metrology, provides solutions to many challenging problems, such as fringe denoising, fringe analysis, and digital holographic reconstruction.
However, different from traditional fringe analysis methods, these deep learning approaches focus mainly on training a DNN to accurately identify an image-to-image transform from massive input and output data pairs, as the physical laws governing the image formation or other domain expertise pertaining to the measurement have not yet been fully exploited in current deep learning practice. Consequently, the performance of deep learning approaches in solving complex physical problems relies heavily on the underlying statistical characteristics within the dataset. In order to push the limits of fringe pattern analysis in terms of speed, accuracy, repeatability, and generalization, the synergy of physics-based traditional methods and data-driven learning approaches become to represent the general trend [Fig. 1(b)].
In a recent publication in Opto-Electronic Advances, Prof. Qian Chen and Prof. Chao Zuo's research group at Nanjing University of Science and Technology reported a physics-informed deep learning method for fringe pattern analysis (PI-FPA), which integrates a lightweight DNN with a learning-enhanced Fourier transform profilometry (LeFTP) module [Fig. 2], enabling more accurate and computationally efficient single-shot phase retrieval. The lightweight network refines the initial phase to further improve the phase accuracy at a low computational cost, compared with universal end-to-end image transform networks (U-Net and its derivatives).
Dynamic 360-degree 3D reconstruction results of a workpiece model by different fringe analysis methods are shown in [Fig. 3]. For the 3-step PS method, when dynamic scenes are measured, the relative motion between the object and the phase-shifting fringe patterns sequentially projected will cause motion artifacts and thus introduce non-negligible errors into 3D reconstruction results. For single-frame fringe analysis, FTP is suitable for dynamic 3D measurement, but yields coarse 3D results with low quality in terms of accuracy and resolution due to the spectrum overlapping. U-Net can further improve the quality of 3D reconstruction, but it cannot reliably retrieve the phase of the object with metal materials which is relatively rare in the training dataset, precluding the recovery of fine surfaces. This experiment demonstrates that the proposed PI-FPA can be applied for high-quality and efficient 3D modeling of complex structure parts.
The proposed physics-informed deep learning technique for fringe pattern analysis (PI-FPA) not only learns the inherent statistical characteristics within the dataset like traditional neural networks, but also masters the physical laws describing the image formation, realizing single-frame phase reconstruction with high precision and high computational efficiency, while exhibiting its good generalization to rare samples never seen by the network. In the future, we will investigate the phase recovery performance of PI-FPA for different types of fringe images, and explore related fringe analysis applications in the fields of interferometry and digital holography in optical metrology, further pushing the limits of fringe pattern analysis in speed, accuracy, repeatability, and generalization.
