Accurate and robust 3D imaging of specular, or mirror-like, surfaces is crucial in fields such as industrial inspection, medical imaging, virtual reality and cultural heritage preservation. Yet anyone who has visited a house of mirrors at an amusement park knows how difficult it is to judge the shape and distance of reflective objects.
This challenge also persists in science and engineering, where the accurate 3D imaging of specular surfaces has long been a focus in both optical metrology and computer vision research. While specialized techniques exist, their inherent limitations often confine them to narrow, domain-specific applications, preventing broader interdisciplinary use.
In a study published March 27 in the journal Optica, University of Arizona researchers from the Computational 3D Imaging and Measurement (3DIM) Lab at the Wyant College of Optical Sciences present a novel approach that significantly advances the 3D imaging of specular surfaces.
Their method seamlessly combines the information captured from two established techniques – Phase Measuring Deflectometry (PMD) and Shape from Polarization (SfP) – which are typically used in optical 3D metrology and computer vision research, respectively. In the new method, the benefits of both techniques complement each other in a manner never fully realized in previous work, paving the way for 3D imaging of specular objects that is both highly accurate and widely applicable.
Well-known for its high accuracy and precision, conventional Phase Measuring Deflectometry (PMD) is and is widely used in high-end applications to inspect optical lenses and telescopic mirrors or to detect defects in car bodies. However, PMD also comes with challenges.
Florian Willomitzer, associate professor of optical sciences, director of the 3DIM lab and principal investigator of the study explained: "In particular, PMD suffers from inherent ambiguity issues. Overcoming these challenges typically requires either additional hardware or prior knowledge about the object's shape and distance—limiting PMD's flexibility for general use.
"On the other hand, Shape from Polarization (SfP) is a well-established 3D imaging method in the computer vision community that is highly flexible," he said. "However, certain geometric assumptions limit its accuracy. This restricts the method to applications with low accuracy requirements or purely qualitative inspection cases."
Bridging the gap between Optical 3D Metrology and Computer Vision Research
The team's new technique integrates the strengths of PMD and SfP while overcoming their weaknesses. Leveraging the geometrical information from deflectometry with polarization cues, the surface shape and normal field of the specular object can be accurately reconstructed—without the need for prior knowledge about the object, complex setups, or specific assumptions about the imaging model.
"We developed a mathematically rigorous and creative approach to combine these two sets of information," said Jiazhang Wang, postdoctoral associate in Willomitzer's lab, first author and lead researcher of the study. "This results in a novel measurement technique that accurately determines the object's shape and surface normals without the typical ambiguities, ensuring both high accuracy and wide applicability. In essence, our new method is bridging the current technology gap between optical 3D metrology and computer vision."
Single-Shot 3D Reconstruction: A Leap Toward Practical Motion-Robust Measurements
Traditional PMD and SfP are "multi-shot" methods and require capturing 8 to 30 or more camera images in succession to reconstruct one single 3D model, making them highly susceptible to motion artifacts.
"The slightest movement during this sequence introduces severe reconstruction errors and renders the results unusable," explained Wang. "By integrating novel hardware designs with advanced reconstruction algorithms, our method can now extract all required information from one single camera image. This can enable real-time, hand-guided measurements and high-speed imaging of dynamic scenes."
Wang, Willomitzer and co-author Oliver Cossairt, adjunct associate professor in electrical and computer engineering at Willomitzer's and Wang's previous institution Northwestern University, shared their excitement for the study's possibilities.
"The single-shot capability is a crucial advancement for applications where motion robustness is essential," Wang said, "such as measuring fast-moving parts on a conveyor belt or scanning objects by hand-guiding the sensor."
Pushing the Limits for Next-Generation 3D Sensors
A key aspect of this study was to understand and analyze the current limitations of 3D imaging on specular surfaces and use this knowledge to develop a sensor concept that overcomes these challenges while building upon the strengths of current PMD and SfP methods.
Willomitzer concluded that this way of tackling current imaging challenges has merit far beyond the "house of mirrors" problem of measuring specular surfaces.
"This mindset aligns closely with one of the core themes of our lab," he said. "We strive to explore and exploit physical and information-theoretical limits to invent, develop and build the next generation of computational 3D imaging systems."
