Artificial vision systems find a wide range of applications, including self-driving cars, object detection, crop monitoring, and smart cameras. Such vision is often inspired by the vision of biological organisms. For instance, human and insect vision have inspired terrestrial artificial vision, while fish eyes have led to aquatic artificial vision. While the progress is remarkable, current artificial visions suffer from some limitations: they are not suitable for imaging both land and underwater environments, and are limited to a hemispherical (180°) field-of-view (FOV).
To overcome these issues, a group of researchers from Korea and USA, including Professor Young Min Song from Gwangju Institute of Science and Technology in Korea, have now designed a novel artificial vision system with an omnidirectional imaging ability, which can work in both aquatic and terrestrial environments. Their study was made available online on 12 July 2022 and published in Nature Electronics on 11 July 2022.
"Research in bio-inspired vision often results in a novel development that did not exist before. This, in turn, enables a deeper understanding of nature and ensure that the developed imaging device is both structurally and functionally effective," says Prof. Song, explaining his motivation behind the study.
The inspiration for the system came from the fiddler crab (Uca arcuata), a semiterrestrial crab species with amphibious imaging ability and a 360° FOV. These remarkable features result from the ellipsoidal eye stalk of the fiddler crab's compound eyes, enabling panoramic imaging, and flat corneas with a graded refractive index profile, allowing for amphibious imaging.
Accordingly, the researchers developed a vision system consisting of an array of flat micro-lenses with a graded refractive index profile that was integrated into a flexible comb-shaped silicon photodiode array and then mounted onto a spherical structure. The graded refractive index and the flat surface of the micro-lens were optimized to offset the defocusing effects due to changes in the external environment. Put simply, light rays traveling in different mediums (corresponding to different refractive indices) were made to focus at the same spot.
To test the capabilities of their system, the team performed optical simulations and imaging demonstrations in air and water. Amphibious imaging was performed by immersing the device halfway in water. To their delight, the images produced by the system were clear and free of distortions. The team further showed that the system had a panoramic visual field, 300o horizontally and 160o vertically, in both air and water. Additionally, the spherical mount was only 2 cm in diameter, making the system compact and portable.
"Our vision system could pave the way for 360° omnidirectional cameras with applications in virtual or augmented reality or an all-weather vision for autonomous vehicles," speculates Prof. Song excitedly.
And it might be soon!
***
Reference
DOI: https://doi.org/10.1038/s41928-022-00789-9
Authors: Mincheol Lee1,2, Gil Ju Lee3,4, Hyuk Jae Jang3, Eehyung Joh1,2, Hyojin Cho1,2, Min Seok Kim3, Hyun Myung Kim3, Kyeong Muk Kang3, Joong Hoon Lee3, Myungbin Kim1,2, Hongwoo Jang5, Ji-Eun Yeo3, Frédo Durand6, Nanshu Lu5,7,9,10,11, Dae-Hyeong Kim1,2,8, and Young Min Song3
Affiliations:
1Center for Nanoparticle Research, Institute for Basic Science (IBS)
2School of Chemical and Biological Engineering, Institute of Chemical Processes, Seoul National University
3School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology
4Department of Electronics Engineering, Pusan National University
5Texas Materials Institute, University of Texas at Austin
6Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology
7Department of Aerospace Engineering and Engineering Mechanics, The University of Texas at Austin
8Department of Materials Science and Engineering, Seoul National University
9Department of Electrical and Computer Engineering, The University of Texas at Austin
10Department of Biomedical Engineering, The University of Texas at Austin, Austin
11Department of Mechanical Engineering, The University of Texas at Austin
About the Gwangju Institute of Science and Technology (GIST)
The Gwangju Institute of Science and Technology (GIST) is a research-oriented university situated in Gwangju, South Korea. Founded in 1993, GIST has become one of the most prestigious schools in South Korea. The university aims to create a strong research environment to spur advancements in science and technology and to promote collaboration between international and domestic research programs. With its motto of "A Proud Creator of Future Science and Technology," GIST has consistently received one of the highest university rankings in Korea.
Website: http://www.gist.ac.kr/
About the authors
Young Min Song is currently a Professor in the School of Electrical Engineering and Computer Science at Gwangju Institute of Science and Technology (GIST), Korea. He received his Ph.D. in Information and Communications from GIST in 2011. From 2011 to 2013, he was a postdoctoral research associate in the Department of Materials Science and Engineering at the University of Illinois at Urbana-Champaign (UIUC), USA. Over the decades, he has predominantly focused his efforts on developing bio-inspired optics and photonics. His group is developing advanced optoelectronic sensors and systems, multifunctional nanophotonics, and optical healthcare systems.
Dae-Hyeong Kim obtained Ph. D. degree in Materials Science and Engineering from University of Illinois at Urbana Champaign (UIUC) in 2009. He was a post-doctoral research associate at UIUC from 2009 to 2011. He joined Seoul National University in 2011 and is currently a Professor in the School of Chemical and Biological Engineering at Seoul National University. Since 2017, he has also served as an associate director of Center for Nanoparticle Research at Institute for Basic Science (IBS). He has been focusing on the research of nanomaterials and deformable devices and their application to bio-integrated and bio-inspired electronics.
Mincheol Lee is a post-doctoral research associate at Seoul National University. He received his Ph. D. degree in the School of Chemical & Biological Engineering from Seoul National University in 2021. He has been focusing on the research of stretchable electronics based on ultrathin semiconductor films and their application to wearable, implantable, and bio-inspired electronics.
Gil Ju Lee is currently serving as an Assistant Professor in the Department of Electronics Engineering at Pusan National University since 2021. He has intensely focused on the research of advanced photonics and optics for next-generation optoelectronics and optical systems. Before coming to Pusan National University, he completed his Postdoctoral training and doctoral course at Gwangju Institute of Science and Technology (GIST).
Hyuk Jae Jang received his B.S. from Department of Physical and Semiconductor Science at Dongguk University and is a Ph. D. candidate under the guidance of Prof. Young Min Song in GIST. His current research interests focus on bio-inspired nano-optics and imaging device systems.
