When would this avocado be suitably ripe for making guacamole? Is the drug I bought on my travels to far-off places the real thing or a fake? How big an apple crop are we getting this year?
Soon, answers to questions like these will be easily obtained, as computer vision technology developed by researchers will be made available to consumers at a reasonable price.
Mikko Toivonen and Chang Rajani, doctoral students in computer science, together with Assistant Professor Arto Klami, have designed computer vision algorithms that can convert photos taken with a phone into extremely accurate hyperspectral images.
To take hyperspectral images using their solution only requires a camera phone and a peripheral device attached to the phone. Photos are converted by computer vision algorithms into hyperspectral images in a cloud service.
Hyperspectral images reveal more
Hyperspectral images are different from regular photographs because they reveal things unseen to the naked eye in the object photographed. The technique is not based on transillumination; rather, hyperspectral images interpret the wavelengths of light more accurately than regular photos.
"Normal photos use three colour channels, such as red, blue and green. In hyperspectral imaging, the light wavelength resolution is finer, comprising a hundred colour channels," Klami explains.
"A simple three-colour camera is unable to distinguish the spectrum of, for example, chlorophyll. In a hyperspectral image taken outdoors, it's easier to identify the bits with chlorophyll, that is, the areas with vegetation," Toivonen says.
Above is an RGB photo of avocados taken through a peripheral device. Below are the spectra of the avocados' surfaces. The colour-coded squares indicate the areas where the spectra have been shot. The avocado on the right is clearly greener than the others, which can be seen as a spike in the blue spectrum curve at 550 nanometres. The spike indicates that the avocado in question is likely to be less ripe than the others.
Many uses, expensive equipment
Hyperspectral cameras date back years, and there are many uses for such imaging. The technology is used, among other fields, in geographical remote sensing and estimating yield sizes in agriculture. Hyperspectral images can also be used to identify counterfeit art and pharmaceuticals.
"Fake drugs are a problem, especially in developing nations. By using a mobile spectral imaging device, pharmacies and consumers could take a photo of a drug and check whether the pill corresponds with a reference spectrum supplied by the drug manufacturer," says Klami.
However, the devices currently available are specialist equipment, with prices starting from several thousand euros. The less expensive technology developed by the University of Helsinki researchers could bring the solution to regular consumers.
The high cost of previous devices is partly because they produce hyperspectral images independently from start to finish. In the researchers' version, images are converted from regular photographs taken with a smartphone equipped with a peripheral device, and the heavy lifting, or developing the hyperspectral images, is carried out by computer vision algorithms in a cloud service.
"The phone peripheral is cheap and compatible with basically all smartphones. Consumers don't need to buy a separate device," Klami says.
Aiming for the consumer market in 2021
The seed for the idea on consumer market spectral imaging was planted a couple of years ago when Toivonen and Rajani were developing computer vision algorithms for hyperspectral imaging as part of their doctoral studies.
"I noticed that we could also make the algorithms create the image, instead of leaving this cumbersome task to the device taking the pictures," Toivonen describes.
His long-standing interest in amateur photography helped him to make this observation. Prototypes for the smartphone peripheral have also been created with Toivonen's personal 3D printer.
The invention is patent pending and next year begins the search for investors. The team's goal is to make Toivonen's home printer obsolete by 2021, replacing it with a spin-out company running the development of the peripheral device and mobile application.
Arto Klami, Mikko Toivonen and Chang Rajani. Photo: Susan Heikkinen
