Phone App Looks Directly Inside Objects and Displays Specific Constituents

No hyperspectral camera required

How did the development engineers working with project manager Dr. Andreas Herzog manage to get by without a prism? Such scans usually require a special hyperspectral camera: It adjusts to different colored light each time and ascertains how much of a color’s light is reflected by an object, thus generating a complete spectral fingerprint of the object. The development engineers use a mathematical model to extract just about any information on an object, e.g. its constituents, from its spectral fingerprint. 

“Since hyperspectral cameras aren’t integrated in smartphones, we simply reversed this principle,” explains Seiffert. “The camera gives us a broadband three-channel sensor, that is, one that scans every wavelength and illuminates an object with different colored light.” This means that, instead of the camera measuring luminous intensity in different colors, the display successively illuminates the object with a series of different colors for fractions of a second. Thus, if the display casts only red light on the object, the object can only reflect red light – and the camera can only measure red light. Intelligent analysis algorithms enable the app to compensate a smartphone’s limited computing performance as well as the limited performance of the camera and display.

The first laboratory version of the app is finished. The engineers are developing a variety of initial applications before it can be released to private users, though. The system first has to be taught with reference scans how to analyze whether apples contain pesticides. Seiffert is hoping that the “HawkSpex® mobile” app might be launched on the market around the end of 2017.

Reference scans are not always needed, though. Some problems only require measuring different distributions of substances or materials rather than specifying individual constituents. Purchasing a car is one example: In this case, the app compares paint to determine whether it is exactly the same color everywhere or has been touched up.

Users will add applications – much like Wikipedia

“There are so many conceivable uses that the market will surely overrun us,” says Seiffert with certainty. That is why the development engineers are relying on an approach modeled after the online encyclopedia Wikipedia. “Once the app is launched on the market by the end of this year, active users will be able to contribute to the whole big thing and create new applications, for instance, that test pesticide exposure of heads of lettuce, by teaching the system such problems,” says Seiffert. This means they would use the app to scan different types of treated and untreated heads of lettuce and send the data to the Fraunhofer IFF. The development engineers would verify the measurements and release the app to all users.

The app has extremely interesting commercial potential, too, and can be used to develop sectors that wouldn’t really benefit from high precision scanners. Examples include quality control of foods, the effectiveness of cosmetic products or even agriculture: Farmers, for instance, could easily obtain information on whether their crops are sufficiently supplied with nutrients or fertilizer is needed.

Contacts and sources: 

M.A. René Maresch 

Fraunhofer Institute for Factory Operation and Automation IFF