For anyone who’s ever wanted to see in infrared like Superman, humans aren’t quite there yet, but robots are getting closer and closer.
Our species doesn’t really have super vision. We can only see in a narrow part of the spectrum, called visible light for a reason. But other than that, other wavelengths are either too long (like infrared and radio) or too short (like UV). The colors we see depend on the wavelengths of light that the eye picks up when they bounce off something. We also have trouble distinguishing colors when there is little or no visible light around. Although a Kryptonian or Daxamite would laugh at our terrible night vision, a new AI can now see in the dark – in colour.
Infrared night vision goggles gave us a wider field of vision. The problem is that they still see in black and white, but now UC Irvine researchers, who recently published a study in PLOS ONE, have developed an AI that can automatically determine what colors are supposed to be on a human face, even when that face is in shadow. They were able to use visible light and several different wavelengths of infrared light to teach the AI the colors seen in human faces and how to recognize them in the absence of light. Kal-el probably couldn’t do that.
“We sought to develop an imaging algorithm powered by optimized deep learning architectures in which the infrared spectral illumination of a scene could be used to predict a rendering of the visible spectrum of the scene as perceived by a human with visible spectrum light”, they said.
So what the AI was able to see is what we would see if our eyes had evolved color vision for darkness. Most of the night vision technologies we know of use infrared light to understand what they see, and the images they produce are then transformed into monochrome visible light images. There is advanced night vision technology that can pick up and amplify visible light in an area to produce an image. While earlier versions of AI night vision exist and were able to detect objects in the infrared, there were still inaccuracies when trying to translate them into color images as they would appear in the visible part of the spectrum. .
What this AI did differently was learn to take what it saw with infrared and near-infrared vision and then reconstruct it in the visible part of the spectrum so we could actually see it. The researchers tested its accuracy using red, green, and blue (RGB) images illuminated with infrared light. This means that all colors can be mixed in the images as long as they are only mixed from these three colors. The researchers figured out how to translate each of these colors from appearing in the infrared to appearing in the visible part of the spectrum, so that the AI would recognize them in the dark.
The bot had to learn to recognize the colors involved in human faces if it wanted to recognize them. For this, images of real faces were taken with a monochrome camera that could respond to both visible and infrared light. The same camera was also used to take pictures in the dark. Now that he knew what colors should be on a human face, he was able to use what he had learned to guess what colors were there and correctly generate images that looked like they were taken in visible light, maybe slightly darker but still eerily similar.
“This work suggests that the prediction of high-resolution images depends more on the formation context than on spectroscopic signatures”, the researchers said.
One day, maybe this technology could be used to create the ultimate night vision goggles that see in color even when the lights go out. Goodbye Superman.