Google uses machine learning to upscale and enhance low quality photos

Image upscaling is often made fun of in movies and television series, for extracting impossible levels of image information from low quality, blurry photos. Typically law enforcement agencies approach a technician with blurry images, and after a few keystrokes, a high resolution images suddenly emerges. Google has developed technology called Rapid and Accurate Image Super-Resolution ( RAISR), that actually works similar to what is shown in this clip from CSI.

The technology is not just for law enforcement agencies. The rapid evolution of both cameras and screens means that all of us have collections of older low resolution photos that we look at in newer high resolution screens. While there are upscaling and enhancing techniques around, none of them can fill in the missing information. Mostly, these algorithms work by guessing pixels based on the pixels nearby, creating aliasing artifacts. Google has a machine learning approach that does not directly bring in the missing image information, but does preserve and build on the underlying structure. [caption id=“attachment_347612” align=“aligncenter” width=“640”]  Google RAISR in action. Image: Google.[/caption] The technique uses machine learning to train on two sets of images, a high resolution version, and a low resolution version. The machine generates a set of filters that compares the two versions, and then enhances the low resolution image with the filters. If a high resolution image is not available, the technique can still be used. The AI is trained on the upscaled image based on traditional linear upscaling techniques. [caption id=“attachment_347614” align=“aligncenter” width=“640”]  Image: Google[/caption] RAISR can be used on mobile phones as well, and processes the images 10 to 100 times faster than traditional methods. The enhanced images are less blurry than current approaches. The RAISR technique can be used to improve pinch to zoom functionality in mobile phones. Lower resolution images can be sent over communication services to conserve on bandwidth, and these images can be restored to the original quality by the recipient. You can see more examples of RAISR in action here.