Super Resolution: Making the invisible visible

Santa Clara (CA) - Intel is developing a technology that promises to uncover hidden information in digital images and videos and create output files of significantly higher resolution and quality. "Super Resolution" (SR) consumes enormous computing resources, but is on track to reduce the bandwidth required to transmit video files and automatically enhance digital pictures sometime in the future.

Improving image quality never has been an issue in Hollywood. When the crime scene investigation team around Jim Brass in the CBS series CSI: once again finds a new case clue by zooming into an image of rather poor quality, its nothing what viewers would get excited anymore. In real life we however know: A bad picture taken with a digital camera is simply bad luck. There is nothing we can do beyond manipulating those images with Photoshop. Zooming into digital pictures rarely reveals objects we have not seen before.

Intel believes its time for a change. The project "computational nanovision" aims to reconstruct images by employing mathematical algorithms and uncovering invisible or incorrect data in an image.

According to Horst Haussecker, principal engineer of the research, Intel began working on computational nanovision in 2002 with the goal to get a clearer picture of structures smaller than 100 nm. Even the most sophisticated electron microscopes are not able anymore to resolve elements such as transistors in great detail but create more noise instead and deliver a low resolution. "It's not as easy as just cranking up the hardware with smaller structures," said Haussecker in a conversation with Tom's Hardware Guide.

In order to improve picture quality and use information that usually is suppressed in an image, Haussecker's research team aims to understand the relationship between real structures and the taken picture. "Small defects can disappear in a picture for example through artifacts. We can use even such information to make a structure not just look better, but reconstruct it correctly," he explained. Two examples of these types of techniques are 3D reconstruction of nano-scale structures from SEM images, and super-resolution (SR) for increasing the resolution and reducing the noise of images.

Of course, Intel noticed that the technology is not only suited for internal use, but can also help consumer to improve the resolution of their images and videos. While consumers are familiar with artificial resolution enhancers such as interpolation, Haussecker said that SR uses a similar approach but is based on probability calculations rather than bi-cubic tools to add missing pixels. "You cannot hallucinate information to reconstruct an object," he said.

He hesitated to say how much the resolution of a typical image could be improved with SR, but indicated than a factor of at least 2x would likely in most scenarios. Since SR is based on a concept of multiple images, the result will improve with the number of source images used to create one improved picture. According to Haussecker, this also works with moving objects such as humans, since the research team was able to find a solution to reposition "moved pixels" back to their original place.

A typical commercial application of SR is in fact crime scene investigation, since real digital resolution enhancements is very much on the horizon and unclear objects that are blurred for example by artifacts can be uncovered with the technology. The resolution of pictures taken with cell phones will be enough for poster-sized prints and low-resolution videos transmitted via a cellphone could be reconstructed to DVD quality movies.

One of the major challenges SR faces is computing power. To be able to be reconstructed in real time, SR requires "100 to 1000 times" the processing speed available in the highest performing desktop processors today. "Today's processors need about 100 hours of processing time to display one hour of reconstructed video," Haussecker said. It will take many years, until processors will provide such performance, given the outlook of Intel's Pat Gelsinger that the chips will be accelerated by a factor 10 by the end of 2008.

Users should not expect the technology to be real-time capable until 2015. But if we look back to 1995, when a Pentium 75 required a whole night to render a simple low-resolution home video, it is simply impressive to see the long way the PC processor has come. And if we believe Haussecker's vision, we will be able to render on a common home PC a HD quality movie from a source with less than half the resolution - in real time.