What is super resolution artificial intelligence?

Super Resolution AI is a process where our input is a coarse, low-resolution video, and the output is the same video, but now with more details and in high resolution. We also refer to this process as image upscaling.

Here is a Podcast of how a fulldome company in Los Angeles uses Super Resolution AI on their projects:

Despite the incredible results seen in practically any of the crime-solving television shows out there, our intuition would perhaps say that this problem, at first sight, sounds impossible.

How could one mathematically fill in the details when these details are completely unknown?

Well, that's only kind of true.

First, let's not confuse super resolution with image inpainting, where we essentially cut an entire part out of an image and try to replace it’, by leaning on our knowledge of the surroundings of the missing part. Also known as image interpolation or video interpolation or Content Aware, those are algorithms all of their own.

With Super Resolution AI, the entirety of the image is known, and the details just require some enhancing. This particular method is not based on neural networks but is a learning-based technique.

Netflix, Hulu, Disney and most of your streaming services use some sort of curriculum learning algorithms in order to give you the best possible video image for the bandwidth.

The best thing about Super Resolution AI is that we can just grab a high-resolution image, convert it to a lower resolution and we immediately have our hands on a training example for the learning algorithm.

During learning, video is converted into frames (images), the images are subdivided into small image patches, and buckets are created to aggregate the information between patches that share similar features.

These features include brightness, textures, and the orientation of the edges. The technique looks at how the small and large resolution images relate to each other when viewed through the lens of these features.

Two remarkably interesting things arose from the research and development of Super Resolution AI:

1. It outperforms existing neural network-based techniques,

2. It only uses 10 thousand images, and one hour of training time, which is in the world of deep neural networks, is so little, it's completely unheard of.

We are close to the point where we could choose to transmit only the lower resolution images through the network and perform super resolution on them locally on our phones and computers. This will lead to significant savings on network bandwidth.