CupScale, an OpenSource software for UpScaling Images and Videos

Zoom in ... Enhance...

With playing around with local Stable Diffusion it became obvious that upscaling of generated images is needed as generated images are relatively low resolution. Most people online recommend using Topazlab’s gigapixel-ai but you need to pay for it and I wanted to check out if there’s an open-source approach.

Holly F* - OneClick Stable Diffusion local install

As it turns out, there is. In some aspects, it turned out to be an even better solution than the commercial ones. It’s called CupScale and is an image upscaling GUI based on Enhanced Super-Resolution Generative Adversarial Network (ESRGAN). The ESRGAN architecture was first proposed in 2018 and is an improvement on Super-Resolution Generative Adversarial Network that suffers from occasionally hallucinating details that result in unpleasant artifacts. ESRGAN architecture achieves consistently better visual quality with more realistic and natural textures than SRGAN and that approach won first place in the PIRM2018-SR Challenge.

Features of Cupscale:

• There are no restrictions on the number of times you can use it to enhance the photographs.

• The software supports batch processing, thereby allowing enhancing multiple images simultaneously.

• Chaining multiple AI models. You can either use them together or interject between the changing models.

• Compatible with GIF, JPEG, PNG, WEBP, DDS, and PNG images alongside MP4 for video.

• Create various types of comparisons (Side-By-Side, 50/50, and before/after animations as GIF or MP4).

• Supports Nvidia CUDA, Vulkan (Works on any modern GPU), and CPU (Works without GPU, but is very slow)

Installation

Installation of CupScale is simple but does require several steps as a one-click installer is not made yet but according to the GitHub page, it is planned. There is an automated installer for all dependencies.

If you run the CupScale and click the Dependency checker, Its second icon from the left on the top right side menu bar, there is an option to install embedded Python and automatically install all of the dependencies.

But I like to install them myself so in addition to CupScale, you need to install Python 3, PyTorch, and OpenCV.

Python installation is straight forward but be sure you enable ADD TO PATH option in the installer. Without it, the system will not know where the installation is and everything will be more tedious.

After installing Python3 you will have access to its package installer called PIP. Go to PyTorch (an open-source machine learning framework) website and select what build, system, installation method, and CUDA version you want. Then copy the command and paste it into your command line.

Installation of the Open Computer Vision library is no more complicated. After PyTorch is installed type in the command in the same terminal window: pip install opencv-python

And that’s it, all your prerequisites are installed. Now, create a folder anywhere on your system and unpack the CupScale release into it. In the root of that folder, you will see Cupscale.exe, run it, and that’s it. To check if everything is properly installed try running a dependency checker from within the CupScale.

Testing

To test let’s use our Lenna test image.

The original is 1024x1024 pixels and was resized to 128x128 pixels.

There is no zoom slider or alike. How much image will be enlarged is determined by the model so 1x (denoising etc) or 4x in front of the model indicates the scaleup factor.

CupScale comes with 3 ESRGAN AI networks bundled in are: RealSR (DF2K_JPEG), BSRGAN, and Real-ESRGAN.

RealSR (DF2K_JPEG), or Real-World Super-Resolution via Kernel Estimation and Noise Injection won a contest called CVPR NTIRE 2020 Challenge on Real-World Super-Resolution. RealSR does a really good job at up-scaling photos. It is better than Waifu2x when up-scaling photos and comparable to it when up-scaling anime and other cartoons.

BSRGAN or deep Blind ESRGAN super-resolver that build upon RealSR dataset and was trained with more various distortions, blurs, etc. And, can be seen as an improvement on RealSR but is more prone to hallucination making it good at filling in missing details.

Real-ESRGAN is the fastest model that helps retain more details, but unfortunately due to the dataset used to train it still blurs details adjacent to other objects.

Comparison of Original 128x128, RealSR, RealESRGAN, BSRGAN

It is amazing what can be interpolated from just a couple of pixels.

Which model you choose is up to you and your use case but BSRGAN looks like it’s the best for low-resolution images where there are not a lot of details as it will fill them in.

But when looking at preserving original details without any modification it is the worst. If upscaling an already larger image, let’s say Lenna that is 1024x1024 pixels the results are quite different. The best model is RealESRGAN with BSRGAN looking the worst, most modified.

Comparison of Original 1024x1024, RealSR, RealESRGAN, BSRGAN

Additional Models

Now, you don’t need to use only those models. You could download any ESRGAN model and by placing it in the model’s folder it will become available on the model selection page. Additional models can be found on UpScale Wiki page and NMKD ESRGAN page. But the included 3 models proved to be the best out of the bunch.

Conclusion

CupScale is an amazing piece of software that should be in every designer’s tool case. Not only for upscaling but photo restoration and denoising. The results are quite comparable to Topazlab’s gigapixel-ai so give it a try.

Video upscaling was not tested here but will check it out in the future as it opens a lot of different possibilities.