Difference between revisions of "Using ESRGAN, Links, And Other Information"

From Upscale Wiki
Jump to navigation Jump to search
(Added usage instructions for Joey's fork)
 
(22 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
{| class="wikitable"
 +
|-
 +
| Make sure that you have followed the installation guide carefully ([https://upscale.wiki/wiki/ESRGAN_Installation_Guide_for_Windows Windows], [https://upscale.wiki/wiki/Installing_ESRGAN_and/or_BasicSR_on_Arch_Linux Arch Linux], [https://upscale.wiki/wiki/ESRGAN_Installation_Guide_for_MacOS MacOS]). We support BlueAmulets fork offically, not the current ESRGAN branch from xinntao, if you use that one, you will get errors.
 +
|}
 +
 
Now that you have installed ESRGAN you can upscale images. There are a few different ways of using ESRGAN. Below I will document the official one. This should work for everyone, but there are a few different applications designed to make the life of the users easier as well as to prevent common pitfalls of using ESRGAN.
 
Now that you have installed ESRGAN you can upscale images. There are a few different ways of using ESRGAN. Below I will document the official one. This should work for everyone, but there are a few different applications designed to make the life of the users easier as well as to prevent common pitfalls of using ESRGAN.
  
 
==Things you need to know before you use ESRGAN==
 
==Things you need to know before you use ESRGAN==
 +
 
* ESRGAN supports only RGB images, that means it will remove alpha / transparency channels if present and it won't work with grayscale images.
 
* ESRGAN supports only RGB images, that means it will remove alpha / transparency channels if present and it won't work with grayscale images.
 
* ESRGAN is limited by the amount of VRAM you have.
 
* ESRGAN is limited by the amount of VRAM you have.
 
'''But''' there are ways around both. At the time of writing this there are some popular tools used by a lot of us to solve this:
 
'''But''' there are ways around both. At the time of writing this there are some popular tools used by a lot of us to solve this:
  
===[https://github.com/ptrsuder/IEU.Winforms Honh's / Ptrsuder's IEU (Image Enhancement Utility)]===
+
== Tools / Ways to use ESRGAN ==
 +
 
 +
=== GUI Tools ===
 
----
 
----
* '''Easy''' to use '''GUI''' which makes it easy to use for users who don't love CLI yet
+
==== [https://github.com/ptrsuder/IEU.Winforms ptrsuder's IEU (Image Enhancement Utility)] ====
* Image splitting works completely different compared to Deorder's scripts
+
----
* '''Windows only''' for now. (Linux support is being worked on by the author, but not public yet)
+
IEU is the recommended option out there right now, it has been used by many people and is considered mature, but if you encounter bugs or have feature requests, feel free to report them on Discord, or on GitHub repository by creating an issue.
* Has a '''lot of advanced options''' available without the need to write any scripts
+
The '''Download''' is under the [https://github.com/ptrsuder/IEU.Winforms/ IEU.Winforms] Releases tab.
* Some new unique features:
+
Please take a bit of your time to read [https://github.com/ptrsuder/IEU.Winforms/wiki/Basic-usage IEU wiki] first.
** It can upscale seamless(tiled) textures (landscape textures for example)
+
 
** It can use a different model for the alpha channel
+
===== Advantages: =====
** It can upscale each channel separately (Red, Green, Blue and Alpha)
+
 
** It can now interpolate models right in the UI
+
* Easy to use
* It is still somewhat experimental, so please report any bugs and issues you have to Honh on Discord, or on GitHub repository
+
* Process any amount of images with all models you select via check-boxes
* The '''Download''' is under [https://github.com/ptrsuder/IEU.Winforms/releases IEU.Winforms Releases]
+
* Control over the output format and naming scheme
 +
* Works around VRAM limitations
 +
* Has a lot of advanced features available without the need to write any scripts
 +
* Can upscale seamless(tiled) textures (landscape textures for example)
 +
* Can use a different model for the alpha channel
 +
* Can upscale each channel separately (Red, Green, Blue and Alpha)
 +
*: This is useful for non diffuse textures, so normal maps (_n.dds typically), specular maps, ...
 +
* Can interpolate different models right in the GUI
 +
* Uses panorama image stitching to merge images. That makes it work with images of any resolution without cropping.
  
===[https://github.com/deorder/texture-upscaler Deorder's scripts (CTP)]===
+
===== Disadvantages =====
 +
 
 +
* No cross platform support
 +
* GUI only, no CLI support
 +
* Uses panorama image stitching to merge images. That makes it slower and blends slightly worse. Most people will likely not notice the difference.
 +
 
 +
=== CLI Tools ===
 
----
 
----
* They require '''Bash''' and imagemagick, both of which are available for '''Windows, Linux and MacOS''', on Windows you can use git bash, msys2 or Cygwin. There are others, but those are the main options
+
==== [https://github.com/JoeyBallentine/ESRGAN Joey's ESRGAN fork] ====
* They run from a bash terminal. That makes them harder to use for many people who never used a CLI (command line interface) tool, but allow for an incredible automation potential
+
----
* They include scripts for training as well as using ESRGAN
+
===== Advantages =====
* You can use them for a lot of other Neural Networks not just BasicSR / ESRGAN
+
 
* '''Don’t use [https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux WSL (Windows Subsystem for Linux)]''' if you run ESRGAN on a '''NVIDIA''' GPU since WSL '''doesn’t support GPU acceleration''', which makes it unable to run ESRGAN in CUDA mode
+
* Cross-Platform
* The scripts follow UNIX philosophy, each script has one function, that way you can script basically anything you want thanks to the power of bash, it allows you to emulate every feature of other tools like IEU and more, if you have the time and patience to do so
+
* Works on headless systems
 +
* Auto splits and merges images in memory
 +
* It has a seamless option
 +
* It can chain models
 +
* Barebones, so you can script whatever you want around them
 +
 
 +
===== Disadvantages =====
 +
 
 +
* You have to enter the full model names if you don't wrap a script around it
 +
 
 +
===== How to use it =====
 +
 
 +
# Put the pictures or textures you want to upscale into the <code>input</code> folder
 +
# Open a terminal window and navigate to the <code>esrgan</code> folder (the folder you cloned the fork into in the install guide)
 +
#* For Windows Shift right click in your <code>esrgan</code> folder and select <code>Git Bash Here</code>.
 +
#* For Linux / MacOS users the process is similar. File managers like Nautilus for example allow you to open a terminal in a folder. If that isn't an option for you can can also navigate using commands. <code>cd</code> allows you to navigate and pwd shows you the current folder. <code>cd ..</code> goes one directory down (for example from <code>/home/combi/code/git/ctp</code> to <code>/home/combi/code/git</code>). Use <code>cd /path/to/whatever</code> to navigate to absolute (full) paths or <code>cd some-folder-in-the-current-folder</code> to navigate to a folder in the current open folder. (<code>pwd</code> = print working directory; cd = change directory) If you want to find out more about a command you can just type <code>man the command-you-want-to-know-about</code> or use the internet
 +
# Enter:
 +
## For Nvidia GPUs
 +
##: <code>python upscale.py models/${theModelYouWantToUse}</code>
 +
## For other GPUs / integrated Graphic
 +
##: <code>python upscale.py --cpu models/${theModelYouWantToUse}</code>
 +
# Don't enter <code>${theModelYouWantToUse}</code> Instead replace that with the name of a model of course As an example, for the default model it would be: <code>python test.py models/RRDB_ESRGAN_x4.pth</code>
 +
# That was it, the results will be in the <code>results</code> folder
 +
 
 +
Other useful Command line switches are:
 +
 
 +
* <code>--input ./input</code> - Your Input folder
 +
* <code>--seamless</code> - For seamless textures (Textures that repeat without seams)
 +
* <code>--output ./output</code> - Your Output folder
 +
* <code>--skip_existing</code> - Skipps already upscaled images that are in your output and input folder
 +
* <code>--tile_size 512</code> - The tile size, if you run out of VRAM, decrease that value
 +
* <code>--binary_alpha</code> - If the alpha / transparency channel should be only black or white
 +
* <code>--alpha_threshold 0.5</code> Sets tha point at which alpha will be white instead of black if using binary alpha
 +
* <code>--alpha_mode 1</code> Different implementation for how Alpha works, if you use binary alpha, it must be 1 otherwise you can try 2.
 +
 
 +
----
 +
==== Blue Amulet's Esrgan test.py script (Guide way) ====
 +
----
 +
 
 +
===== Advantages =====
 +
 
 +
* Nothing additional needs to be installed
 +
* Fastest option out there
 +
 
 +
===== Disadvantages =====
 +
 
 +
* Only works for some image formats (yes, jpg and png work)
 +
* No splitting of images, so you can run out of VRAM easily as images get bigger
 +
* You are limited by the amount of channels a model supports. Nearly all models out there require images to have exactly 3 channels (RGB).
 +
 
 +
===== How to use it =====
  
==Use ESRGAN The official Way (just ESRGAN, no other tool)==
 
 
# Put the pictures or textures you want to upscale into the <code>LR</code> folder
 
# Put the pictures or textures you want to upscale into the <code>LR</code> folder
# If you want to use a model with a scale other than 4, you need to edit the test.py file. Just open it with your text editor and change the scale to the one from your model. The scale of each model is documented on our wiki. If you want to run an artifact removal model, like my jpg model for example change <code>scale=4</code> to <code>scale=1</code>
 
 
# Open a terminal window and navigate to the <code>esrgan</code> folder
 
# Open a terminal window and navigate to the <code>esrgan</code> folder
#* For Windows Shift right click in your <code>esrgan</code> folder and select <code>Open PowerShell window here</code>. For some users it might say <code>Command Line</code> instead, if that is the case for you click on that and procced
+
#* For Windows Shift right click in your <code>esrgan</code> folder and select <code>Git Bash Here</code>.
 
#* For Linux / MacOS users the process is similar. File managers like Nautilus for example allow you to open a terminal in a folder. If that isn't an option for you can can also navigate using commands. <code>cd</code> allows you to navigate and pwd shows you the current folder. <code>cd ..</code> goes one directory down (for example from <code>/home/combi/code/git/ctp</code> to <code>/home/combi/code/git</code>). Use <code>cd /path/to/whatever</code> to navigate to absolute (full) paths or <code>cd some-folder-in-the-current-folder</code> to navigate to a folder in the current open folder. (<code>pwd</code> = print working directory; cd = change directory) If you want to find out more about a command you can just type <code>man the command-you-want-to-know-about</code> or use the internet
 
#* For Linux / MacOS users the process is similar. File managers like Nautilus for example allow you to open a terminal in a folder. If that isn't an option for you can can also navigate using commands. <code>cd</code> allows you to navigate and pwd shows you the current folder. <code>cd ..</code> goes one directory down (for example from <code>/home/combi/code/git/ctp</code> to <code>/home/combi/code/git</code>). Use <code>cd /path/to/whatever</code> to navigate to absolute (full) paths or <code>cd some-folder-in-the-current-folder</code> to navigate to a folder in the current open folder. (<code>pwd</code> = print working directory; cd = change directory) If you want to find out more about a command you can just type <code>man the command-you-want-to-know-about</code> or use the internet
 
# Enter:
 
# Enter:
#: <code>python test.py models/${theModelYouWantToUse}</code>
+
## For Nvidia GPUs
 +
##: <code>python test.py models/${theModelYouWantToUse}</code>
 +
## For other GPUs / integrated Graphic
 +
##: <code>python test.py --cpu models/${theModelYouWantToUse}</code>
 
# Don't enter <code>${theModelYouWantToUse}</code> Instead replace that with the name of a model of course As an example, for the default model it would be: <code>python test.py models/RRDB_ESRGAN_x4.pth</code>
 
# Don't enter <code>${theModelYouWantToUse}</code> Instead replace that with the name of a model of course As an example, for the default model it would be: <code>python test.py models/RRDB_ESRGAN_x4.pth</code>
 
# That was it, the results will be in the <code>results</code> folder
 
# That was it, the results will be in the <code>results</code> folder
  
==Tips when using ESRGAN==
+
----
 +
==== [https://github.com/deorder/texture-upscaler Deorder's scripts] ====
 +
----
 +
===== Advantages =====
 +
 
 +
* Cross-Platform
 +
* Works on headless systems
 +
* Script for splitting images
 +
* Script for merging images (to work around VRAM limitations)
 +
* Barebones, you can use them for a lot of other Neural Networks
 +
* Barebones, so you can script whatever you want around them
 +
 
 +
===== Disadvantages =====
 +
 
 +
* Outdated, they use ImageMagick 6
 +
* Problems when using it for images with dimensions not dividable though the patch size (by default 256x256)
 +
* '''Don’t use [https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux WSL (Windows Subsystem for Linux)]''' if you run ESRGAN on a '''NVIDIA''' GPU since WSL '''doesn’t support GPU acceleration''', which makes it unable to run ESRGAN in CUDA mode. Use git bash or something similar instead.
 +
 
 +
===== Requirements =====
 +
 
 +
* bash (We install that via the windows guide. MacOS and Linux tend to come with it out of the box, no need to do anything for those there)
 +
* imagemagick
 +
 
 +
== Tips when using ESRGAN ==
 +
 
 
# When upscaling '''compressed textures''' use a 1x decompression model for the format first and or downscale them first by at least 50% with ,code>nearest neighbor</code> or <code>box</code> filtering first, before upscaling them in ESRGAN
 
# When upscaling '''compressed textures''' use a 1x decompression model for the format first and or downscale them first by at least 50% with ,code>nearest neighbor</code> or <code>box</code> filtering first, before upscaling them in ESRGAN
 
# ESRGAN runs '''much''' faster on Nvidia GPUs, you can compile pytorch yourself for AMD GPUs but at the moment that is quite difficult to do
 
# ESRGAN runs '''much''' faster on Nvidia GPUs, you can compile pytorch yourself for AMD GPUs but at the moment that is quite difficult to do
# If you '''run out of VRAM''', use '''Deorder’s scripts''', which will split the texture in smaller parts first
+
# If you '''run out of VRAM''', use '''Deorder’s scripts''' or IEU, which will split the texture in smaller parts first
# If you have textures in sub-folders, use Deorder’s scripts, they fully support sub-folders
+
# If you have textures in sub-folders, use Deorder’s scripts or IEU with "Preserve folder structure" mode selected
# If your textures contain an alpha channel (transparency), use Deorder’s scripts, split and recombine the color and alpha channels
+
# If your textures contain an alpha channel (transparency), use Deorder’s scripts or IEU
# Try out different models. On our GitHub you will find a lot of different Models, that we trained our self
+
# Try out different models. In our [https://upscale.wiki/wiki/Model_Database Model Database] you will find a lot of different models, that we trained ourself
 
# If you are still not happy with the results despite having tried out different models, consider training your own and sharing it with us later
 
# If you are still not happy with the results despite having tried out different models, consider training your own and sharing it with us later
  
==Additional Information==
+
== Additional Notices ==
===Consider Linux===
+
 
** Especially if you are on Windows, to get a noticable speed boost, almost all Neural Networks  including ESRGAN / BasicSR were made with Linux in mind. The speed boost is especially noticable when training and / or when using Deorder's scripts
+
=== Windows ===
** There are a lot of different Linux Distributions, you can choose any of them, if you decide to go that route:
+
 
*** '''I recommend [https://www.archlinux.org/ Arch Linux]''' unlike most other Distributions it allows you to completely customize your installation, but requires a lot of time and patience, there is a nice written guide on it's wiki. The main advantage beside the possible customization is that arch has no version it is a rolling release, which means as soon as there is an update you will get it, that includes the Linux kernel and drivers, so you might have a better time on arch based distributions than on others
+
* When upscaling large images (depending on your GPU for example 1000x1000px images) on Windows, it's possible for the operating system to kill the ESRGAN process if it takes too long. This can be fixed using the Nvidia Nsight Monitor app that is installed alongside the CUDA toolkit. [https://docs.nvidia.com/gameworks/content/developertools/desktop/timeout_detection_recovery.htm Here are Instructions for doing so]
****If you don't have time or patience to install it the hard way, [https://manjaro.org/ Manjaro] is also a good option which is based on Arch Linux
 
**** Finally, while I don’t like it, many people use [https://www.ubuntu.com/ Ubuntu], many tools where made on that distribution, mainly because so many people use it, but despite that you might have a harder time there, especially with installing the NVIDIA driver and CUDA
 
===Windows===
 
** When upscaling large images (depending on your GPU for example 1000x1000px images) on Windows, it's possible for the operating system to kill the ESRGAN process if it takes too long. This can be fixed using the Nvidia Nsight Monitor app that is installed alongside the CUDA toolkit. [https://docs.nvidia.com/gameworks/content/developertools/desktop/timeout_detection_recovery.htm Here are Instructions for doing so]
 

Latest revision as of 20:41, 24 September 2020

Make sure that you have followed the installation guide carefully (Windows, Arch Linux, MacOS). We support BlueAmulets fork offically, not the current ESRGAN branch from xinntao, if you use that one, you will get errors.

Now that you have installed ESRGAN you can upscale images. There are a few different ways of using ESRGAN. Below I will document the official one. This should work for everyone, but there are a few different applications designed to make the life of the users easier as well as to prevent common pitfalls of using ESRGAN.

Things you need to know before you use ESRGAN

  • ESRGAN supports only RGB images, that means it will remove alpha / transparency channels if present and it won't work with grayscale images.
  • ESRGAN is limited by the amount of VRAM you have.

But there are ways around both. At the time of writing this there are some popular tools used by a lot of us to solve this:

Tools / Ways to use ESRGAN

GUI Tools


ptrsuder's IEU (Image Enhancement Utility)


IEU is the recommended option out there right now, it has been used by many people and is considered mature, but if you encounter bugs or have feature requests, feel free to report them on Discord, or on GitHub repository by creating an issue. The Download is under the IEU.Winforms Releases tab. Please take a bit of your time to read IEU wiki first.

Advantages:
  • Easy to use
  • Process any amount of images with all models you select via check-boxes
  • Control over the output format and naming scheme
  • Works around VRAM limitations
  • Has a lot of advanced features available without the need to write any scripts
  • Can upscale seamless(tiled) textures (landscape textures for example)
  • Can use a different model for the alpha channel
  • Can upscale each channel separately (Red, Green, Blue and Alpha)
    This is useful for non diffuse textures, so normal maps (_n.dds typically), specular maps, ...
  • Can interpolate different models right in the GUI
  • Uses panorama image stitching to merge images. That makes it work with images of any resolution without cropping.
Disadvantages
  • No cross platform support
  • GUI only, no CLI support
  • Uses panorama image stitching to merge images. That makes it slower and blends slightly worse. Most people will likely not notice the difference.

CLI Tools


Joey's ESRGAN fork


Advantages
  • Cross-Platform
  • Works on headless systems
  • Auto splits and merges images in memory
  • It has a seamless option
  • It can chain models
  • Barebones, so you can script whatever you want around them
Disadvantages
  • You have to enter the full model names if you don't wrap a script around it
How to use it
  1. Put the pictures or textures you want to upscale into the input folder
  2. Open a terminal window and navigate to the esrgan folder (the folder you cloned the fork into in the install guide)
    • For Windows Shift right click in your esrgan folder and select Git Bash Here.
    • For Linux / MacOS users the process is similar. File managers like Nautilus for example allow you to open a terminal in a folder. If that isn't an option for you can can also navigate using commands. cd allows you to navigate and pwd shows you the current folder. cd .. goes one directory down (for example from /home/combi/code/git/ctp to /home/combi/code/git). Use cd /path/to/whatever to navigate to absolute (full) paths or cd some-folder-in-the-current-folder to navigate to a folder in the current open folder. (pwd = print working directory; cd = change directory) If you want to find out more about a command you can just type man the command-you-want-to-know-about or use the internet
  3. Enter:
    1. For Nvidia GPUs
      python upscale.py models/${theModelYouWantToUse}
    2. For other GPUs / integrated Graphic
      python upscale.py --cpu models/${theModelYouWantToUse}
  4. Don't enter ${theModelYouWantToUse} Instead replace that with the name of a model of course As an example, for the default model it would be: python test.py models/RRDB_ESRGAN_x4.pth
  5. That was it, the results will be in the results folder

Other useful Command line switches are:

  • --input ./input - Your Input folder
  • --seamless - For seamless textures (Textures that repeat without seams)
  • --output ./output - Your Output folder
  • --skip_existing - Skipps already upscaled images that are in your output and input folder
  • --tile_size 512 - The tile size, if you run out of VRAM, decrease that value
  • --binary_alpha - If the alpha / transparency channel should be only black or white
  • --alpha_threshold 0.5 Sets tha point at which alpha will be white instead of black if using binary alpha
  • --alpha_mode 1 Different implementation for how Alpha works, if you use binary alpha, it must be 1 otherwise you can try 2.

Blue Amulet's Esrgan test.py script (Guide way)


Advantages
  • Nothing additional needs to be installed
  • Fastest option out there
Disadvantages
  • Only works for some image formats (yes, jpg and png work)
  • No splitting of images, so you can run out of VRAM easily as images get bigger
  • You are limited by the amount of channels a model supports. Nearly all models out there require images to have exactly 3 channels (RGB).
How to use it
  1. Put the pictures or textures you want to upscale into the LR folder
  2. Open a terminal window and navigate to the esrgan folder
    • For Windows Shift right click in your esrgan folder and select Git Bash Here.
    • For Linux / MacOS users the process is similar. File managers like Nautilus for example allow you to open a terminal in a folder. If that isn't an option for you can can also navigate using commands. cd allows you to navigate and pwd shows you the current folder. cd .. goes one directory down (for example from /home/combi/code/git/ctp to /home/combi/code/git). Use cd /path/to/whatever to navigate to absolute (full) paths or cd some-folder-in-the-current-folder to navigate to a folder in the current open folder. (pwd = print working directory; cd = change directory) If you want to find out more about a command you can just type man the command-you-want-to-know-about or use the internet
  3. Enter:
    1. For Nvidia GPUs
      python test.py models/${theModelYouWantToUse}
    2. For other GPUs / integrated Graphic
      python test.py --cpu models/${theModelYouWantToUse}
  4. Don't enter ${theModelYouWantToUse} Instead replace that with the name of a model of course As an example, for the default model it would be: python test.py models/RRDB_ESRGAN_x4.pth
  5. That was it, the results will be in the results folder

Deorder's scripts


Advantages
  • Cross-Platform
  • Works on headless systems
  • Script for splitting images
  • Script for merging images (to work around VRAM limitations)
  • Barebones, you can use them for a lot of other Neural Networks
  • Barebones, so you can script whatever you want around them
Disadvantages
  • Outdated, they use ImageMagick 6
  • Problems when using it for images with dimensions not dividable though the patch size (by default 256x256)
  • Don’t use WSL (Windows Subsystem for Linux) if you run ESRGAN on a NVIDIA GPU since WSL doesn’t support GPU acceleration, which makes it unable to run ESRGAN in CUDA mode. Use git bash or something similar instead.
Requirements
  • bash (We install that via the windows guide. MacOS and Linux tend to come with it out of the box, no need to do anything for those there)
  • imagemagick

Tips when using ESRGAN

  1. When upscaling compressed textures use a 1x decompression model for the format first and or downscale them first by at least 50% with ,code>nearest neighbor or box filtering first, before upscaling them in ESRGAN
  2. ESRGAN runs much faster on Nvidia GPUs, you can compile pytorch yourself for AMD GPUs but at the moment that is quite difficult to do
  3. If you run out of VRAM, use Deorder’s scripts or IEU, which will split the texture in smaller parts first
  4. If you have textures in sub-folders, use Deorder’s scripts or IEU with "Preserve folder structure" mode selected
  5. If your textures contain an alpha channel (transparency), use Deorder’s scripts or IEU
  6. Try out different models. In our Model Database you will find a lot of different models, that we trained ourself
  7. If you are still not happy with the results despite having tried out different models, consider training your own and sharing it with us later

Additional Notices

Windows

  • When upscaling large images (depending on your GPU for example 1000x1000px images) on Windows, it's possible for the operating system to kill the ESRGAN process if it takes too long. This can be fixed using the Nvidia Nsight Monitor app that is installed alongside the CUDA toolkit. Here are Instructions for doing so