✨ AniSD Suite Documentation ‐ Faithful Anime DVD Restoration - Sirosky/Upscale-Hub GitHub Wiki

⏩ Quickstart / TL;DR

  • General-Purpose: 2x_AniSD_G6i1_SPAN_215K and 2x_AniSD_AC_G6i2b_SPAN_190K. These should be capable of handing most DVD sources, but if not, look at the Heavyweight Options below.
  • Heavyweight Options: 2x_AniSD_AC_RealPLKSR_127500. RealPLKSR demonstrated superior generalization capabilities to even SwinIR, while consuming far less VRAM and roughly the same inference speed. Depending on system specifications, DirectML inference and TensorRT inference are both good options. 2x_AniSD_DC_DAT2_97500 is probably king in terms of quality, but I strongly recommend having a 4090 + TensorRT for DAT2 video inference (and a lot of patience!).

AniSD Chart

🙆 Overview

AniSD is a suite of specialized SISR models trained to restore and upscale standard definition digital anime from ~2000s and onwards, including both both WEB and DVD releases. Faithfulness to the source and natural-looking output are the guiding principles behind the training of the AniSD models. This means avoiding oversharpened output (which can look especially absurd on standard definition sources), minimizing upscaling artifacts, retaining the natural detail of the source and of course, fixing the standard range of issues found in many DVD/WEB release (chroma issues, compression, haloing/ringing, blur, dotcrawl, banding, etc.).

To achieve this all this, the AniSD models are specialized to fulfill specific roles and types of sources, such that you can select the best model(s) for your needs. There are several categories of AniSD models:

Upscaling

  • AniSD: These are the baseline AniSD models. They are intended for "newer" standard definition sources, including WEB releases. Will handle low compression, noise, haloing and chroma issues.
  • AniSD Augmented Clean (AniSD AC): As the name suggested, the AniSD AC models are trained to deal with stronger degradations. They will handle medium to heavy compression, noise, haloing, chroma and dotcrawl issues.
  • AniSD Deep Clean (AniSD DC): The final stand, so to speak. If a source isn't properly cleaned up by AniSD or AniSD AC, that's when you use AniSD DC. These models will handle heavy compression, noise, haloing, chroma and dotcrawl. They may even work well on 90s sources. However, this comes at the expense of inference speed.

Utility

  • AniSD PostScale (AniSD PS): As the AniSD upscaling models are all 2x models, AniSD PS serves two purposes. First, if you want to 4x a source, you can run it through an AniSD upscaling model first, then run it through AniSD PS. Alternatively, if you downscale a source before upscaling (commonly done on very low-quality sources), you can then use AniSD PS to upscale by another 2x. While haphazardly chaining upscaling models can often produce undesireable or unoptimal results, AniSD PS is trained specifically to handle AniSD output, and as such, has near perfect preservation of the fine detail and original look of the input.
  • AniSD DeBleed (AniSD DB): The only model in the AniSD suite that is 1x. This model is trained to deal with chroma bleed, and will help clean up a source before you run one of the upscaling models. The model isn't as potent as Zarxrax's Bleedout model, but also won't cause the drastic color shifting which Bleedout causes.

🏠 Architecture Breakdown

A variety of SISR architectures are represented in the AniSD suite. Below is an overview of the options.

  • AniSD: The lightweight options are Compact and SPAN. The heavyweight options are RealPLKSR and SwinIR. 
  • AniSD AC: The lightweight options are Compact and SPAN. The heavyweight options are RealPLKSR, SwinIR and CRAFT.
  • AniSD DC: The lightweight option is SPAN. The heavyweight options are RealPLKSR, DAT2 and CRAFT. However, this category of models is really intended to be used with the DAT2 model.
  • AniSD PS: Only Compact is available.
  • AniSD DB: Only SPAN is available.

Speeds

The below summarizes the speeds of each of the archs used in AniSD. As you can see, Compact and SPAN easily lead the pack. Given that both are very capable lightweight archs, they should be the default pick in most scenarios. RealPLKSR, which is not included in the table below, has roughly the same inference speed as SwinIR TensorRT, which means it's actually quite quick for a heavyweight architecture. On my system, DirectML inference for RealPLKSR was actually faster than SwinIR TensorRT, though results will likely vary between systems. Given RealPLKSR's superb DML inference speed, I would consider CRAFT (whose primary niche before was Pytorch inference speed) to be less useful now. Lastly, DAT2 is a real snail, really only viable for video when used with TensorRT.

For more information on how to run AniSD models with TensorRT, refer to this wiki page.

Pytorch (4090, 640x480 input)

  • Compact: 10.30 FPS
  • SPAN: 6.92 FPS
  • CRAFT: 2.83 FPS
  • SwinIR: 2.45 FPS
  • DAT2: .47 FPS

TensorRT (4090, 640x480 input)

  • SPAN: 209.3 FPS
  • Compact: 193.56 FPS
  • SwinIR Small (1x3x480x320, FP16, FP32 ONNX): 12.53 FPS
  • DAT2 (1x3x480x320, BF16, FP32 ONNX): 1.24 FPS
  • CRAFT: N/A.

📝 Further Notes / Model Selection

Given the number of models involved, this section is intended to provide further detail on the role of each model.

  • AniSD: SPAN is the overall recommended pick, with Compact as a fallback if for whatever reason the source doesn't work well with SPAN. SPAN has demonstrated better generalization capabilities than Compact (meaning it will do better across a wider range of sources), and is also slightly faster than Compact at TRT inference. This does come at a very slight cost to color accuracy, but it's hardly noticeable. As AniSD is primarily directed at "cleaner" SD sources, you probably don't need to try one of the heavier archs. But if you want the best quality, there is an AniSD RealPLKSR option. There is also a SwinIR option as well.
  • AniSD ACSPAN is the recommended lightweight pick, for the same reasons it is the recommended pick for AniSD. SPAN's greater generalization capabilities are even more valuable here, given that AniSD AC is intended to work across a broader range of more heavily-degraded sources than AniSD. Compact still remains an option as well. Should the lightweight options provide inadequate, RealPLKSR is the overall recommended heavyweight pick due to excellent all-around performance and speed, though SwinIR remains capable. RealPLKSR and SwinIR both benefit from surprisingly quick TRT inference speeds, and RealPLKSR is even faster with DML on my system. Note that the RealPLKSR model is a bit worse at color accuracy than the SwinIR model, and will attempt to color correct SD colorspace into HD colorspace. It probably won't be an issue, but it's something worth considering. But if you're planning to just use Pytorch inference, you can consider CRAFT as well.
  • AniSD DC: If a source is too gnarly for AniSD or AniAC, give the RealPLKSR version of AniSD DC a try. It is surprisingly capable while retaining RealPLKSR's excellent DML inference speed. However, if that is inadequate, the last hope is the DAT2 version. The DAT2 version is extremely capable, and in 99% of scenarios will produce better results than anything else here. But this comes with several downsides. First, the color accuracy isn't very good. This is an issue innate with DAT2, which I haven't been able to resolve. Second, it is extremely VRAM hungry, which is to be expected for an arch of its size. Lastly, it is extremely slow with Pytorch inference, and still quite slow with TRT inference. You'll need to be patient! There's also a SPAN version of AniSD DC, which may be worth trying out-- but it'll be significantly worse than the heavyweight options in terms of inference quality (though also infinitely faster at inference speed as well).
    • Tip: You can also try downscaling a source before upscaling. This can often help deal with extremely bad sources. But this can come at the expense of detail retention, especially if you're doing significant downscaling first (such as 50%). That being said, DAT2 and CRAFT are very capable at retaining fine detail. Test and find a balance that works best.
  • AniSD PS: Does what it says on the tin. Simple as. Use it to reach target resolutions without impacting the source's look.
  • AniSD DB: On sources with significant color bleed, use it before running an upscaling model. The model isn't as potent as Zarxrax's Bleedout, but also won't color shift like Bleedout. The model will also struggle with rainbowing, which has proven to be difficult to train the model for.
⚠️ **GitHub.com Fallback** ⚠️