5. Hiresfix Guidance: a few examples - minsky91/krita-ai-diffusion-hires GitHub Wiki

Hiresfix Guidance explained with examples

Hiresfix Guidance is a new feature included in the Hires edition of Krita AI that makes the plugin’s built-in Hiresfix method customizable (see the Hiresfix Guidance option here). This addition has 2 purposes:

(1) to allow for more flexible Hiresfix treatment of large images that are generated with 100% strength (that is, from scratch). By standard Hiresfix design in Krita AI, such images are generated first at the checkpoint’s native resolution, automatically resized back to the canvas dimensions and then refined at 40% strength, all in a single behind-the-scenes operation. This technique helps to make the generated features, mostly human form, look much more natural and detailed in the output, but unfortunately often adds distortions of its own. The new Hiresfix Guidance option makes the 40% factor variable, extending it with slider values within a 20% - 60% range, which allows to finetune the refinement and prevent distortions in the output in more cases than before;

(2) prevent various tiling artefacts from appearing in the output when Tiled Diffusion is used for hires refining.

Here are some collected examples of such distortions and artefacts.

The top part of the image below is an original rendering containing typical face distortions characteristic of an image generated at a native checkpoint resolution and upscaled without applying Hiresfix; the bottom part is the refined version that has Hiresfix applied. In this particular image, all the mangled features have been magically fixed. This, however, is not always the case.

The image below is beyond repair, of course. The ‘usual’ deformities such as the extraneous leg and malformed hands may have little to do with Hiresfix, but the bulges on the neck and the shoulder are most likely its side effect.

The case (2) issue with Hiresfix (when Tiled Diffusion is enabled) is demonstrated with the image below:

This kind of effect, with multiple faint, miniature versions of the character in the prompt are caused by the tiled character of img2img processing when the denoise value is too high (which happens when Hiresfix is being applied). The remedy is to change Hiresfix Guidance value from zero (corresponding to the 40% factor) to a non-zero positive one, and see which value produces the best result. If you like the character’s rendering and want to keep it somehow, you can re-render the image with the same seed but higher Hiresfix Guidance value; this will make the miniature copies go away, but likely by the price of less refined face features. Making a blend of two images with the best bits kept on their Krita layers is the recommended solution here; it’s also usually the fastest. Below is the result achieved using this blending technique:

Note that Hiresfix can successfully enhance not only face features but other objects as well, like pointe shoes below:

Here’s another example of successful Hiresfix application (supposedly featuring the Monty Python bunch having a laugh in between takes of an unknown movie):

A collection of full-size Hiresfix Guidance example images, including their metadata, is found in the folder here.

Fixing Hiresfix for Flux

The Hiresfix feature of Krita AI Hires includes a fix for an issue frequently reported when generating images of a higher than native resolution with Flux models, the infamous screen grid pattern. The pattern is eliminated when Hiresfix is applied with Tiled Diffusion activated, and so is the blurriness that is typically introduced when rendering such images. The images below demonstrate the effect (top row images are generated with standard Krita AI, bottom ones with Krita AI Hires using the same prompt and parameters):

To clarify: this time it’s not the Hiresfix Guidance option that was used to fix the artefacts. Rather, it’s the Flux native image resolution range, as defined in the Hires version, that caused Hiresfix to be applied, fixing thereby the artefacts. In contrast, the standard version, because of its built-in definition of Flux native resolution range, will not apply Hiresfix to images of 1.5K size and above (which I believe it’s a glitch to fix), and so the artefacts ensue.

What’s more, both generating and img2img-processing large images is also much faster with the Hires edition: for the 2x2.4K images above, the processing times were 2.9 times faster for Hires (80s against 230s) (*).

The full set of png images demonstrating the Flux artefact issue and the renderings fixed by the Hires version, along with their metadata and generated workflows, can be found in this folder.

(*) The examples were produced on a Windows 11 PC equipped with 2.50 GHz Intel Core 9 185H chip and 32 GB RAM, using SSD storage. The GPU was an NVIDIA GeForce RTX 4070 Ti SUPER with 16 GB VRAM, operated by the Comfy server v0.3.19. No TeaCache or any other accelerator extension has been used.