Tips & Tricks ‐ Learning Progression - InfluxOW/Stable-Diffusion-Text-To-Person GitHub Wiki
It's pretty exciting to see how your model learns and gradually transforms base image. And we can make a grid out of it. And it's quite easy to do.
For example, you have a character LoRA in your prompt <lora:my_character:1.0>. To make a first grid, we'll need simple X/Y/Z Plot. Change your LoRA weight from 1.0 to 0.0 for ascending progression. Then use Prompt S/R type with increasing weights.
You'll get something like this.
It's still good but it could be better if scene wouldn't change that much. To achieve this we'll require ControlNet. Install the extension and download one of three or all the three ControlNet models: OpenPose, Canny and Depth. Place them in stable-diffusion-webui\extensions\sd-webui-controlnet\models.
Now you just have to drag'n'drop the image you want to use as a reference and enable ControlNet.
OpenPose will apply the reference character pose to the final image.
Depth will apply the reference image depth map to the final image.
Canny will apply the reference contour to the final image. You may require to change thresholds.
You can also make such a progression from an already created image but it will require more manual work. If you take any generated image and use it as a reference in any ControlNet model, it will be slightly changed. This is the main problem here.
For example, this is my reference image.
It was generated with LoRA weight 0.9. To make a progression grid and end up with the same image as the reference one, we have to manually set LoRA weight and ControlNet model weight for every step. For example, we start with 0.0 LoRA weight and 1.0 ControlNet model weight. In the next step we have to increase LoRA weight by 10% and decrease ControlNet model weight by 10%. In my case it's going to be 0.09 LoRA weight and 0.9 ControlNet model weight. Depending on ControlNet model you're using, you could get different intermediate images, but, eventually, you'll receive the reference image. In the end, you just need to manually merge them on a single grid.
Next - Tips & Tricks ‐ Stylization
