OOM

To get the most out of your training a card with at least 12GB of VRAM is reccomended. Supported currently are only 10GB and higher VRAM GPUs

Low VRAM

Settings known to use more VRAM

• High Batch Size
• Set Gradients to None When Zeroing
• Use EMA
• Full Precision
• Default Memory attention
• Cache Latents
• Text Encoder

Settings that lowers VRAM

• Low Batch Size
• Gradient Checkpointing
• fp16/bf16 precision
• xformers/flash_attention
• Step Ratio of Text Encoder Training 0 (no text encoder)

Overtraining

wip