Posted on 2023/06/14 by GoAPI
LoRA Models: A Beginner's Guide
In this tutorial, we will explore LoRA (Low-Rank Adaptation) models for fine-tuning Stable Diffusion models. LoRA models offer a balance between file size and training power, making them a popular choice for users with extensive model collections. LoRA models are typically 10 to 100 times smaller than standard checkpoint models. This makes them attractive for users who face storage limitations on their personal computers. With manageable file sizes ranging from 2 to 200 MBs, LoRA models offer an excellent solution to the storage problem. Unlike textual inversion or Dreambooth, LoRA models cannot be used alone. They require a model checkpoint file to function. LoRA models modify styles by applying small changes to the accompanying model file, specifically targeting the cross-attention layers. The key advantage of LoRA models lies in their ability to break down matrices into low-rank matrices. By doing this, LoRA models can store significantly fewer numbers, resulting in smaller file sizes. However, this reduction in file size does not significantly impact the fine-tuning power of the models. To find LoRA models, you can visit Civitai, a platform that hosts a large collection of LoRA models. By applying the LoRA filter, you can browse through various styles such as female portraits, anime, and realistic illustrations. It is worth mentioning that Civitai may contain NSFW (Not Safe For Work) content, so make sure to apply the appropriate filter if necessary. Alternatively, Hugging Face also offers a smaller collection of LoRA models. To use LoRA models in the AUTOMATIC1111 Stable Diffusion GUI, follow these steps:
Install the LoRA model: Place the model files in the folder stable-diffusion-webui/models/Lora.
Use the LoRA model in prompts: Include the phrase "<lora:filename:multiplier>" in your prompt, where "filename" is the file name of the LoRA model (without the extension) and "multiplier" is the weight applied to the LoRA model. The default multiplier is 1, and setting it to 0 disables the model.
To ensure the correct filename is used, you can click on the model button within the GUI. Under the Lora tab, you will find a list of installed LoRA models to choose from. Clicking on a specific model will automatically insert the correct LoRA phrase into the prompt. Some additional notes on using LoRA models:
Adjust the multiplier to control the effect of the LoRA model. Setting it to 0 disables the model.
For LoRA models trained with Dreambooth, a trigger keyword may be required. You can find this keyword on the model's page.
Multiple LoRA models can be used simultaneously, and they can also be combined with embeddings.
In AUTOMATIC1111, the LoRA phrase is not considered part of the prompt and is removed after the model is applied. Therefore, you cannot use prompt syntax like [keyword1:keyword2: 0.8] with LoRA models.
By following these guidelines, you can access and utilize LoRA models in AUTOMATIC1111, enhancing your ability to fine-tune and experiment with different styles for image generation.