Stable Diffusion

Stable Diffusion is an open-source diffusion model for generating images from textual descriptions. Note: as of writing there is rapid development both on the software and user side. Take everything you read here with a grain of salt.

How to Use

• beta.dreamstudio.ai: official web service
• Official Github page
• basujindal fork: fork that uses less VRAM at the cost of speed
• waifu-diffusion fork: fork that ???

Prompt Design

Guidelines for creating better prompts.

Prompt Length

Be descriptive. The model does better if you give it longer, more detailed descriptions of what you want. Use redundant descriptions for parts of the prompt that you care about.

Note however, that there is a hard limit regarding the length of prompts. Everything after a certain point - 75 or 76 CLIP tokens depending on how you count - is simply cut off. As a consequence it is preferable to use keywords that describe what you want concisely and to avoid keywords that are unrelated to the image you want.

Punctuation

Use it. Separating keywords by commas, periods, or even null characters ("\0") improves image quality. It's not yet clear which type of punctuation or which combination works best.

Keywords

Known good keywords (determined by keywords as prompts without other keywords).

• Anime-style images: avoid "anime", "manga", and "waifu". Instead try "アニメ" (Japanese way to write anime), "chibi", "Kyoto Animation", "light novel illustration", "Pixiv fanbox", "shonen", "Studio Ghibli", "visual novel CG", or "visual novel cover art". Order of recommended keywords is simply alphabetical; style differs slightly.

Useful Links

• krea.ai: Website that lets you explore keywords
• clip-retrieval: Project that lets you determine the relationship between images and keywords, works in either direction. Online version here