HuggingFace

• based on stabilityai/stable-diffusion-2-1#examples on HF
• $ pipenv install diffusers transformers accelerate scipy safetensors
• in PoC code
  • change torch_dtype to torch.float32 (ref)
  • change pipe.to parameter to cpu

import torch
from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler

model_id = "stabilityai/stable-diffusion-2-1"

# Use the DPMSolverMultistepScheduler (DPM-Solver++) scheduler here instead
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float32)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
pipe = pipe.to("cpu")

prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt).images[0]
    
image.save("astronaut_rides_horse.png")

Stability-AI GitHub

• clone Stability-AI/stablediffusion
• download checkpoints (.ckpt and .safetensors files) from HF and put into stablediffusion/checkpoints
• `$ pipenv install torch transformers diffusers invisible-watermark omegaconf einops torchvision lightning
• $ pipenv install -e .
• in stablediffusion/ldm/models/diffusion/ddpm.py comment out import and use of rank_zero_only
• in configs/stable-diffusion/v2-inference-v.yaml set use_fp16 to False (mby relevant)
• $ python3 scripts/txt2img.py --prompt "a professional photograph of an astronaut riding a horse" --ckpt checkpoints/v2-1_768-ema-pruned.ckpt --config configs/stable-diffusion/v2-inference-v.yaml --H 768 --W 768 --precision full --steps 50 --n_iter 1 --n_samples 1 --device cpu

AUTOMATIC1111 with web UI

• based on AUTOMATIC1111/stable-diffusion-webui
• # apt install wget git python3 python3-venv
• $ bash <(wget -qO- https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh) (CAUTION — immediately runs script)
• $ ./webui.sh --skip-torch-cuda-test --no-half
• then open http://127.0.0.1:7860
• extensions
  • ControlNet
    • img2img
      • openpose + canny
    • also use script “img2img alternative test”

Stable Diffusion 1.0 XL

• stabilityai/stable-diffusion-xl-base-1.0
• stabilityai/stable-diffusion-xl-refiner-1.0
• Misc
  • 500 rabbits

Styles (put in styles.csv of webui
(larger collection)

name,prompt,negative_prompt
"Style: Enhance","breathtaking {prompt} . award-winning, professional, highly detailed","ugly, deformed, noisy, blurry, distorted, grainy"
"Style: Anime","anime artwork {prompt} . anime style, key visual, vibrant, studio anime, highly detailed","photo, deformed, black and white, realism, disfigured, low contrast"
"Style: Photographic","cinematic photo {prompt} . 35mm photograph, film, bokeh, professional, 4k, highly detailed","drawing, painting, crayon, sketch, graphite, impressionist, noisy, blurry, soft, deformed, ugly"
"Style: Digital art","concept art {prompt} . digital artwork, illustrative, painterly, matte painting, highly detailed","photo, photorealistic, realism, ugly"
"Style: Comic book","comic {prompt} . graphic illustration, comic art, graphic novel art, vibrant, highly detailed","photograph, deformed, glitch, noisy, realistic, stock photo"
"Style: Fantasy art","ethereal fantasy concept art of {prompt} . magnificent, celestial, ethereal, painterly, epic, majestic, magical, fantasy art, cover art, dreamy","photographic, realistic, realism, 35mm film, dslr, cropped, frame, text, deformed, glitch, noise, noisy, off-center, deformed, cross-eyed, closed eyes, bad anatomy, ugly, disfigured, sloppy, duplicate, mutated, black and white"
"Style: Analog film","analog film photo {prompt} . faded film, desaturated, 35mm photo, grainy, vignette, vintage, Kodachrome, Lomography, stained, highly detailed, found footage","painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured"
"Style: Neonpunk","neonpunk style {prompt} . cyberpunk, vaporwave, neon, vibes, vibrant, stunningly beautiful, crisp, detailed, sleek, ultramodern, magenta highlights, dark purple shadows, high contrast, cinematic, ultra detailed, intricate, professional","painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured"
"Style: Isometric","isometric style {prompt} . vibrant, beautiful, crisp, detailed, ultra detailed, intricate","deformed, mutated, ugly, disfigured, blur, blurry, noise, noisy, realistic, photographic"
"Style: Lowpoly","low-poly style {prompt} . low-poly game art, polygon mesh, jagged, blocky, wireframe edges, centered composition","noisy, sloppy, messy, grainy, highly detailed, ultra textured, photo"
"Style: Origami","origami style {prompt} . paper art, pleated paper, folded, origami art, pleats, cut and fold, centered composition","noisy, sloppy, messy, grainy, highly detailed, ultra textured, photo"
"Style: Line art","line art drawing {prompt} . professional, sleek, modern, minimalist, graphic, line art, vector graphics","anime, photorealistic, 35mm film, deformed, glitch, blurry, noisy, off-center, deformed, cross-eyed, closed eyes, bad anatomy, ugly, disfigured, mutated, realism, realistic, impressionism, expressionism, oil, acrylic"
"Style: Craft clay","play-doh style {prompt} . sculpture, clay art, centered composition, Claymation","sloppy, messy, grainy, highly detailed, ultra textured, photo"
"Style: Cinematic","cinematic film still {prompt} . shallow depth of field, vignette, highly detailed, high budget Hollywood movie, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy","anime, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured"
"Style: 3d-model","professional 3d model {prompt} . octane render, highly detailed, volumetric, dramatic lighting","ugly, deformed, noisy, low poly, blurry, painting"
"Style: pixel art","pixel-art {prompt} . low-res, blocky, pixel art style, 8-bit graphics","sloppy, messy, blurry, noisy, highly detailed, ultra textured, photo, realistic"
"Style: Texture","texture {prompt} top down close-up","ugly, deformed, noisy, blurry"