mirror of
https://github.com/huggingface/diffusers.git
synced 2026-01-27 17:22:53 +03:00
bfe37f3159
* Accept latents as input for StableDiffusionPipeline. * Notebook to demonstrate reusable seeds (latents). * More accurate type annotation Co-authored-by: Suraj Patil <surajp815@gmail.com> * Review comments: move to device, raise instead of assert. * Actually commit the test notebook. I had mistakenly pushed an empty file instead. * Adapt notebook to Colab. * Update examples readme. * Move notebook to personal repo. Co-authored-by: Suraj Patil <surajp815@gmail.com>
Inference Examples
Installing the dependencies
Before running the scipts, make sure to install the library's dependencies:
pip install diffusers transformers ftfy
Image-to-Image text-guided generation with Stable Diffusion
The image_to_image.py script implements StableDiffusionImg2ImgPipeline. It lets you pass a text prompt and an initial image to condition the generation of new images. This example also showcases how you can write custom diffusion pipelines using diffusers!
How to use it
from torch import autocast
import requests
from PIL import Image
from io import BytesIO
from image_to_image import StableDiffusionImg2ImgPipeline, preprocess
# load the pipeline
device = "cuda"
pipe = StableDiffusionImg2ImgPipeline.from_pretrained(
"CompVis/stable-diffusion-v1-4",
revision="fp16",
torch_dtype=torch.float16,
use_auth_token=True
).to(device)
# let's download an initial image
url = "https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg"
response = requests.get(url)
init_image = Image.open(BytesIO(response.content)).convert("RGB")
init_image = init_image.resize((768, 512))
init_image = preprocess(init_image)
prompt = "A fantasy landscape, trending on artstation"
with autocast("cuda"):
images = pipe(prompt=prompt, init_image=init_image, strength=0.75, guidance_scale=7.5)["sample"]
images[0].save("fantasy_landscape.png")
You can also run this example on colab 
Tweak prompts reusing seeds and latents
You can generate your own latents to reproduce results, or tweak your prompt on a specific result you liked. This notebook shows how to do it step by step. You can also run it in Google Colab .