mirror of
https://github.com/huggingface/diffusers.git
synced 2026-01-27 17:22:53 +03:00
c53a850604
* [Batched Generators] all batched generators * up * up * up * up * up * up * up * up * up * up * up * up * up * up * up * up * hey * up again * fix tests * Apply suggestions from code review Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * correct tests Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
74 lines
2.8 KiB
Plaintext
74 lines
2.8 KiB
Plaintext
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
specific language governing permissions and limitations under the License.
|
|
-->
|
|
|
|
# Re-using seeds for fast prompt engineering
|
|
|
|
A common use case when generating images is to generate a batch of images, select one image and improve it with a better, more detailed prompt in a second run.
|
|
To do this, one needs to make each generated image of the batch deterministic.
|
|
Images are generated by denoising gaussian random noise which can be instantiated by passing a [torch generator](https://pytorch.org/docs/stable/generated/torch.Generator.html#generator).
|
|
|
|
Now, for batched generation, we need to make sure that every single generated image in the batch is tied exactly to one seed. In 🧨 Diffusers, this can be achieved by not passing one `generator`, but a list
|
|
of `generators` to the pipeline.
|
|
|
|
Let's go through an example using [`runwayml/stable-diffusion-v1-5`](runwayml/stable-diffusion-v1-5).
|
|
We want to generate several versions of the prompt:
|
|
|
|
```py
|
|
prompt = "Labrador in the style of Vermeer"
|
|
```
|
|
|
|
Let's load the pipeline
|
|
|
|
```python
|
|
>>> from diffusers import DiffusionPipeline
|
|
|
|
>>> pipe = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16)
|
|
>>> pipe = pipe.to("cuda")
|
|
```
|
|
|
|
Now, let's define 4 different generators, since we would like to reproduce a certain image. We'll use seeds `0` to `3` to create our generators.
|
|
|
|
```python
|
|
>>> import torch
|
|
|
|
>>> generator = [torch.Generator(device="cuda").manual_seed(i) for i in range(4)]
|
|
```
|
|
|
|
Let's generate 4 images:
|
|
|
|
```python
|
|
>>> images = pipe(prompt, generator=generator, num_images_per_prompt=4).images
|
|
>>> images
|
|
```
|
|
|
|
|
|
|
|
Ok, the last images has some double eyes, but the first image looks good!
|
|
Let's try to make the prompt a bit better **while keeping the first seed**
|
|
so that the images are similar to the first image.
|
|
|
|
```python
|
|
prompt = [prompt + t for t in [", highly realistic", ", artsy", ", trending", ", colorful"]]
|
|
generator = [torch.Generator(device="cuda").manual_seed(0) for i in range(4)]
|
|
```
|
|
|
|
We create 4 generators with seed `0`, which is the first seed we used before.
|
|
|
|
Let's run the pipeline again.
|
|
|
|
```python
|
|
>>> images = pipe(prompt, generator=generator).images
|
|
>>> images
|
|
```
|
|
|
|
|