ProCreate, Don't Reproduce! Propulsive Energy Diffusion for Creative Generation

• Main Authors: Lu, Jack, Teehan, Ryan, Ren, Mengye
• Format: Journal Article
• Language: English
• Published: 05-08-2024
• Subjects: Computer Science - Computer Vision and Pattern Recognition
• DOI: 10.48550/arxiv.2408.02226

In this paper, we propose ProCreate, a simple and easy-to-implement method to
improve sample diversity and creativity of diffusion-based image generative
models and to prevent training data reproduction. ProCreate operates on a set
of reference images and actively propels the generated image embedding away
from the reference embeddings during the generation process. We propose FSCG-8
(Few-Shot Creative Generation 8), a few-shot creative generation dataset on
eight different categories -- encompassing different concepts, styles, and
settings -- in which ProCreate achieves the highest sample diversity and
fidelity. Furthermore, we show that ProCreate is effective at preventing
replicating training data in a large-scale evaluation using training text
prompts. Code and FSCG-8 are available at
https://github.com/Agentic-Learning-AI-Lab/procreate-diffusion-public. The
project page is available at https://procreate-diffusion.github.io.