Hierarchical Text-Conditional Image Generation with CLIP Latents

Abstract

Contrastive models like CLIP have been shown to learn robust representations of images that capture both semantics and style. We propose a two-stage model: a prior that generates a CLIP image embedding given a text caption, and a decoder that generates an image conditioned on the image embedding. We call the resulting model DALL-E 2.

Key Findings

• 1Combined CLIP embeddings with diffusion models for text-to-image generation
• 2Achieved photorealistic image generation from text descriptions
• 3Demonstrated image editing capabilities through text-guided manipulation
• 4Introduced a two-stage architecture (prior + decoder) for generation
• 5Showed strong zero-shot generation capabilities

Impact & Significance