CLIP Interrogator - AI Image Tools Tool

Overview

CLIP Interrogator is an open-source prompt-engineering tool (by pharmapsychotic) that analyzes an input image and generates optimized text prompts you can feed into text-to-image models such as Stable Diffusion. It combines Salesforce’s BLIP captioning to produce a base caption and OpenCLIP to match the image against curated lists of artists, styles, mediums and other “flavors,” then composes an enriched prompt for use in generative pipelines. ([github.com](https://github.com/pharmapsychotic/clip-interrogator)) ([replicate.com](https://replicate.com/pharmapsychotic/clip-interrogator)) The project is distributed as a pip package, a Colab notebook, a Stable Diffusion Web UI extension, and a hosted Replicate/Hugging Face implementation — making it usable both locally and via API. It supports multiple OpenCLIP backbones (recommended: ViT-L-14/openai for SD 1.x and ViT-H-14/laion2b_s32b_b79k for SD 2.x), has configurable VRAM modes (default ~6.3GB, low-VRAM ~2.7GB), and can be extended to rank a custom list of labels. The repository is actively used by the community (GitHub stars, Replicate run counts) and appears in popular SD GUIs and forums. ([github.com](https://github.com/pharmapsychotic/clip-interrogator))

Key Features

• Combines BLIP captions with OpenCLIP matching to produce enriched prompts.
• Supports multiple CLIP backbones; recommended presets for SD 1.x and 2.x.
• Low-VRAM mode reduces memory from ~6.3GB to ~2.7GB for smaller GPUs.
• Available as pip package, Colab notebook, Replicate/Hugging Face and SD Web UI extension.
• Can rank against custom term lists (LabelTable) to match domain-specific vocabularies.

Example Usage

Example (python):

from PIL import Image
from clip_interrogator import Config, Interrogator

# Load image
image = Image.open('example.jpg').convert('RGB')

# Configure (use recommended CLIP for SD 1.x)
config = Config(clip_model_name='ViT-L-14/openai')
ci = Interrogator(config)

# Generate an image-to-text prompt
prompt = ci.interrogate(image)
print('Interrogated prompt:')
print(prompt)

# Notes:
# - Install with: pip install clip-interrogator==0.5.4 (or 0.6.0 WIP for BLIP2 support)
# - For large models ensure torch + CUDA are installed and sufficient GPU VRAM is available.

Pricing

Open-source (MIT) — free to run locally. Hosted runs on Replicate are billed per prediction (Replicate lists an approximate cost of ~$0.024 per run, varying by deployment).

Benchmarks

Replicate runs: 4.9M runs (public runs on Replicate) (Source: https://replicate.com/pharmapsychotic/clip-interrogator)

Approximate cost per run (Replicate): $0.024 per run (approximate; varies by deployment) (Source: https://replicate.com/pharmapsychotic/clip-interrogator)

GitHub popularity: ≈2.9k stars, 440 forks (Source: https://github.com/pharmapsychotic/clip-interrogator)

Default VRAM usage: ≈6.3 GB (default), ≈2.7 GB (low-VRAM mode) (Source: https://github.com/pharmapsychotic/clip-interrogator)

Latest release: v0.6.0 (release date listed: Mar 20, 2023) (Source: https://github.com/pharmapsychotic/clip-interrogator)

Key Information

• Category: Image Tools
• Type: AI Image Tools Tool