Zero-Shot Image Classifier — Free, No API Key

Zero-shot image classification asks a different question from an ordinary classifier. Instead of predicting from a fixed list of classes the model was trained on, you bring your own labels and the model scores how well the image matches each. This tool uses CLIP (Radford et al., 2021), which was trained on 400 million image–caption pairs to place matching images and text close together in one shared embedding space. Because that space is shared, a brand-new label written as text can be compared to an image directly — no per-label training required.

The pipeline is deterministic given the model weights (CLIP paper §2.4). Every step below runs server-side through the Hugging Face Inference API on openai/clip-vit-base-patch32 (CLIP ViT-B/32 · ~151M parameters · 224×224 input):

1. Encode the image.The image passes through CLIP's vision encoder to produce one embedding vector v.
2. Encode each label. Each label is inserted into the prompt template (default a photo of a {}) and run through the text encoder to produce a vector tₓ.
3. Normalise and compare. Both vectors are L2-normalised to unit length, then the cosine similarity sₓ = v · tₓ is the dot product — how aligned the image is with that label.
4. Scale by the logit scale. CLIP multiplies each similarity by a learned temperature that released checkpoints train up to about 100, giving logits zₓ = 100· sₓ.
5. Convert to probabilities. A softmax across your labels turns the logits into confidences that sum to 100%: pₓ = e^zₓ / Σ e^zʲ. The highest is the predicted label.

The large logit scale is why CLIP looks so confident: a modest similarity gap of 0.31 versus 0.22 becomes a logit gap of 31 versus 22, which softmax pushes to roughly 100% versus 0%. There is a second, equivalent way to read the scores — an independent sigmoid per label, σ(zₓ) = 1 / (1 + e^−zₓ), which asks “does this label fit, yes or no?” without comparing labels to each other. For exactly two labels the two views agree exactly, because single-label softmax equals the sigmoid of the logit difference — the cross-check this tool's math is verified against. The hosted CLIP endpoint returns the softmax (compare-labels) scores; the sigmoid formula is shown here for anyone reading raw logits from a self-hosted model.

• Radford et al. (2021) — Learning Transferable Visual Models From Natural Language Supervision (CLIP), arXiv:2103.00020
• Hugging Face model card — openai/clip-vit-base-patch32 (weights + prompt-template convention)
• Hugging Face — Zero-Shot Image Classification task documentation

The methodology, logit scale, and worked-example numbers on this page were cross-checked against the CLIP paper and the model card on 2026-07-12. Uploads are capped at 5.0 MB and up to 20 labels; inference runs server-side and the image is discarded after scoring.