Reproducing Zero-Shot Visual Anomaly Detection

https://amazon-visual-anomaly.s3.us-west-2.amazonaws.com/VisA_20220922.tar

VisA (Zou et al., Amazon, "SPot-the-Difference"), 12 real object classes, 10,821 images (9,621 normal, 1,200 real labelled anomalies). License: CC BY 4.0 -- verified by downloading Amazon's own amazon-science/spot-diff repository and reading its LICENSE-DATASET file directly, not by trusting a mirror. Two other industrial-defect datasets we considered first -- MVTec AD and a popular Kaggle casting-defect set -- turned out to be CC BY-NC-SA and CC BY-NC-ND respectively once checked at their real source (a third-party mirror had mislabelled the Kaggle one as CC BY 4.0). Worth checking any dataset's actual license page yourself before using a result commercially, not a re-host's claim about it.

import csv, numpy as np
from PIL import Image
from vaasx.ac.encoders.vision import VisionEncoder

enc = VisionEncoder(device="cpu")

def load_split(csv_path, category):
    train, test = [], []
    with open(csv_path, newline="") as f:
        for row in csv.DictReader(f):
            if row["object"] != category:
                continue
            (train if row["split"] == "train" else test).append(row)
    return train, test

def embed(root, row):
    img = Image.open(f"{root}/{row['image']}").convert("RGB")
    return enc.encode(img)

train_rows, test_rows = load_split("split_csv/1cls.csv", "pcb1")
ref = np.stack([embed("VisA", r) for r in train_rows])          # normal-only
ref = ref / np.linalg.norm(ref, axis=1, keepdims=True)

test_emb = np.stack([embed("VisA", r) for r in test_rows])
test_emb = test_emb / np.linalg.norm(test_emb, axis=1, keepdims=True)

anomaly_score = 1.0 - (test_emb @ ref.T).max(axis=1)             # nearest-neighbor distance

Zero-shot: no training happens anywhere in this script. The reference set is frozen ViT-B/16 embeddings of normal images only; every test image is scored by how far it sits from its nearest normal neighbor.

from sklearn.metrics import roc_auc_score

y_true = [1 if r["label"] == "anomaly" else 0 for r in test_rows]
print("AUROC:", roc_auc_score(y_true, anomaly_score))

Run this across pcb1, macaroni1, and capsules and you should land close to: pcb1 0.909, macaroni1 0.823, capsules 0.752 -- mean 0.828. All three well above the 0.5 chance floor, with a believable spread: pcb1's defects (missing/misplaced components) are visually gross and separate most strongly; capsules' defects (print/colour variation) are subtler and separate least, though still clearly above chance.

from vaasx import Bootstrap
from vaasx.ac import ArtificialCognition

brain = Bootstrap(api_key="YOUR_KEY", device_id="inspection_line_3")
ac = ArtificialCognition(brain)

fused = ac.encode(vision=normal_image)
ac.ingest(fused, metadata={"label": "normal"})

query_fused = ac.encode(vision=new_image)
hits = ac.retrieve(query_fused, k=5)

The encoder runs entirely on your own machine -- production only stores and indexes the vector it receives via /ac/ingest, gated on the ac capability (Professional tier and above). We ran exactly this against live production: ingested one normal and one anomalous embedding, queried with a held-out normal image, and got back the correct ranking -- 0.957 similarity to the other normal image, 0.838 to the anomalous one.

# a local MCP tool, not a library call -- see vision_mcp/README.md
vision_watch_start(rtsp_url="rtsp://user:pass@camera-ip/stream",
                    name="line_3", mode="baseline", max_ingest=50)
vision_watch_status(name="line_3")   # poll while it ingests known-normal frames
vision_watch_start(name="line_3", mode="check", threshold=0.4)  # then watch for anomalies

A separate local MCP server adds RTSP ingestion on top of the same encoder: a background watcher samples a live camera at a configurable interval, decodes and encodes every frame locally, and sends only the embedding onward -- never a pixel. We verified this against a real residential camera, not a simulated feed: vision_watch_start connected over RTSP/TCP, sampled 5 frames, encoded and ingested 3 of them against live production (three real POST /ac/ingest calls, each 200 OK), then auto-stopped cleanly at the configured limit -- zero reconnects, zero errors, and the camera's credentials never appeared in any tool output.

What's not yet verified against real hardware: live anomaly alerting when a real defect appears on a real feed (exercised so far only against a synthetic test camera), multiple cameras running at once, and how the reconnect logic behaves under an actual camera dropping out mid-stream.

What this reproduces: a frozen, untrained image embedding separating real defects from normal parts on a real, commercially-licensed benchmark across three industrial domains, with the full client-encode/server-ingest/server-query path verified against live production. What it does not show: results on the other 9 VisA categories, whether custom training would do better than zero-shot, performance on any specific customer's actual product images, or audio/depth/thermal modalities -- those remain untouched and unverified. Treat the embedding, the fusion layer, and the production round trip as what's proven here.

More guides: Reproduction Guides. Full reference: Developer Guide. Related: OIE IMU Embedding Encoder guide (the same Artificial Cognition substrate, a different modality).