Google AI Research Releases Agile Deliberation Framework for Gray-Area Image Classifiers

Google Research has released Agile Deliberation, a human-in-the-loop framework for building image classifiers around subjective concepts—such as "healthy food," "inspiring artwork," or clickbait in content moderation. Most existing tooling assumes users already know the concept boundary. Agile Deliberation assumes they do not, and helps them discover it.

TL;DR

• Framework: Prompts Gemini 2.5 Pro and Gemini 2.5 Flash through the API, without fine-tuning.
• Runs on: A Jupyter or Colab notebook. Retrieval uses CLIP embeddings and a FAISS index on CPU or GPU. The demo index starts at about 260 MB.
• Performance: An average of 10.5 F1 points above zero-shot Gemini 2.5 Flash across 2 concepts and 18 live sessions.
• Best case: "Paid to play" reaches an F1 of 0.59 vs. 0.48 zero-shot, with precision up 22 points (0.35 to 0.57).
• Worst case: Recall drops 17 points on "paid to play" (0.80 to 0.63) and 15 points on "healthy food."

What Is Agile Deliberation?

Agile Deliberation is a Google Research framework that turns a vague concept into a structured written definition. That definition serves two purposes: it acts as a human-readable policy, and it becomes the text prompt for a VLM classifier. Each round preserves the candidate definition...

via MarkTechPost