Running an LLM locally is increasingly appealing for projects involving sensitive data or workflows that must operate entirely on your own hardware. With local models, you maintain full control over privacy, customization, and cost—no cloud dependency required.
Importantly, local LLMs are no longer limited to text-only interactions. Modern multimodal models can process images, audio, and more, opening up a wide range of new use cases.
In this article, we'll build a practical multimodal workflow using Gemma 4 (via Ollama) to analyze a collection of photos. The key idea is to integrate the model's multimodal capabilities into a larger, deterministic process, where each stage produces structured output that downstream steps can consume.
To make this concrete, I'll use a personal example: a recent trip to Finland, where I captured many photos. I'll show how the workflow transforms those photos into organized memory records, and how the same pipeline can power a small application.
1. The Multimodal Workflow
The term "multimodal workflow" combines two key concepts:
Multimodal: The LLM accepts inputs beyond plain text—here, images. We'll use the Gemma 4 family from Google, which handles both image and text inputs effectively.
Workflow: Unlike an autonomous agent loop that decides its next action, the LLM operates within a predefined sequence. Each stage receives an input, processes it, and passes a structured output to the next stage. The LLM acts as a function performing semantic transformations, making the flow predictable and easy to debug.
For our case study, the goal is to turn a folder of trip photos into organized, searchable memory records. This requires analyzing each photo and extracting a consistent, structured description. Once we have per-photo records, we can synthesize them to understand the entire trip as a whole.
This naturally yields a three-stage pipeline:
photos = preparephotos("Finlandtrip")
photo_memories = [
analyze_photo(
image=photo.image,
metadata=photo.metadata,
)
for photo in photos
]
tripmemory = synthesizetrip(photo_memories)
- Prepare photos: Python preprocesses each image, extracts metadata (e.g., capture time, GPS coordinates), and scales if needed.
- Analyze each photo: The local Gemma 4 model examines each image and returns a structured record (e.g., JSON) describing its visual content.
- Synthesize the collection: All individual records are passed back to Gemma 4 to produce a cohesive summary of the entire trip.
Now, let's build each stage.
2. Building the Workflow with Gemma 4
2.1 Running Gemma 4 Locally
First, we need to set up a local runtime for Gemma 4. We'll use Ollama, a popular tool for running LLMs locally with a simple CLI and API.
- Windows/macOS: Download and run the installer from the Ollama website.
- Linux: Install via terminal:
curl -fsSL https://ollama.com/install.sh | shOnce Ollama is installed, pull the Gemma 4 model. We'll use the gemma4:e4b variant, which is optimized for edge devices (e.g., laptops, desktops) while retaining strong multimodal performance:
ollama pull gemma4:e4bNext, install the required Python packages:
pip install ollama pillow pydanticollama– Python client to connect to the local Ollama server.pillow– Image processing (opening, resizing, EXIF extraction).pydantic– Define and validate structured output schemas.
2.2 From Photo to Structured Record
For each photo, we want to generate a structured record that captures the visual content. But before feeding images to Gemma 4, we need to preprocess them.
We implement a prepare_photo() function that:
- Resizes the image to a maximum dimension (e.g., 1280px) to reduce memory usage and speed up inference.
- Extracts available EXIF metadata (capture time, GPS coordinates, camera model) using Pillow.
- Returns an object containing the processed image and metadata.
This deterministic preprocessing ensures the LLM receives consistent, optimized inputs.
For the analysis stage, we define a Pydantic model to structure the output. A typical photo memory record might include:
from pydantic import BaseModel, Fieldfrom typing import List, Optional
class PhotoMemory(BaseModel):
description: str = Field(..., description="A concise summary of the photo's content")
subjects: List[str] = Field(..., description="Main objects/people/scenes detected")
mood: Optional[str] = Field(None, description="Overall mood or atmosphere")
notable_details: Optional[List[str]] = Field(None, description="Interesting or unusual elements")
Then, the analyze_photo() function sends the image and metadata to Gemma 4 via the Ollama API, requesting a JSON response that matches the PhotoMemory schema. We can use Ollama's format parameter to enforce JSON output.
import ollama
def analyze_photo(image, metadata) -> PhotoMemory:
prompt = f"""
You are a travel memory assistant. Analyze the provided image.
Photo metadata: {metadata}
Return a JSON object with fields: description, subjects, mood, notable_details.
"""
response = ollama.chat(
model="gemma4:e4b",
messages=[{"role":"user","content":prompt}],
images=[image], # pass the image bytes
format="json"
)
return PhotoMemory.modelvalidatejson(response['message']['content'])
With this structured output, downstream processing becomes straightforward—no need to parse free-form text.
2.3 Generating a Trip Summary
Once we have all individual PhotoMemory records, we can synthesize them into a single trip-level summary. Again, we use Gemma 4, but now with text-only input (the collection of structured records).
class TripMemory(BaseModel):summary: str = Field(..., description="Overall narrative of the trip")
theme: str = Field(..., description="Dominant theme or highlight")
notable_moments: List[str] = Field(..., description="Key events/impressions")
def synthesizetrip(photomemories: List[PhotoMemory]) -> TripMemory:
inputtext = "\n".join([m.json() for m in photomemories])
prompt = f"""
You are a travel assistant. Based on the following structured records of photos from a trip, create a coherent summary.
Records:
{input_text}
Return a JSON object with fields: summary, theme, notable_moments.
"""
response = ollama.chat(
model="gemma4:e4b",
messages=[{"role":"user","content":prompt}],
format="json"
)
return TripMemory.modelvalidatejson(response['message']['content'])
This stage demonstrates how multiple structured outputs can be fed back into the model to generate higher-level insights.
3. Example: Analyzing Trip Photos
Let's see the workflow in action with real photos from my Finland trip.
3.1 Preparing the Photos
I started with a folder named Finlandtrip containing 20+ JPG files. The preparephotos() function iterates through the folder, loads each image, applies resizing, and extracts EXIF metadata.
Below is a sample output from the metadata extraction:
{"filename": "IMG20250615124502.jpg",
"datetime": "2025-06-15 12:45:02",
"gps": {"lat": 60.1699, "lon": 24.9384},
"camera": "Pixel 8 Pro"
}
3.2 Analyzing a Single Photo
Here's an example of a photo taken at Helsinki's Market Square. The analyze_photo() output was:
{"description": "A bustling market square with various outdoor stalls selling fresh produce and handicrafts, surrounded by old neoclassical buildings.",
"subjects": ["market stalls", "fruits and vegetables", "handicrafts", "historical architecture"],
"mood": "vibrant",
"notable_details": ["A red and white striped awning", "A seagull perched on a lamp post"]
}
The structured record captures both the obvious and subtle elements, making it easy to search or filter later.
3.3 Synthesizing the Trip Summary
After processing all photos, the synthesize_trip() function produced:
{"summary": "The trip was a mix of urban exploration in Helsinki and tranquil nature visits to lakes and forests. Highlights included the bustling Market Square, the serene Suomenlinna sea fortress, and an unexpected encounter with a moose.",
"theme": "Finnish nature and city life",
"notable_moments": ["Market Square visit", "Suomenlinna ferry ride", "Moose sighting at Nuuksio National Park"]
}
This kind of structured summary can be used for journaling, travel blogs, or as input to other applications.
4. Powering a Small Application
The same workflow can be turned into a simple application. For instance, I built a photo-to-memory web app using Flask (or Streamlit) that:
- Allows users to upload a folder of images (or a single image).
- Runs the same preprocessing and analysis stages in the background.
- Displays the generated structured records and the final summary.
Below is a snippet of the Flask route:
from flask import Flask, request, jsonifyimport os
app = Flask(name)
@app.route('/analyze', methods=['POST'])
def analyze():
files = request.files.getlist('images')
photos = [preparephotofrom_bytes(f.read()) for f in files]
memories = [analyze_photo(p.image, p.metadata) for p in photos]
trip = synthesize_trip(memories)
return jsonify({
'photo_memories': [m.dict() for m in memories],
'trip_summary': trip.dict()
})
Because each stage is a pure function returning structured data, scaling to a full application is straightforward—you could even parallelize photo analysis for faster throughput.
5. Conclusion
In this article, we demonstrated how to build a multimodal workflow using a local LLM (Gemma 4 via Ollama) to process a collection of photos. The key takeaways:
- Local LLMs are viable: With Ollama, running Gemma 4 on a desktop is simple and private.
- Structured outputs are key: By constraining the model to return JSON, we integrate it seamlessly into a large, deterministic pipeline.
- Multimodal expands possibilities: From photo memory apps to accessibility tools, the ability to process images locally opens up countless applications.
As local models continue to improve (e.g., larger context windows, faster inference), we can expect even more complex workflows to run entirely offline, giving users full control over their data.
Note: In 2026, Gemma models have improved significantly; check the latest Ollama library for the newest versions and features.
Happy building!