How to Implement Structured Output with Local LLMs

Local large language models (LLMs) are an attractive option for building applications that prioritize data privacy and reduce reliance on cloud APIs. Running a model locally, however, is only the first step. In real-world workflows, the local LLM often operates as part of a larger system, where its responses must be consumed by other components—and free-form text is notoriously difficult to work with in such pipelines.

That is where structured output comes in. By defining an expected schema in advance, the local serving runtime can constrain the LLM's generation to follow that structure. As a result, the model returns a regular Python object that your code can parse effortlessly.

In this post, we'll walk through a concrete case study that demonstrates this pattern in practice. We'll use Gemma 4 as the local LLM, Ollama as the serving runtime, and Pydantic to define and validate the output schema. This approach remains highly relevant in 2026, as more applications adopt hybrid architectures that combine local models for privacy-sensitive steps with cloud models for heavy reasoning.

---

1. How to Implement Structured Output with a Local LLM

1.1 A Smart-Home Case Study

Imagine you're building a smart-home application. The user asks a simple question:

Should the dishwasher run now or later?

Before answering, the application must extract device information, timing constraints, and electricity tariffs from household notes. Because these notes contain private details, a local LLM is a natural fit for the first step. It transforms the raw text into a structured object that keeps only the facts essential for scheduling—stripping away personal information.

This sanitized object can then be passed to a more capable cloud LLM for reasoning and scheduling. Here, we focus on the local transformation step, which remains a critical bottleneck for privacy-preserving AI pipelines in 2026.

The household context we'll use is:

USER_QUESTION = "Should the dishwasher run now or later?"

SMART_HOME_CONTEXT = """
It is currently 18:30.

The activity log records that the robot vacuum completed today's kitchen pass
at 16:10 and returned to its dock. No more vacuuming is needed today.

The dishwasher's earliest start is 18:30. A cycle takes 90 minutes and uses about 1.2 kWh.
It must be complete before breakfast at 06:30. Because the dishwasher is beside
the bedrooms, it must stop running by 22:30.

The EV charger's earliest start is 18:30. Charging will take 120 minutes and use about
14 kWh. The car must be charged before its driver leaves at 07:00.

The dryer's earliest start is 19:00. Its cycle takes 75 minutes and uses about
3.2 kWh. It contains the football kit, which must be dry by 23:00. The dryer is
too loud later in the evening, so it must stop running by 21:30.

The washing machine's earliest start is 20:00. Its cycle takes 60 minutes and
uses about 0.9 kWh. It contains tomorrow's work clothes and must finish by 05:30.

A kitchen pass with the robot vacuum takes 45 minutes and uses about 0.2 kWh.
The vacuum's earliest start was 15:00.

The home energy controller permits only one flexible load to run at a time.
Electricity costs 0.45 per kWh from 17:00 to 20:00, 0.22 from 20:00 to 00:00,
0.12 from 00:00 to 06:00, and 0.25 from 06:00 to 17:00.
""".strip()

The local LLM's goal is to retain scheduling facts—like start times, duration, and energy usage—while discarding personal details such as "football kit" or "work clothes" that aren't relevant to the scheduling logic.

1.2 Define the Expected Structure

Next, we need to specify what the sanitized object should look like. The downstream component requires five key pieces of information:

• The current time
• The device mentioned in the user's question
• The controller's capacity (e.g., how many loads can run simultaneously)
• The electricity price schedule
• The list of devices that still need scheduling, each with its runtime and timing constraints

We can model this with Pydantic, which has become the de facto standard for schema validation in Python LLM applications. Here's a clean definition:

from typing import Annotated, List, Optional
from pydantic import BaseModel, Field

ClockTime = Annotated[
    str,
    Field(description="Time in 24-hour format (e.g., 18:30)")
]

class DeviceSchedule(BaseModel):
    """A device that requires scheduling."""
    device_name: str = Field(description="Name of the device (e.g., dishwasher)")
    earliest_start: ClockTime
    cycle_duration_minutes: int = Field(description="Duration in minutes")
    energy_usage_kwh: float = Field(description="Energy consumption in kWh")
    must_finish_by: ClockTime = Field(description="Latest finish time")
    must_stop_by: Optional[ClockTime] = Field(default=None, description="Optional stop time")

class ElectricityPrice(BaseModel):
    """A time-of-use electricity tariff."""
    start_time: ClockTime
    end_time: ClockTime
    price_per_kwh: float

class SanitizedHomeContext(BaseModel):
    """Structured representation of the household notes."""
    current_time: ClockTime
    device_in_question: str = Field(description="Device mentioned in the user question")
    controller_capacity: int = Field(description="Max simultaneous loads (e.g., 1 means only one at a time)")
    price_schedule: List[ElectricityPrice]
    devices_requiring_schedule: List[DeviceSchedule]

With these models in place, we now have a clear contract for the LLM's output. In the next section, we'll see how to enforce this schema during generation using Ollama's structured output support.

via Towards Data Science