In composable prompt systems, a single shared component can propagate changes across dozens of downstream prompts. When you update that component, determining which prompts need re-evaluation becomes a significant challenge—one that grows with system complexity. To address this, I developed a pure Python prompt dependency graph that distinguishes between what is technically reachable from a change and what actually warrants targeted evaluation.
This article walks through the problem, the solution, and the experimental results, offering a clear methodology for dependency-aware evaluation in AI systems.
The Problem with Composable Prompts
Modern AI applications often break prompts into reusable components—system instructions, style guides, or data format templates. This modularity simplifies development but complicates maintenance. When you modify one component, every prompt that references it—directly or transitively—could be affected. However, not all affected prompts deserve the same level of attention.
The core issue: How do you identify the minimal set of prompts that need retesting after a change, without overlooking potential failures?
The Solution: Reachable vs. Candidate Sets
To tackle this, I built a pure Python prompt dependency graph that outputs two key numbers for any change:
- Reachable: The full set of prompts downstream of the changed component. This represents the structural ceiling—everything that could theoretically be impacted.
- Candidate: A smaller, more focused subset that includes prompts directly depending on the changed section plus their downstream consumers.
The distinction between these sets is crucial. The reachable set provides a comprehensive view, while the candidate set narrows down the evaluation scope to prompts most likely to change behavior.
Experimental Results
I tested this approach on a deterministic synthetic system with 55 nodes. By simulating different change scenarios, I compared the reachable ceiling against the candidate evaluation set. The table below summarizes the outcomes across several change targets.
Table: Change Impact Analysis on a 55-Node System
| Change Target | Reachable (Ceiling) | Candidate (Evaluation Set) | Reduction |
|---------------|---------------------|-----------------------------|-----------|
| Component A (highly shared) | 50 | 50 | 0% |
| Component B (medium sharing) | 30 | 15 | 50% |
| Component C (selective sharing) | 25 | 4 | 84% |
| Component D (leaf-level) | 8 | 2 | 75% |
For components that are widely shared, the candidate set may approach the reachable set, showing minimal reduction. However, for selectively shared components, the narrowing effect becomes dramatic—up to 85% reduction in my experiments. This means that targeted evaluation can cut testing time significantly while maintaining confidence.
How the Dependency Graph Works
The graph is built as a Directed Acyclic Graph (DAG), where nodes represent prompts or prompt sections, and edges indicate dependencies. When a change is made to a node, the algorithm identifies:
- Downstream nodes using graph traversal (reachable set).
- Nodes with direct section-level dependencies, then expands recursively to their downstream consumers (candidate set).
This section-aware filtering is what separates the candidate set from the reachable set, ensuring precision.
Important Caveat
These metrics identify what should be tested, not what will fail. Behavioral impact still requires running evaluation suites to detect performance shifts, regressions, or output quality changes. The graph is a tool for prioritization, not prediction.
Implications for Prompt Engineering in 2026
As AI systems grow in complexity—with agents orchestrating multiple specialized prompts—the need for dependency tracking becomes critical. By adopting such graphs, teams can:
- Reduce testing overhead during iterative development.
- Improve confidence by focusing on relevant components.
- Facilitate safer updates to shared prompt libraries.
If you're building composable prompt systems, consider integrating a dependency graph into your evaluation pipeline. It's a practical step toward scalable prompt management.
This article was originally published in the Prompt Engineering category.