Overview
Agentic AI systems are increasingly adopting automated pipelines that integrate multiple tools. While prior research and benchmarks have focused on task success and task completion, the study of agent-to-tool interaction—particularly within biology agentic workflows—remains limited.
This study investigates a specific class of failures in agent-to-tool interaction where a tool invocation appears successful, yet some or all of the information or functionality delivered via API or wrapper is incomplete or missing—with no communication or notification to the user or the agent about the missing information. We term these silent failures, because neither the user nor the agent is aware that a failure has occurred.
Methodology
To identify such silent failures, we developed an audit mechanism and examined 15 scientific tools integrated within the ToolUniverse environment, along with their associated API documentation and tool documentation.
ToolUniverse serves as our experimental environment rather than the object of the study itself.
The study is structured around 7 failure loci characterizing where failures occur within the chain. Candidate failures were discovered via LLM-based methods and automated testing, then manually validated.
Findings
We observed 91 validated failures, the most frequent being:
- Missing data or fields
- Inconsistencies in search, filtering, or ranking criteria
Regarding where these failures originate, the distribution was heavily skewed toward lower layers of the stack:
- API layer: 51 failures
- Wrapper layer: 25 failures
The results show that silent failures originate upstream of the triggering event and propagate downstream into apparently valid scientific outputs, with the potential for silent failure amplification across the pipeline.
Implications and Proposal
To address these failures, we propose the concept of contextual reliability and suggest mechanisms for:
- Testing
- Disclosing
- Monitoring
- Measuring
such failures across the agent-tool interaction pipeline.
Metadata
- Subjects: Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
- Cite as: arXiv:2609.26836 [cs.AI]
- DOI: 10.48550/arXiv.2609.26836
- Submitted: 21 Sep 2026
- Authors: Shreya Gopalan, Devansh Singh, Sundaraparipurnan Narayanan
via ArXiv AI