When an AI agent starts serving customers and executing transactions, testing also has to validate how it behaves: what it decides and what actions it takes. In this post we explain how we approach AI agent testing at Qualis, from what we test in an agent to how we measure its quality with evidence.
Why AI agent testing is different
An agent interprets instructions, makes decisions, looks up information, uses tools and takes actions. Each of those steps can fail in a way traditional testing does not account for, because the same input does not always produce the same output.
That is why, beyond validating that the software works, we account for four risks specific to agents:
- Variability. The same scenario can produce different results.
- Interpretation. The agent can misunderstand an instruction.
- Business rules. It can make decisions outside the defined limits.
- Actions. It can use wrong data or take improper actions.
The last two risks are about the agent's limits, and that is where the harness comes in. The harness is the system that sets the limits within which the AI can act. It defines which data and tools the agent can access and which actions it can take with them. It also coordinates execution within those limits.
Testing an agent means testing its harness too: checking that the limits are well defined and that they hold when someone tries to push them. An agent can answer every request well and still reach data it should not see if the harness fails.
What we test in an AI agent
Our testing strategy for agents covers six areas. Each one targets a different kind of failure:
- Functional and conversational. That the agent understands intents and instructions, follows the rules, keeps the context and handles multi-turn conversations.
- Security and attacks. Prompt injection, jailbreaks, data exposure and access controls. This is the area where the harness is tested against attempts to push its limits.
- Response quality. Hallucinations, inconsistencies, faithfulness to sources and relevance of what it answers.
- Tools and integrations. The APIs it calls, the actions it triggers, the permissions it runs with, the errors it receives and the results of each process.
- Automation and regression. Repeatable test suites and detection of changes between versions.
- Performance and scalability. Latency, concurrency, consumption, cost and stability under load.
The areas complement each other. An agent with good answers can fail at using tools, and one that works well in one version can change its behavior in the next if the model or the instructions change.
How we design a testing strategy for agents
The approach is structured and risk-based. We organize it in five steps:
- Understanding the agent. Its purpose, its architecture, the sources it consults, the tools it uses and its limits. This is where the harness is mapped: what the agent can do and what it is barred from doing.
- Risk identification. What impact each failure would have on the business and which controls need to be checked.
- Scenario design. Positive, negative, adversarial and integration cases.
- Execution and evaluation. Tests are run and responses are evaluated with model assistance, which makes it possible to cover a volume of responses that would be hard to review by hand.
- Measurement and improvement. Results and regressions are analyzed, and actions are defined to improve the agent's reliability.
The order matters. If scenarios are designed before understanding the agent and its limits, the tests cover what the agent should do and can miss what it should never do.
Example scenarios: from the expected to the unexpected
This is what some concrete scenarios look like for an agent operating at a bank or fintech:
| Test scenario | What we validate |
|---|---|
| Request for a third party's balance | Privacy and authorization |
| Attempt to change the instructions | Resistance to prompt injection |
| Data input | That it is complete, consistent and current, and comes from the right source |
| External API error | Failure handling without making up results |
| Change of intent mid-conversation | Context and adaptation |
| Execution of a financial transaction | Permission limits and confirmation before transacting |
We test what the agent should do and what it should never do. Qualis Lab, AI Agent Testing
Several of these scenarios test what happens when someone asks for something improper or when something fails around the agent. They are the ones that check that the harness holds its limits in situations nobody planned for.
How we measure an agent's quality
An agent's quality is measured with evidence. We use six metrics:
| Metric | What it measures |
|---|---|
| Task Success Rate | Tasks completed correctly |
| Faithfulness | Fidelity to the sources |
| Safety Compliance | Compliance with policies and restrictions |
| Tool Accuracy | Correct tool selection and execution |
| Consistency | Stability across repeated tests |
| Latency | Response and execution times |
Safety Compliance and Tool Accuracy look closely at the harness. The first measures that policies and restrictions are followed. The second measures that the agent picks and runs its tools correctly within those limits. Consistency requires running each scenario several times, because the same scenario can produce different results.
AI agents also need a quality strategy. At Qualis we combine our software quality expertise with techniques for evaluating intelligent systems, with these goals:
- Reduce operational and security risks.
- Identify incorrect behavior before production.
- Validate business rules and automated decisions.
- Measure reliability with evidence.
- Support the ongoing evolution of AI solutions.
If your agent runs on Salesforce, our post on Agentforce agents covers where to start. How much testing an agent needs depends on what it can do and what happens if it gets it wrong, and that is what we define within our testing and QA service.