Skip to main content

How Xenovia traces tool calls

Most agent risk comes from tool use, not text generation. Xenovia captures tool calls at two levels:

1. LLM-level tool tracing (proxy mode)

When an agent requests a tool call via chat/completions or responses, Xenovia records:
  • Tool name, arguments, and the model’s call ID
  • Tool result (from the subsequent tool role message)
  • Latency between call and result
  • Status (success / error)
These records appear as child steps within the trace (tool_call and tool_result steps), linked to the parent trace via parent_trace_id.

2. MCP tool tracing

Xenovia Runtime implements PreMCPHook and PostMCPHook for Model Context Protocol tool tracing. When an MCP-enabled agent invokes a tool:
  • PreMCPHook: records tool name, arguments, and call ID.
  • PostMCPHook: records result, status, and latency.
MCP tool traces are persisted as trace steps alongside LLM traces, giving a unified view of both LLM outputs and tool executions.

3. SDK-level tool gating (SDK mode)

For tools executed outside the LLM call — database writes, API calls, file operations — use the Xenovia Python SDK:

Access control model

1

Discover

Proxy traffic registers which tools and models an agent is using. Tool names are extracted from every chat/completions request.
2

Classify

In the platform, annotate observed tools with owner, environment, and criticality. This metadata feeds into policy evaluation via input.tool_names and input.tools.
3

Scope

Write Rego policies that allowlist, blocklist, or redact based on tool identity. Policies can reference input.tool_names (a []string), input.tools (full tool schemas), or input.system_prompt for context.
4

Enforce

Policy runs before every LLM call (request-stage) and after every LLM response (response-stage). Tool names requested in the response can be evaluated in the response-stage policy.
5

Record

Every tool call — whether allowed, blocked, or redacted — is recorded in the trace. Operator escalations include the full tool argument context.
  • Least-privilege by default. Blocklist high-risk tools explicitly rather than relying on broad allowlists.
  • Separate by environment. Use different proxies (or different policies on the same proxy) for dev, staging, and prod. Never share production credentials.
  • Require approval for destructive operations. Use intent escalation for tools with irreversible effects (deletions, financial writes, cross-system state changes).
  • Review unused grants periodically. Traces show which tools are actually called. Remove policy permissions for tools that are no longer active.

Policy example: tool access control

Evidence per tool interaction

Each tool call in the trace captures: