LlamaIndex

Setup

The agent example uses the workflow API available in LlamaIndex 0.13 and later. It requires a proxy configured with a model that supports native tool calling. The agent example is checked with Python 3.12, llama-index-core 0.13.0 / llama-index-llms-openai 0.5.0 and core 0.14.24 / OpenAI integration 0.8.1 using simulated HTTP responses. Validate your proxy’s policy and provider configuration separately.

pip install "llama-index-core>=0.13" llama-index-llms-openai llama-index-embeddings-openai llama-index-readers-file
export XENOVIA_API_KEY=xe_...
export XENOVIA_PROXY_ID=your-proxy-id

import os
import uuid
from llama_index.llms.openai import OpenAI
from llama_index.core import Settings

session_id = str(uuid.uuid4())

llm = OpenAI(
    model="gpt-4o-mini",
    api_key=os.environ["XENOVIA_API_KEY"],
    api_base=f"https://runtime.xenovia.io/{os.environ['XENOVIA_PROXY_ID']}/v1",
    default_headers={"X-Xenovia-Session-Id": session_id}
)

Settings.llm = llm

Setting Settings.llm globally applies to all LlamaIndex components that use an LLM — query engines, chat engines, agents — without further configuration. Use the same https://runtime.xenovia.io/{proxy_id}/v1 base URL across the OpenAI-compatible integrations, including the OpenAI Agents SDK. The older /a/{proxy_id}/openai/v1 route is not part of the current runtime API. Use llama_index.llms.openai.OpenAI for every provider the proxy can route to, including Anthropic, Groq, Bedrock, and Gemini. The model argument is a placeholder that the runtime replaces with the proxy’s configured model; provider-native LlamaIndex classes cannot target the proxy.

Embeddings

The runtime does not expose /v1/embeddings yet, so keep the embedding model pointed directly at your provider. An embeddings call sent to the proxy URL returns 404.

from llama_index.embeddings.openai import OpenAIEmbedding

embed_model = OpenAIEmbedding(
    api_key=os.environ["OPENAI_API_KEY"]  # embeddings go straight to the provider
)

Settings.embed_model = embed_model

Only the LLM calls (query synthesis, chat, agent steps) are governed and traced by Xenovia today.

RAG pipeline

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine()
response = query_engine.query("What are the main topics covered?")
print(response)

Agentic query engine

Use FunctionAgent so tool definitions are sent as native tools metadata. The QueryEngineTool below reuses the RAG query engine from the preceding example.

import asyncio
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.tools import QueryEngineTool

tool = QueryEngineTool.from_defaults(
    query_engine=query_engine,
    name="docs_search",
    description="Search internal documentation"
)

agent = FunctionAgent(
    tools=[tool],
    llm=llm,
    system_prompt="Use docs_search to answer questions about internal documentation."
)

async def main():
    response = await agent.run(user_msg="Find information about governance policies")
    print(response)

if __name__ == "__main__":
    asyncio.run(main())

In a notebook, use await main() instead of asyncio.run(main()), because a notebook already has an event loop. The older ReActAgent.from_tools(...) and synchronous .chat() examples do not apply to workflow agents. Modern ReActAgent remains available, but describes tools in its prompt instead of supplying the native tool metadata used by FunctionAgent. Each LLM call, including query synthesis inside docs_search, routes through Xenovia. The session header set during setup groups these calls in Traces. Request-stage policy checks the tools offered to the model; this is separate from authorizing a local action after the model selects a tool and generates its arguments.

Session tracking

The setup example creates one X-Xenovia-Session-Id before building the query engine and agent. Reuse it for the conversation, and create a new client and session ID for a separate conversation. Replacing llm after creating an agent does not update the client already held by that agent.

Handling policy blocks

For a direct query-engine call, a request-stage 403 propagates as openai.PermissionDeniedError:

from openai import PermissionDeniedError

try:
    response = query_engine.query("Drop the database")
except PermissionDeniedError as e:
    print(f"Blocked by policy: {e.message}")

Workflow agents can wrap that error in WorkflowRuntimeError. Use the following handler with the agent above; it recognizes direct and wrapped permission errors and re-raises unrelated failures:

from llama_index.core.workflow.errors import WorkflowRuntimeError
from openai import PermissionDeniedError

async def run_with_policy_handling(message: str):
    try:
        return await agent.run(user_msg=message)
    except (PermissionDeniedError, WorkflowRuntimeError) as error:
        cause = error
        while cause is not None:
            if isinstance(cause, PermissionDeniedError):
                trace_id = cause.response.headers.get("X-Xenovia-Trace-Id")
                print(f"Blocked by policy: {cause.message}; trace: {trace_id}")
                return None
            cause = cause.__cause__
        raise

Call await run_with_policy_handling(...) from your async entry point. This handles a 403 returned before streaming begins. FunctionAgent streams by default; failures after a stream has started can surface differently and must also be tested with the configured provider and response policies.

Start Here

Integrations

Platform

Setup

Embeddings

RAG pipeline

Agentic query engine

Session tracking

Handling policy blocks

​Setup

​Embeddings

​RAG pipeline

​Agentic query engine

​Session tracking

​Handling policy blocks

Setup

Embeddings

RAG pipeline

Agentic query engine

Session tracking

Handling policy blocks