Skip to content

Quickstart

Run your first eval on a traced agent:

# test_travel_agent.py
import pytest
from my_app import travel_agent     # your agent

@pytest.mark.asyncio
async def test_books_a_flight(monocle_trace_asserter):
    await monocle_trace_asserter.run_agent_async(
        travel_agent, "langgraph", "Book a flight to Mumbai on 22 Nov")

    monocle_trace_asserter.called_tool("book_flight").contains_input("Mumbai")
    monocle_trace_asserter.with_evaluation("okahu") \
        .check_eval("hallucination", expected="no_hallucination")

pip install monocle_test_tools
pytest test_travel_agent.py

That is the whole loop: run the agent, assert what it did, judge the answer. Each assertion passes or fails like any pytest; the trace and eval results also land in Okahu.

Before this works: a traced agent + a key

Getting an agent instrumented and sending traces to Okahu is covered elsewhere, so start there rather than repeat it:

  • From a ready-made app — the Examples page has traced travel agents for LangGraph, Google ADK, AWS Strands and Microsoft Agent Framework. Clone one, add your keys, and it already emits Monocle traces to Okahu.
  • Your own agent — see Evaluate AI agents for the four pieces, and Instrument for turning on tracing in one line.

Set your key once:

export OKAHU_API_KEY=okh_...        # Okahu Portal > Settings > API keys
export MONOCLE_EXPORTER=okahu       # send traces to Okahu

Next

Check agent behaviour covers every assertion type: calls, inputs and outputs, limits, code-based checks, and evals.