Monocle Support Matrix¶
This document provides a comprehensive overview of all frameworks and libraries supported by Monocle's instrumentation system. Each framework is instrumented to collect telemetry data including events, attributes, and traces.
Inference Frameworks¶
| Framework | Class | Method | Description |
|---|---|---|---|
| OpenAI | Completions |
create |
Chat completions, embeddings, and responses. Collects: model, tokens, finish_reason, usage stats |
| OpenAI | AsyncCompletions |
create |
Async chat completions. Collects: model, tokens, finish_reason, usage stats |
| OpenAI | Embeddings |
create |
Text embeddings generation. Collects: model, input tokens, embedding dimensions |
| OpenAI | AsyncEmbeddings |
create |
Async text embeddings generation. Collects: model, input tokens, embedding dimensions |
| OpenAI | Responses |
create |
Response generation. Collects: model, tokens, finish_reason, usage stats |
| OpenAI | AsyncResponses |
create |
Async response generation. Collects: model, tokens, finish_reason, usage stats |
| Anthropic | Messages |
create |
Claude chat completions. Collects: model, input/output tokens, stop_reason |
| Anthropic | AsyncMessages |
create |
Async Claude chat completions. Collects: model, input/output tokens, stop_reason |
| Gemini | Models |
generate_content |
Google Gemini text generation. Collects: model, tokens, finish_reason, safety ratings |
| Gemini | AsyncModels |
generate_content |
Async Google Gemini text generation. Collects: model, tokens, finish_reason, safety ratings |
| Gemini | Models |
embed_content |
Google Gemini embeddings. Collects: model, input tokens, embedding dimensions |
| Mistral | Chat |
complete |
Mistral chat completions. Collects: model, tokens, finish_reason, usage stats |
| Mistral | Chat |
complete_async |
Async Mistral chat completions. Collects: model, tokens, finish_reason, usage stats |
| Mistral | Chat |
stream |
Mistral streaming completions. Collects: model, tokens, streaming events |
| Mistral | Chat |
stream_async |
Async Mistral streaming completions. Collects: model, tokens, streaming events |
| Mistral | Embeddings |
create |
Mistral embeddings generation. Collects: model, input tokens, embedding dimensions |
| Hugging Face | InferenceClient |
chat_completion |
Hugging Face chat completions. Collects: model, tokens, finish_reason |
| Hugging Face | AsyncInferenceClient |
chat_completion |
Async Hugging Face chat completions. Collects: model, tokens, finish_reason |
| Azure AI | ChatCompletionsClient |
complete |
Azure AI chat completions. Collects: model, tokens, finish_reason, usage stats |
| Azure AI | ChatCompletionsClient (async) |
complete |
Async Azure AI chat completions. Collects: model, tokens, finish_reason, usage stats |
| LiteLLM | OpenAIChatCompletion |
completion |
OpenAI via LiteLLM. Collects: model, tokens, finish_reason, usage stats |
| LiteLLM | AzureChatCompletion |
completion |
Azure via LiteLLM. Collects: model, tokens, finish_reason, usage stats |
| LiteLLM | Various providers | completion |
DeepSeek and other providers via LiteLLM. Collects: model, tokens, finish_reason, usage stats. Supports: DeepSeek (api.deepseek.com), Together AI, Perplexity, Groq, Cohere, Fireworks, and more |
LangChain¶
| Framework | Class | Method | Description |
|---|---|---|---|
| LangChain | BasePromptTemplate |
invoke |
Prompt template execution. Collects: template variables, prompt text |
| LangChain | BasePromptTemplate |
ainvoke |
Async prompt template execution. Collects: template variables, prompt text |
| LangChain | BaseChatModel |
invoke |
Chat model inference. Collects: model, messages, tokens, finish_reason |
| LangChain | BaseChatModel |
ainvoke |
Async chat model inference. Collects: model, messages, tokens, finish_reason |
| LangChain | LLM |
_generate |
LLM text generation. Collects: model, prompt, tokens, finish_reason |
| LangChain | LLM |
_agenerate |
Async LLM text generation. Collects: model, prompt, tokens, finish_reason |
| LangChain | BaseLLM |
invoke |
Base LLM invocation. Collects: model, input, tokens, finish_reason |
| LangChain | BaseLLM |
ainvoke |
Async base LLM invocation. Collects: model, input, tokens, finish_reason |
| LangChain | BaseRetriever |
invoke |
Document retrieval. Collects: query, retrieved documents, scores |
| LangChain | BaseRetriever |
ainvoke |
Async document retrieval. Collects: query, retrieved documents, scores |
| LangChain | BaseOutputParser |
invoke |
Output parsing. Collects: raw output, parsed result |
| LangChain | BaseOutputParser |
ainvoke |
Async output parsing. Collects: raw output, parsed result |
| LangChain | RunnableSequence |
invoke |
Chain execution. Collects: input, intermediate outputs, final result |
| LangChain | RunnableSequence |
ainvoke |
Async chain execution. Collects: input, intermediate outputs, final result |
| LangGraph | CompiledStateGraph |
invoke |
Graph-based agent execution. Collects: agent state, tool calls, decisions |
| LangGraph | CompiledStateGraph |
ainvoke |
Async graph-based agent execution. Collects: agent state, tool calls, decisions |
| LangGraph | BaseTool |
run |
Tool execution. Collects: tool name, input parameters, output result |
| LangGraph | BaseTool |
arun |
Async tool execution. Collects: tool name, input parameters, output result |
LlamaIndex¶
| Framework | Class | Method | Description |
|---|---|---|---|
| LlamaIndex | BaseRetriever |
retrieve |
Document retrieval. Collects: query, retrieved nodes, similarity scores |
| LlamaIndex | BaseRetriever |
aretrieve |
Async document retrieval. Collects: query, retrieved nodes, similarity scores |
| LlamaIndex | BaseQueryEngine |
query |
Query processing. Collects: query, response, source nodes |
| LlamaIndex | BaseQueryEngine |
aquery |
Async query processing. Collects: query, response, source nodes |
| LlamaIndex | CustomLLM |
chat |
Custom LLM chat. Collects: model, messages, tokens, finish_reason |
| LlamaIndex | CustomLLM |
achat |
Async custom LLM chat. Collects: model, messages, tokens, finish_reason |
| LlamaIndex | OpenAI |
chat |
OpenAI LLM via LlamaIndex. Collects: model, messages, tokens, finish_reason |
| LlamaIndex | OpenAI |
achat |
Async OpenAI LLM via LlamaIndex. Collects: model, messages, tokens, finish_reason |
| LlamaIndex | MistralAI |
chat |
Mistral LLM via LlamaIndex. Collects: model, messages, tokens, finish_reason |
| LlamaIndex | MistralAI |
achat |
Async Mistral LLM via LlamaIndex. Collects: model, messages, tokens, finish_reason |
| LlamaIndex | AgentWorkflow |
run |
Multi-agent workflow execution. Collects: workflow state, agent interactions |
| LlamaIndex | ReActAgent |
run |
ReAct agent execution. Collects: agent reasoning, tool calls, final answer |
| LlamaIndex | ReActAgent |
achat |
Async ReAct agent execution. Collects: agent reasoning, tool calls, final answer |
| LlamaIndex | FunctionAgent |
finalize |
Function agent finalization. Collects: agent state, final output |
| LlamaIndex | FunctionAgent |
take_step |
Function agent step execution. Collects: step input, reasoning, output |
| LlamaIndex | FunctionTool |
call |
Function tool execution. Collects: tool name, parameters, result |
| LlamaIndex | FunctionTool |
acall |
Async function tool execution. Collects: tool name, parameters, result |
| LlamaIndex | AgentWorkflow |
_call_tool |
Multi-agent tool calls. Collects: tool name, parameters, result |
| LlamaIndex | Anthropic |
chat |
Anthropic LLM via LlamaIndex. Collects: model, messages, tokens, finish_reason |
| LlamaIndex | Anthropic |
achat |
Async Anthropic LLM via LlamaIndex. Collects: model, messages, tokens, finish_reason |
| LlamaIndex | Gemini |
chat |
Gemini LLM via LlamaIndex. Collects: model, messages, tokens, finish_reason |
| LlamaIndex | Gemini |
achat |
Async Gemini LLM via LlamaIndex. Collects: model, messages, tokens, finish_reason |
Haystack¶
| Framework | Class | Method | Description |
|---|---|---|---|
| Haystack | InMemoryEmbeddingRetriever |
run |
In-memory document retrieval. Collects: query, retrieved documents, scores |
| Haystack | OpenSearchEmbeddingRetriever |
run |
OpenSearch document retrieval. Collects: query, retrieved documents, scores |
| Haystack | OpenAIGenerator |
run |
OpenAI text generation. Collects: model, prompt, tokens, finish_reason |
| Haystack | OpenAIChatGenerator |
run |
OpenAI chat generation. Collects: model, messages, tokens, finish_reason |
| Haystack | Pipeline |
run |
Haystack pipeline execution. Collects: pipeline steps, intermediate outputs |
| Haystack | AnthropicChatGenerator |
run |
Anthropic chat generation. Collects: model, messages, tokens, finish_reason |
| Haystack | GoogleAIGeminiChatGenerator |
run |
Google AI Gemini chat generation. Collects: model, messages, tokens, finish_reason |
Web Frameworks¶
| Framework | Class | Method | Description |
|---|---|---|---|
| Flask | Flask |
wsgi_app |
HTTP request processing. Collects: method, URL, status_code, response_time |
| Flask | Response |
__call__ |
HTTP response handling. Collects: status_code, headers, response_size |
| FastAPI | FastAPI |
__call__ |
HTTP request processing. Collects: method, URL, status_code, response_time |
| FastAPI | Response |
__call__ |
HTTP response handling. Collects: status_code, headers, response_size |
| aiohttp | Application |
_handle |
Async HTTP request processing. Collects: method, URL, status_code, response_time |
| requests | Session |
request |
HTTP client requests. Collects: method, URL, status_code, response_time |
Cloud Platforms¶
| Framework | Class | Method | Description |
|---|---|---|---|
| AWS Botocore | ClientCreator |
create_client |
AWS service client creation. Collects: service_name, region, client_type |
| Azure Functions | AzureFunctionRouteWrapper |
run_async |
Azure Function execution. Collects: function_name, execution_time, status |
| Azure Functions | AzureFunctionRouteWrapper |
run_sync |
Azure Function execution. Collects: function_name, execution_time, status |
| AWS Lambda | LambdaFunctionRouteWrapper |
run_async |
Lambda Function execution. Collects: function_name, execution_time, status |
| AWS Lambda | LambdaFunctionRouteWrapper |
run_sync |
Lambda Function execution. Collects: function_name, execution_time, status |
Agent Frameworks¶
| Framework | Class | Method | Description |
|---|---|---|---|
| Google ADK | BaseAgent |
run_async |
Google ADK agent execution. Collects: agent state, delegation events |
| Google ADK | FunctionTool |
run_async |
Google ADK tool execution. Collects: tool name, parameters, result |
| Google ADK | Runner |
run_async |
Google ADK runner execution. Collects: request context, execution flow |
| Agents | Runner |
run |
Agent runner execution. Collects: agent state, tool calls, decisions |
| Agents | Runner |
run_sync |
Sync agent runner execution. Collects: agent state, tool calls, decisions |
| Agents | AgentRunner |
_run_single_turn |
Single agent turn execution. Collects: turn input, reasoning, output |
| Agents | FunctionTool |
__init__ |
Function tool initialization. Collects: tool definition, parameters |
| Agents | Handoff |
__init__ |
Agent handoff initialization. Collects: handoff context, delegation info |
| Teams AI | OpenAIModel |
complete_prompt |
Teams AI prompt completion. Collects: model, prompt, response, tokens |
| Teams AI | ActionPlanner |
complete_prompt |
Teams AI action planning. Collects: plan, actions, reasoning |
| A2A | A2AClient |
send_message |
A2A client messaging. Collects: message content, routing info, response |
Protocol Frameworks¶
| Framework | Class | Method | Description |
|---|---|---|---|
| MCP | BaseSession |
send_request |
MCP protocol requests. Collects: request type, parameters, response |
| MCP | convert_mcp_tool_to_langchain_tool |
N/A | MCP to LangChain tool conversion. Collects: tool definition, conversion result |
Provider URLs¶
The following provider URLs are configured or extracted by Monocle's instrumentation:
| Provider | Default API Endpoint | Notes |
|---|---|---|
| OpenAI | https://api.openai.com/v1/ |
Extracted from client configuration |
| Anthropic | https://api.anthropic.com |
Extracted from client configuration |
| Mistral | https://api.mistral.ai |
Hardcoded default endpoint |
| Gemini | https://generativelanguage.googleapis.com/ |
Extracted from client configuration |
| Hugging Face | https://api-inference.huggingface.co/v1/ |
Hardcoded default endpoint |
| Azure OpenAI | Variable (Azure endpoint) | Extracted from client configuration |
| Azure AI Inference | Variable (Azure endpoint) | Extracted from client configuration |
| DeepSeek | https://api.deepseek.com |
Supported via LiteLLM |
| LiteLLM Providers | Variable | Supports multiple providers including DeepSeek, Together AI, Perplexity, Groq, Cohere, Fireworks, and more. Endpoint extracted from client configuration |
Data Collection Summary¶
Each framework collects relevant telemetry data including:
- Inference Events: Model calls, token usage, finish reasons, response times
- Retrieval Events: Query processing, document retrieval, similarity scores
- Agent Events: Agent reasoning, tool calls, delegation events, workflow state
- HTTP Events: Request/response data, status codes, response times, headers
- Tool Events: Tool execution, parameters, results, error handling
- Pipeline Events: Multi-step execution, intermediate outputs, final results
This comprehensive instrumentation enables detailed observability across the entire AI application stack, from individual model calls to complex multi-agent workflows.