Okahu
What is Okahu
Overview
AI Observability is proactive monitoring of your AI apps and cloud infra they run on to understand how to make them work better
Problem
Today’s GenAI applications are complex. They comprises of multiple technology components like workflow code, models, model hosting services, vector databases etc. Monitoring each components behavior, tracking the coorelations between components and understanding big picture is very hard.
Solution
Okahu captures traces and metrics from all the GenAI application component. We automatically discover the relationships between components and build the big picture. Okahu lets you can define your goals like performance and reliabilit, track your applications behavior, help you identify and resolve the problems.
Getting Started
Explore Okahu playground
It’s super easy to setup the okahu tenant. Just login to Okahu portal with your email or social account. We’ll create a tenant for you with sample data.
You can explore the Okahu playground to see what capabilties Okahu offer
Application insights
Okahu uses the traces and metrics data to measure your applications behavior against the your pre-defined goals.
Component map
Okahu automatically discovers the GenAI components used by your application and their dependenecies to build the big picture.
Inference and Prompts
Okahu offers full range of RestAPI to programatically use Okahu’s capabilities. Clone Okahu’s Postman collection to explore the Rest APIs. You’ll need the API key that can be retrieved from settings tab in Okahu portal. The example above shows the details of prompt used in a inference request by the demo application.
Infrastructure metrics
Okahu collects various metrics like GPU or CPU utilization from infrastructure services like Azure OpenAI or NVDIA Triton. You can view these metrics using Okahu’s rest APIs.
Bring your app to Okahu
Once you explore the playground and understand the value Okahu provides, it’s time to monitor your application.
Monocle for indivisual developer
Okahu started a community driven open source project called Monocle under Linux Foundation Data&AI. Monocle provides a Python library that enables a GenAI developer to generate traces from their application runs. You and review these traces locally to help build and tune the application. Please refer to Monocle user guide for enabling your application. We invite you to get involved in the Monocle.It’s a community driven open source project. Your suggestions and contribution are very welcome!
Okahu for teams and enterprises
Okahu extendeds Monocle’s technology to graduate from local development environment to pre-prod and prod where you want to collaborate with other stakeholder like developers, DevOps and leadership. Okahu python SDK (fully API compatile with Monocle) enables you consolidate all the traces in your Okahu tenant. You can now define your application in Okahu and track your goals. Please refer to Okahu application guide to learn how to enable your application in various application hosting options from laptop to Azure Function, AWS Lambda etc.
Bring your GenAI infra and services to Okahu
Okahu collects the metrics and traces from your infrastructure components like Azure OpenAI, AWS Sagemaker or NVDIA Triton. You need to deploy and configure the Okahu agent in your environment where these services are running. For more details, please refer to Okahu’s infra agent guide.
Provide feedback
We welcome your feedback. Please email us at cx@okahu.ai.