monoai.observability

MonoAI Observability

MonoAI provides built-in support for multiple observability and monitoring services to help you track, debug, and analyze your AI model interactions.

Supported Services

Langfuse

Open-source observability and analytics platform for LLM applications.

Dependencies:

pip install langfuse==2.59.7

Logfire

Pydantic's observability platform for monitoring and debugging applications.

Dependencies:

pip install opentelemetry-api==1.25.0 opentelemetry-sdk==1.25.0 logfire

DeepEval

Open-source evaluation framework for LLM applications.

LangSmith

LangChain's platform for debugging, testing, and monitoring LLM applications.

Dependencies:

pip install langsmith==0.1.11

⚠️ MonoAI is currently based on litellm, so other services supported by litellm could be used out-of-the-box.

Setup

1. Configure Services

Add the observability field to your ai.yaml configuration file:

observability: ["langfuse", "logfire"]

2. Add API Keys

Create an observability.keys file in your project root and add the required credentials:

# observability.keys
LANGFUSE_PUBLIC_KEY=your_langfuse_public_key
LANGFUSE_SECRET_KEY=your_langfuse_secret_key
LOGFIRE_TOKEN=your_logfire_token

With Metadata

Pass custom metadata in the metadata parameter to track additional context with services that support it:

# Synchronous request with metadata
response = model.ask(
    "Hello, how are you?", 
    metadata={
        "user_id": "12345",
        "session_id": "abc-def-ghi",
        "feature": "chat",
        "environment": "production"
    }
)

View Source

 1"""
 2# MonoAI Observability
 3
 4MonoAI provides built-in support for multiple observability and monitoring services to help you track, debug, and analyze your AI model interactions.
 5
 6## Supported Services
 7
 8### [Langfuse](https://www.langfuse.com/)
 9Open-source observability and analytics platform for LLM applications.
10
11**Dependencies:**
12```bash
13pip install langfuse==2.59.7
14```
15
16### [Logfire](https://logfire.ai/)
17Pydantic's observability platform for monitoring and debugging applications.
18
19**Dependencies:**
20```bash
21pip install opentelemetry-api==1.25.0 opentelemetry-sdk==1.25.0 logfire
22```
23
24### [DeepEval](https://deeveval.com/)
25Open-source evaluation framework for LLM applications.
26
27### [LangSmith](https://www.langchain.com/langsmith)
28LangChain's platform for debugging, testing, and monitoring LLM applications.
29
30**Dependencies:**
31```bash
32pip install langsmith==0.1.11
33```
34
35⚠️ MonoAI is currently based on litellm, so [other services supported by litellm](https://docs.litellm.ai/docs/observability/agentops_integration) could be used out-of-the-box.
36
37
38## Setup
39
40### 1. Configure Services
41
42Add the `observability` field to your `ai.yaml` configuration file:
43
44```yaml
45observability: ["langfuse", "logfire"]
46```
47
48### 2. Add API Keys
49
50Create an `observability.keys` file in your project root and add the required credentials:
51
52```bash
53# observability.keys
54LANGFUSE_PUBLIC_KEY=your_langfuse_public_key
55LANGFUSE_SECRET_KEY=your_langfuse_secret_key
56LOGFIRE_TOKEN=your_logfire_token
57```
58
59### With Metadata
60
61Pass custom metadata in the `metadata` parameter to track additional context with services that support it:
62
63```python
64# Synchronous request with metadata
65response = model.ask(
66    "Hello, how are you?", 
67    metadata={
68        "user_id": "12345",
69        "session_id": "abc-def-ghi",
70        "feature": "chat",
71        "environment": "production"
72    }
73)
74```
75"""