diff options
| author | Dhravya Shah <[email protected]> | 2025-09-28 16:42:06 -0700 |
|---|---|---|
| committer | Dhravya Shah <[email protected]> | 2025-09-28 16:42:06 -0700 |
| commit | 2093b316d9ecb9cfa9c550f436caee08e12f5d11 (patch) | |
| tree | 07b87fbd48b0b38ef26b9d5f839ad8cd61d82331 /apps/docs/memory-router | |
| parent | Merge branch 'main' of https://github.com/supermemoryai/supermemory (diff) | |
| download | supermemory-2093b316d9ecb9cfa9c550f436caee08e12f5d11.tar.xz supermemory-2093b316d9ecb9cfa9c550f436caee08e12f5d11.zip | |
migrate docs to public
Diffstat (limited to 'apps/docs/memory-router')
| -rw-r--r-- | apps/docs/memory-router/overview.mdx | 158 | ||||
| -rw-r--r-- | apps/docs/memory-router/usage.mdx | 214 | ||||
| -rw-r--r-- | apps/docs/memory-router/with-memory-api.mdx | 113 |
3 files changed, 485 insertions, 0 deletions
diff --git a/apps/docs/memory-router/overview.mdx b/apps/docs/memory-router/overview.mdx new file mode 100644 index 00000000..4c142e5e --- /dev/null +++ b/apps/docs/memory-router/overview.mdx @@ -0,0 +1,158 @@ +--- +title: "Overview" +description: "Transform any LLM into an intelligent agent with unlimited context and persistent memory" +sidebarTitle: "Overview" +--- + +The Memory Router is a transparent proxy that sits between your application and your LLM provider, automatically managing context and memories without requiring any code changes. + +<Note> +**Live Demo**: Try the Memory Router at [supermemory.chat](https://supermemory.chat) to see it in action. +</Note> + +<Tip> +**Using Vercel AI SDK?** Check out our [AI SDK integration](/ai-sdk/overview) for the cleanest implementation with `@supermemory/tools/ai-sdk` - it's our recommended approach for new projects. +</Tip> + +## What is the Memory Router? + +The Memory Router gives your LLM applications: + +- **Unlimited Context**: No more token limits - conversations can extend indefinitely +- **Automatic Memory Management**: Intelligently chunks, stores, and retrieves relevant context +- **Zero Code Changes**: Works with your existing OpenAI-compatible clients +- **Cost Optimization**: Save up to 70% on token costs through intelligent context management + +## How It Works + +<Steps> + <Step title="Proxy Request"> + Your application sends requests to Supermemory instead of directly to your LLM provider + </Step> + + <Step title="Context Management"> + Supermemory automatically: + - Removes unnecessary context from long conversations + - Searches relevant memories from previous interactions + - Appends the most relevant context to your prompt + </Step> + + <Step title="Forward to LLM"> + The optimized request is forwarded to your chosen LLM provider + </Step> + + <Step title="Async Memory Creation"> + New memories are created asynchronously without blocking the response + </Step> +</Steps> + +## Key Benefits + +### For Developers + +- **Drop-in Integration**: Just change your base URL - no other code changes needed +- **Provider Agnostic**: Works with OpenAI, Anthropic, Google, Groq, and more +- **Shared Memory Pool**: Memories created via API are available to the Router and vice versa +- **Automatic Fallback**: If Supermemory has issues, requests pass through directly + +### For Applications + +- **Better Long Conversations**: Maintains context even after thousands of messages +- **Consistent Responses**: Memories ensure consistent information across sessions +- **Smart Retrieval**: Only relevant context is included, improving response quality +- **Cost Savings**: Automatic chunking reduces token usage significantly + +## When to Use the Memory Router + +The Memory Router is ideal for: + +<Tabs> + <Tab title="Perfect For"> + - **Chat Applications**: Customer support, AI assistants, chatbots + - **Long Conversations**: Sessions that exceed model context windows + - **Multi-Session Memory**: Users who return and continue conversations + - **Quick Prototypes**: Get memory capabilities without building infrastructure + </Tab> + + <Tab title="Consider API Instead"> + - **Custom Retrieval Logic**: Need specific control over what memories to fetch + - **Non-Conversational Use**: Document processing, analysis tools + - **Complex Filtering**: Need advanced metadata filtering + - **Batch Operations**: Processing multiple documents at once + </Tab> +</Tabs> + +## Supported Providers + +The Memory Router works with any OpenAI-compatible endpoint: + +| Provider | Base URL | Status | +|----------|----------|---------| +| OpenAI | `api.openai.com/v1` | ✅ Fully Supported | +| Anthropic | `api.anthropic.com/v1` | ✅ Fully Supported | +| Google Gemini | `generativelanguage.googleapis.com/v1beta/openai` | ✅ Fully Supported | +| Groq | `api.groq.com/openai/v1` | ✅ Fully Supported | +| DeepInfra | `api.deepinfra.com/v1/openai` | ✅ Fully Supported | +| OpenRouter | `openrouter.ai/api/v1` | ✅ Fully Supported | +| Custom | Any OpenAI-compatible | ✅ Supported | + +<Warning> +**Not Yet Supported**: +- OpenAI Assistants API (`/v1/assistants`) +</Warning> + +## Authentication + +The Memory Router requires two API keys: + +1. **Supermemory API Key**: For memory management +2. **Provider API Key**: For your chosen LLM provider + +You can provide these via: +- Headers (recommended for production) +- URL parameters (useful for testing) +- Request body (for compatibility) + +## How Memories Work + +When using the Memory Router: + +1. **Automatic Extraction**: Important information from conversations is automatically extracted +2. **Intelligent Chunking**: Long messages are split into semantic chunks +3. **Relationship Building**: New memories connect to existing knowledge +4. **Smart Retrieval**: Only the most relevant memories are included in context + +<Note> +Memories are shared between the Memory Router and Memory API when using the same `user_id`, allowing you to use both together. +</Note> + +## Response Headers + +The Memory Router adds diagnostic headers to help you understand what's happening: + +| Header | Description | +|--------|-------------| +| `x-supermemory-conversation-id` | Unique conversation identifier | +| `x-supermemory-context-modified` | Whether context was modified (`true`/`false`) | +| `x-supermemory-tokens-processed` | Number of tokens processed | +| `x-supermemory-chunks-created` | New memory chunks created | +| `x-supermemory-chunks-retrieved` | Memory chunks added to context | + +## Error Handling + +The Memory Router is designed for reliability: + +- **Automatic Fallback**: If Supermemory encounters an error, your request passes through unmodified +- **Error Headers**: `x-supermemory-error` header provides error details +- **Zero Downtime**: Your application continues working even if memory features are unavailable + +## Rate Limits & Pricing + +### Rate Limits +- No Supermemory-specific rate limits +- Subject only to your LLM provider's limits + +### Pricing +- **Free Tier**: 100k tokens stored at no cost +- **Standard Plan**: $20/month after free tier +- **Usage-Based**: Each conversation includes 20k free tokens, then $1 per million tokens diff --git a/apps/docs/memory-router/usage.mdx b/apps/docs/memory-router/usage.mdx new file mode 100644 index 00000000..61dadfbf --- /dev/null +++ b/apps/docs/memory-router/usage.mdx @@ -0,0 +1,214 @@ +--- +title: "Usage" +description: "How to implement the Memory Router in your application" +sidebarTitle: "Usage" +--- + +Add unlimited memory to your LLM applications with just a URL change. + +## Prerequisites + +You'll need: +1. A [Supermemory API key](https://console.supermemory.ai) +2. Your LLM provider's API key + +## Basic Setup + +<Steps> + <Step title="Get Your API Keys"> + **Supermemory API Key:** + 1. Sign up at [console.supermemory.ai](https://console.supermemory.ai) + 2. Navigate to **API Keys** → **Create API Key** + 3. Copy your key + + **Provider API Key:** + - [OpenAI](https://platform.openai.com/api-keys) + - [Anthropic](https://console.anthropic.com/settings/keys) + - [Google Gemini](https://aistudio.google.com/app/apikey) + - [Groq](https://console.groq.com/keys) + </Step> + + <Step title="Update Your Base URL"> + Prepend `https://api.supermemory.ai/v3/` to your provider's URL: + + ``` + https://api.supermemory.ai/v3/[PROVIDER_URL] + ``` + </Step> + + <Step title="Add Authentication"> + Include both API keys in your requests (see examples below) + </Step> +</Steps> + +## Provider URLs + +<CodeGroup> + +```text OpenAI +https://api.supermemory.ai/v3/https://api.openai.com/v1/ +``` + +```text Anthropic +https://api.supermemory.ai/v3/https://api.anthropic.com/v1/ +``` + +```text Google Gemini +https://api.supermemory.ai/v3/https://generativelanguage.googleapis.com/v1beta/openai/ +``` + +```text Groq +https://api.supermemory.ai/v3/https://api.groq.com/openai/v1/ +``` + +</CodeGroup> + +## Implementation Examples + +<Tabs> + <Tab title="Python"> + ```python + from openai import OpenAI + + client = OpenAI( + api_key="YOUR_OPENAI_API_KEY", + base_url="https://api.supermemory.ai/v3/https://api.openai.com/v1/", + default_headers={ + "x-supermemory-api-key": "YOUR_SUPERMEMORY_API_KEY", + "x-sm-user-id": "user123" # Unique user identifier + } + ) + + # Use as normal + response = client.chat.completions.create( + model="gpt-4", + messages=[ + {"role": "user", "content": "Hello!"} + ] + ) + + print(response.choices[0].message.content) + ``` + </Tab> + + <Tab title="TypeScript"> + ```typescript + import OpenAI from 'openai'; + + const client = new OpenAI({ + apiKey: process.env.OPENAI_API_KEY, + baseURL: 'https://api.supermemory.ai/v3/https://api.openai.com/v1/', + defaultHeaders: { + 'x-supermemory-api-key': process.env.SUPERMEMORY_API_KEY, + 'x-sm-user-id': 'user123' // Unique user identifier + } + }); + + // Use as normal + const response = await client.chat.completions.create({ + model: 'gpt-4', + messages: [ + { role: 'user', content: 'Hello!' } + ] + }); + + console.log(response.choices[0].message.content); + ``` + </Tab> + + <Tab title="cURL"> + ```bash + curl -X POST "https://api.supermemory.ai/v3/https://api.openai.com/v1/chat/completions" \ + -H "Authorization: Bearer YOUR_OPENAI_API_KEY" \ + -H "x-supermemory-api-key: YOUR_SUPERMEMORY_API_KEY" \ + -H "x-sm-user-id: user123" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4", + "messages": [{"role": "user", "content": "Hello!"}] + }' + ``` + </Tab> +</Tabs> + +## Alternative: URL Parameters + +If you can't modify headers, pass authentication via URL parameters: + +<CodeGroup> + +```python Python +client = OpenAI( + api_key="YOUR_OPENAI_API_KEY", + base_url="https://api.supermemory.ai/v3/https://api.openai.com/v1/chat/completions?userId=user123" +) + +# Then set Supermemory API key as environment variable: +# export SUPERMEMORY_API_KEY="your_key_here" +``` + +```typescript TypeScript +const client = new OpenAI({ + apiKey: process.env.OPENAI_API_KEY, + baseURL: 'https://api.supermemory.ai/v3/https://api.openai.com/v1/chat/completions?userId=user123' +}); + +// Set Supermemory API key as environment variable: +// SUPERMEMORY_API_KEY="your_key_here" +``` + +```bash cURL +curl -X POST "https://api.supermemory.ai/v3/https://api.openai.com/v1/chat/completions?userId=user123" \ + -H "Authorization: Bearer YOUR_OPENAI_API_KEY" \ + -H "x-supermemory-api-key: YOUR_SUPERMEMORY_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello!"}]}' +``` + +</CodeGroup> + +## Conversation Management + +### Managing Conversations + +Use `x-sm-conversation-id` to maintain conversation context across requests: + +```python +# Start a new conversation +response1 = client.chat.completions.create( + model="gpt-4", + messages=[{"role": "user", "content": "My name is Alice"}], + extra_headers={ + "x-sm-conversation-id": "conv_123" + } +) + +# Continue the same conversation later +response2 = client.chat.completions.create( + model="gpt-4", + messages=[{"role": "user", "content": "What's my name?"}], + extra_headers={ + "x-sm-conversation-id": "conv_123" + } +) +# Response will remember "Alice" +``` + +### User Identification + +Always provide a unique user ID to isolate memories between users: + +```python +# Different users have separate memory spaces +client_alice = OpenAI( + api_key="...", + base_url="...", + default_headers={"x-sm-user-id": "alice_123"} +) + +client_bob = OpenAI( + api_key="...", + base_url="...", + default_headers={"x-sm-user-id": "bob_456"} +) +``` diff --git a/apps/docs/memory-router/with-memory-api.mdx b/apps/docs/memory-router/with-memory-api.mdx new file mode 100644 index 00000000..fcb09361 --- /dev/null +++ b/apps/docs/memory-router/with-memory-api.mdx @@ -0,0 +1,113 @@ +--- +title: "Use with Memory API" +description: "Combine the Memory Router with Memory API for maximum control" +sidebarTitle: "Use with Memory API" +--- + +The Memory Router and Memory API share the same memory pool. When you use the same `user_id`, memories are automatically shared between both systems. + +## How They Work Together + +<Note> +**Key Insight**: Both the Router and API access the same memories when using identical `user_id` values. This enables powerful hybrid implementations. +</Note> + +### Shared Memory Pool + +```python +# Memory created via API +from supermemory import Client + +api_client = Client(api_key="YOUR_SUPERMEMORY_KEY") + +# Add memory via API +api_client.memories.add({ + "content": "User prefers Python over JavaScript for backend development", + "user_id": "user123" +}) + +# Later, in your chat application using Router +from openai import OpenAI + +router_client = OpenAI( + api_key="YOUR_OPENAI_KEY", + base_url="https://api.supermemory.ai/v3/https://api.openai.com/v1/", + default_headers={ + "x-supermemory-api-key": "YOUR_SUPERMEMORY_KEY", + "x-sm-user-id": "user123" # Same user_id + } +) + +# Router automatically has access to the API-created memory +response = router_client.chat.completions.create( + model="gpt-4", + messages=[{"role": "user", "content": "What language should I use for my new backend?"}] +) +# Response will consider the Python preference +``` + +## Pre-load Context via API + +Use the API to add documents and context before conversations: + +```python +# Step 1: Load user's documents via API +api_client.memories.add({ + "content": "https://company.com/product-docs.pdf", + "user_id": "support_agent_123", + "metadata": {"type": "product_documentation"} +}) + +# Step 2: Support agent uses chat with Router +router_client = OpenAI( + base_url="https://api.supermemory.ai/v3/https://api.openai.com/v1/", + default_headers={"x-sm-user-id": "support_agent_123"} +) + +# Agent has automatic access to product docs +response = router_client.chat.completions.create( + model="gpt-4", + messages=[{"role": "user", "content": "How does the enterprise pricing work?"}] +) +``` + + +## Best Practices + +### 1. Consistent User IDs + +Always use the same `user_id` format across both systems: + +```python +# ✅ Good - consistent user_id +api_client.memories.add({"user_id": "user_123"}) +router_headers = {"x-sm-user-id": "user_123"} + +# ❌ Bad - inconsistent user_id +api_client.memories.add({"user_id": "user-123"}) +router_headers = {"x-sm-user-id": "user_123"} # Different format! +``` + +### 2. Use Container Tags for Organization + +```python +# API: Add memories with tags +api_client.memories.add({ + "content": "Q3 revenue report", + "user_id": "analyst_1", + "containerTag": "financial_reports" +}) + +# Router: Memories are automatically organized +# The Router will intelligently retrieve from the right containers +``` + +### 3. Leverage Each System's Strengths + +| Use Case | Best Choice | Why | +|----------|------------|-----| +| Chat conversations | Router | Automatic context management | +| Document upload | API | Batch processing, custom IDs | +| Search & filter | API | Advanced query capabilities | +| Quick prototypes | Router | Zero code changes | +| Memory management | API | Full CRUD operations | |