About Respan
Overview
Respan is an open-source AI gateway with built-in observability and evals, built by the team formerly known as Keywords AI. Developers point an OpenAI-compatible SDK at a single endpoint to reach 1,000+ models across OpenAI, Anthropic, Google Gemini and Vertex AI, AWS Bedrock, Groq, Together AI, and other providers. Routing, tracing, and scoring read from the same request data, so any request sent through the gateway is captured without extra instrumentation.
The platform applies fallback lists, response caching, and spend limits to every routed request, then turns each LLM call, tool run, retrieval, and agent turn into a span inside one trace. Teams score output with LLM judges, code checks, and human reviewers, and watch requests, errors, cost, latency, and tokens on one dashboard. Respan joined Y Combinator's Winter 2024 batch, is backed by Gradient Ventures, and reports processing more than 80 trillion tokens.
Key Benefits
- One OpenAI-compatible endpoint reaches 1,000+ models, and switching providers takes a single-word change.
- Fallback lists keep requests flowing, with Respan reporting 99.99% uptime alongside fallbacks against 96.43% for a single provider.
- Response caching serves repeat calls instantly and reports the cost and token savings behind each cache hit.
- Budgets and rate limits apply per key, per customer, or org-wide, with warnings before requests are blocked.
- Evaluators combine an LLM judge, deterministic code checks, and human review into one score per output.
- Threshold alerts on cost, errors, latency, or tokens reach Slack, email, or a webhook.
How It Works
Developers start by pointing the SDK they already use at the Respan endpoint and generating an API key. From the dashboard they configure fallback lists, caching, and budgets, then watch live traffic arrive as traces and dashboard metrics. To grade quality, they sample real requests from logs by filter and sampling rate or upload a CSV, attach an evaluator, and run each candidate model or prompt against that dataset. Comparing score distributions side by side shows which version wins, and deploying the same evaluator on filtered, sampled production spans surfaces regressions in real time.
Use Cases
- Support engineering teams trace every agent turn behind customer replies and score sampled tickets for faithfulness.
- Platform engineers route traffic across providers with fallbacks and load balancing to survive provider outages.
- AI product teams compare prompt versions on a dataset pulled from real traffic before shipping a change.
- Finance and growth leads cap model spend with per-customer budgets and get warned as usage nears each limit.
- Healthcare teams running agents need a signed BAA, PII masking, and custom retention on Enterprise.
- Solo developers prototyping an agent start on the Free tier with the full platform and 100k logs.
Why Choose This Product
Respan keeps routing and evaluation on one data path, so a request that passes through the gateway requires no separate instrumentation to be traced or scored. Fallbacks, caching, and spend limits ship as built-in gateway behavior rather than bolt-on services, and the documentation covers a broad set of agent frameworks and LLM SDKs. Self-hosting is not supported today, and the team lists it as roadmap work.
Respan Pros & Cons
- Single OpenAI-compatible endpoint covering 1,000+ models
- Automatic fallbacks with a reported 99.99% uptime
- Evaluators mix LLM judges, code checks, and human review
- SOC 2 Type II, ISO 27001, GDPR, and HIPAA coverage documented
- Full platform, 100k logs, and 5 datasets on the free tier
- Self-hosting is not currently supported
- Free tier retains logs only 7 days and caps datasets at 5
- HIPAA compliance requires a paid add-on
- Team tier seats beyond five cost extra per member
Key Features
Unified Model Gateway
A single OpenAI-compatible endpoint reaches 1,000+ models across OpenAI, Anthropic, Google, AWS Bedrock, Groq, Together AI, and other providers.
Automatic Fallbacks
Respan moves to the next model on a fallback list the moment one errors or rate-limits, supporting a reported 99.99% uptime.
Response Caching
Caching stores a request and its response, then serves the same call again with tracked cost and token savings per hit.
Budgets And Limits
Spend budgets and rate limits apply per key, per customer, or org-wide, with warnings near limits and blocked requests past them.
Composable Evaluators
An LLM judge, a deterministic code check, and a human reviewer compose into one evaluator that turns any output into a single score.
Production Datasets
Teams pull real requests from logs by filter and sampling rate or upload a CSV so every test case came from actual users.
Score Comparisons
Running a model or prompt against a dataset lets teams compare score distributions side by side before shipping a change.
Span-Level Tracing
Every LLM call, tool run, retrieval, and agent turn becomes a span holding its input, output, latency, cost, and tokens.
Metric Alerts
Alerts fire to Slack, email, or a webhook when cost, errors, latency, or tokens cross a threshold over a chosen window.
Prompt Management
A collaborative prompt IDE includes version control, one-click deployment, release management, a model playground, and Jinja templates.
Respan Pricing
View full pricing โ- Full platform access
- 100k logs retained 7 days
- 1k scores
- 5 datasets
- 2 evaluators
- 10 prompts
- Everything in Free
- 10k scores and 100k logs retained 30 days
- Unlimited datasets, evaluators, and prompts
- Private Slack channel and SOC 2 report
- 5 team members included
- 99.99% uptime SLA
- Custom packages with volume discount
- SAML SSO and advanced admin roles
- Custom SLAs and dedicated support engineer
- HIPAA BAA, PII masking, and data retention management
- Custom retention periods and conditional retention
| Feature | Free Free | Team $199/mo | Enterprise Custom |
|---|---|---|---|
| Full platform access | |||
| 100k logs retained 7 days | |||
| 1k scores | |||
| 5 datasets | |||
| 2 evaluators | |||
| 10 prompts | |||
| Everything in Free | |||
| 10k scores and 100k logs retained 30 days | |||
| Unlimited datasets, evaluators, and prompts | |||
| Private Slack channel and SOC 2 report | |||
| 5 team members included | |||
| 99.99% uptime SLA | |||
| Custom packages with volume discount | |||
| SAML SSO and advanced admin roles | |||
| Custom SLAs and dedicated support engineer | |||
| HIPAA BAA, PII masking, and data retention management | |||
| Custom retention periods and conditional retention |
Pricing extracted from the product website and may change. Check the source for current details.
Frequently asked questions about Respan
What is Respan?
Respan is an open-source AI gateway with built-in observability and evals. Teams can self-host the gateway for free or run it on Respan to route, trace, and evaluate every request across 1,000+ models. It is backed by Y Combinator (W24) and Gradient Ventures and reports processing 80+ trillion tokens.
Is Respan the same as Keywords AI?
Yes. Keywords AI rebranded to Respan in 2026. The company started at the University of Illinois and joined Y Combinator's Winter 2024 batch under the Keywords AI name.
Is Respan an AI gateway or an observability platform?
Both. The gateway routes requests across models and providers, observability traces what agents did in production, and evals score output quality. All of it reads from the same data, so a request routed through the gateway is automatically traced and can be evaluated without extra instrumentation.
Which models and providers does Respan support?
Respan's gateway provides a single OpenAI-compatible endpoint for 1,000+ models across every major provider, including OpenAI, Anthropic, Google Gemini, Google Vertex AI, Azure OpenAI, AWS Bedrock, Groq, Fireworks, Together AI, Perplexity, OpenRouter, Nebius, and Novita.
Is Respan SOC 2 compliant?
Yes. Respan is SOC 2 Type II certified. Data is encrypted in transit with TLS 1.2+ and at rest with AES-256, access follows least-privilege controls, and Respan runs regular penetration testing and security audits. SOC 2 reports are available on the Team plan and above.
How long does Respan retain logs and traces?
Retention depends on the plan: Free keeps logs and traces for 7 days, Team for 30 days, and Enterprise is custom. Team and Enterprise also support custom and conditional retention periods, and Enterprise adds retention management, PII masking, and the option to omit logs.

