ai-gateway - AI Developer Tools Tool

Overview

ai-gateway (LangDB AI Gateway) is an open-source, Rust‑built API gateway that centralizes and governs requests to multiple LLM providers using an OpenAI‑compatible API surface. It exposes standard endpoints (chat completions, embeddings, model listing, image generation) so existing OpenAI‑style clients and frameworks can switch providers with minimal code changes. The gateway is designed for production use: it focuses on high throughput, low latency and predictable behavior via a compiled Rust implementation and supports provider failover, model virtualization and advanced routing. ([docs.rs](https://docs.rs/crate/ai-gateway/latest?utm_source=openai)) Operational features include built‑in cost and rate controls, configurable guardrails for content validation, and rich observability — OpenTelemetry‑compatible tracing with span-level detail and metrics export for monitoring and billing attribution. The project provides both a self‑hosted OSS gateway and a managed/enterprise offering (LangDB) for teams that prefer a hosted control plane. Roadmap items and docs call out prompt caching, more guardrail tooling, and broader MCP/Model Context Protocol support for tooling and external MCP servers. ([docs.langdb.ai](https://docs.langdb.ai/?utm_source=openai))

Installation

Install via docker:

curl -sL https://raw.githubusercontent.com/langdb/ai-gateway/main/config.sample.yaml -o config.sample.yaml

ai-gateway serve --host 0.0.0.0 --port 3000 --config config.yaml

Key Features

• OpenAI‑compatible API endpoints: /v1/chat/completions, /v1/embeddings, /v1/models, /v1/images/generations.
• Multi‑provider routing and failover: OpenAI, Google Gemini, Anthropic, AWS Bedrock, Mistral, Cohere, and others.
• Cost control and rate limiting: hourly/daily/monthly limits and spend-based controls for teams and applications.
• Observability & tracing: OpenTelemetry/OpenInference‑compatible spans, detailed request traces, and metrics export.
• Guardrails & content filtering: configurable input/output validators via LangDB Guardrails library to enforce safety.

Community

LangDB’s AI Gateway is actively developed and documented: the project and companion crates (e.g., langdb_guardrails, ai-gateway crate) have regular releases and changelogs (see crate/release listings). The project maintains public docs and self‑hosted guides, a samples repository and client SDKs (e.g., pylangdb) for integration. Community touchpoints include the GitHub repo, issues/discussions, and the LangDB documentation site; the company also publishes a managed/enterprise product and roadmap details on its site. For implementation details, configuration examples and recent release notes see the project docs and crate pages. ([lib.rs](https://lib.rs/crates/ai-gateway?utm_source=openai))

Key Information

• Category: Developer Tools
• Type: AI Developer Tools Tool