Agentic Traffic Gateway

What it is

A self-contained demo of agentic traffic management built on agentgateway, the open source Rust data plane (a Linux Foundation project) for agent-to-LLM, agent-to-tool, and agent-to-agent communication. One gateway process fronts three kinds of traffic at once: an OpenAI-compatible chat completions route, a Model Context Protocol tool server multiplexed under a single /mcp endpoint, and two toy A2A agents (a researcher and a summarizer) exposed under path prefixes.

The workloads are deliberately small. Each A2A agent is a FastAPI service under 100 lines speaking A2A JSON-RPC and serving an agent card; the MCP server uses the official Python SDK's Streamable HTTP transport and exposes three deterministic tools (get_weather, lookup_docs, word_count). The point is not model quality. The point is that every request a client makes, whether it targets an LLM, a tool, or another agent, passes through one policy-enforcing, trace-emitting hop.

The demo ships in two flavors from one config model: a Docker Compose quickstart driven by a static YAML file, and Kubernetes manifests that use the Gateway API plus the project's AgentgatewayBackend and AgentgatewayPolicy CRDs. Both target agentgateway v1.4.1, with every config field checked against the published JSON schema and the upstream examples rather than invented.

Why it matters

Agentic systems multiply egress points. A typical agent loop calls an LLM provider, several tool servers, and increasingly other agents, each with its own credentials, timeouts, and failure modes. Without a gateway, every client framework re-implements credential handling, rate limiting, and observability slightly differently, and nobody can answer basic questions like how many tool calls per minute a given workflow makes.

Centralizing that traffic buys three concrete things. First, credential custody: the OpenAI key lives only in the gateway (an environment variable in compose, a Secret in Kubernetes), so clients and agent code never hold provider keys at all. Second, uniform policy: the demo applies local rate limiting (10 requests per second with a burst of 5) and retries (3 attempts, 250ms backoff, on 500/502/503) to the A2A routes declaratively, with no application code involved. Third, observability: the proxy emits OTLP traces for all three protocols to a single collector, so LLM calls, tool calls, and agent calls show up in one trace stream.

Architecture decisions

Static config versus CRDs. The compose path uses the file-based config so the whole stack runs with one command and no cluster; the k8s path follows the documented Gateway API route, because that is where the project invests its control plane. Keeping both honest forced the config shapes to come from the schema, not from memory.

Path-prefix routing with rewrite for A2A. A2A servers serve at root, so each agent gets a prefix (/agents/researcher) that the gateway strips before forwarding. This lets multiple agents share one listener without hostnames, and agent cards stay reachable through the gateway at /.well-known/agent-card.json under each prefix.

MCP multiplexing as the default. The tool server is declared as a target in an MCP backend rather than a plain HTTP route, so adding a second server later is a one-line change and tool namespacing is handled by the gateway.

Known limits, stated plainly: the agents return canned output, the rate limits are per-replica local limits rather than global, and the manifests were validated by parsing and schema comparison, not by a live cluster run, since no Docker daemon was available on the authoring machine.