How to Lock Down a Self-Hosted LLM Gateway

Self-hosted LLM gateways such as LiteLLM are usually deployed by an engineering team in an afternoon and then forgotten by everyone except the applications calling them. The September 2026 addition of CVE-2026-59822 to CISA's Known Exploited Vulnerabilities catalog, backed by Wiz honeypotHoneypot🛡️A decoy system deployed to be attacked so defenders can observe exploitation attempts safely. Honeypot networks give early warning that a vulnerability has moved from theoretical to actively exploited, often before official catalogs like CISA KEV confirm it. telemetry showing attackers probing the proxy with one-character bearer tokens and extracting provider keys from process memory, is a reasonable trigger for a deliberate hardening pass. This guide is that pass, in the order that gives the most risk reduction per hour. It is written around LiteLLM because that is the product under attack, but every step applies to any gateway that holds provider keys and exposes tool routes.

Step 1: Find Every Instance

Gateways spread. The production one is in Kubernetes; a second is on a developer's workstation with the same provider keys; a third runs in a CI pipeline for evaluation jobs. Each one holds credentials, and the honeypot data shows attackers scanning for the product rather than for a specific organization.

Search container registries for the gateway image, search infrastructure code for the chart or compose file, and search secrets stores for the environment variable that holds the master key. Record the version of each instance. For LiteLLM the security floor as of September 2026 is 1.84.0: it closes the MCP authentication bypassAuthentication Bypass📖A security vulnerabilityVulnerability🛡️A weakness in software, hardware, or processes that can be exploited by attackers to gain unauthorized access or cause harm. that allows an attacker to circumvent the login verification process and gain unauthorized access to a system without providing valid credentials., and it comes after 1.83.7, which closed the MCP test-endpoint command injectionCommand Injection🛡️A security vulnerability that allows attackers to execute arbitrary operating system commands on the host system through a vulnerable application., and 1.83.0, which closed the admin configuration endpoint. An instance that was upgraded once in the spring can still be on a vulnerable build.

Step 2: Decide What Network Can Reach It

The single most effective control is placement. A gateway that only application servers call has no business being reachable from the internet, and in most deployments it should not be reachable from the general corporate network either.

Put it on a private subnet or service mesh, and allow inbound connections only from the application tier and from an administrative bastion. If a remote application genuinely needs it, front it with an authenticating reverse proxy or a mutual-TLS listener rather than opening the gateway's own port. The GitHub advisory for the LiteLLM bypass gave a specific interim workaround for teams that could not patchPatch🛡️A software update that fixes security vulnerabilities, bugs, or adds improvements to an existing program. immediately: block `/mcp/` and the related MCP routes at the reverse proxy. That is a good permanent rule as well. Expose only the paths the applications use, and deny the rest by default.

Step 3: Disable Routes You Do Not Use

Gateways accumulate features. The MCP gateway, admin UI, configuration update endpoint, pass-through routes to upstream providers, and test endpoints each add attack surfaceAttack Surface🛡️The sum of all points where an unauthorized user could attempt to enter or extract data from a system: exposed services, interfaces, accounts, and integrations. Reducing attack surface means removing reachability, not just patching., and each of the LiteLLM CVEs this year lived in one of them. If the organization does not use MCP through the gateway, do not run the MCP routes. If configuration is managed in code, block the configuration update endpoint at the proxy so that a compromised low-privilege key cannot rewrite the proxy's environment. The 1.84.0 release also enabled authentication by default on pass-through routes and restricted UI asset endpoints to authenticated users, so the project itself is moving in this direction.

The mechanics of how the MCP route concentrates capability are covered in How MCP Gateways Work and Why They Concentrate Risk. Every enabled route is a place where an authentication bug becomes access to everything behind the gateway.

Step 4: Fix Key Hygiene

Three separate credential classes need attention.

The master key mints virtual keys and changes configuration. It should live in a secrets manager, be injected at runtime, and be known to as few people as possible. If it has ever been in a compose file in a repository, rotate it now. Wiz's honeypots recorded attackers testing the documentation example key `sk-1234` against the completions endpoint, which only works if someone left the example in place.

Virtual keys are what applications hold. Issue one per application, set a budget and a model allowlist on each, and set an expiry. A virtual key that can call every model with no spending cap is a provider key with extra steps.

Upstream provider keys are what the gateway holds. Give the gateway its own provider keys rather than reusing ones that other systems also hold, so that a gateway compromise has a bounded rotation. Set spending limits on the provider side too; the honeypot attackers were monetizing access, and a cap turns a stolen key into a small bill instead of a large one.

Step 5: Scope Tool Credentials Individually

If the MCP gateway stays enabled, every upstream MCP server it connects to should have its own credential, scoped to the minimum that tool needs. A tool that reads a knowledge base should hold a read-only token. A tool that files tickets should be able to create tickets in one project and nothing else. Never give a tool the same cloud IAM role as the gateway container.

Prefer the delegated authentication mode, where the gateway validates its own key before forwarding an upstream token, over transparent passthrough, which the LiteLLM documentation describes as forwarding the client's Authorization header verbatim with no gateway admission. Transparent passthrough has legitimate uses, but the CVE-2026-59822 bug lived precisely in the code that decided which mode a request was in.

Step 6: Control Egress

The honeypot attackers fetched payloads from external hosts, connected miners to a Monero pool, and used DNS callbacks to an out-of-band domain to confirm that blind prompt injections had executed. All three depend on the gateway being able to reach arbitrary internet destinations.

A gateway needs to reach its configured model providers and its configured MCP servers. Enumerate those destinations and allow only them, by hostname through an egress proxy or by address at the network layer. Send DNS through an internal resolver that logs queries. This one control breaks the miner download, the pool connection and the callback channel at once, and it converts the attacker's success signal into your detection signal.

Step 7: Log Tool Calls and Watch Process Ancestry

Log every request to the gateway with the caller identity, the model or tool invoked, and the outcome, and ship the logs off the host. An authentication bypass that grants anonymous sessions shows up in such logs as tool invocations with an empty or malformed identity, if anyone is looking.

On the host, alert on the gateway process spawning a shell, Python interpreter or curl. The RCE chain that Wiz documented executed as a child of the proxy and then deleted its staging directory, so the process tree is the durable evidence. Watch for hidden directories under temp paths and for detached processes whose executable no longer exists on disk.

Step 8: Verify Images and Pin Versions

The March 2026 TeamPCP incident put trojanized LiteLLM releases on PyPI for about three hours. Install from a pinned version, verify the container signature, and mirror the package through an internal registry with a scanning step. The 1.84.0 release notes include a Cosign command for verifying the published image against the project's public key. Build the verification into the deployment pipeline so that it runs every time rather than once.

Step 9: Schedule the Next Pass

Hardening decays. Put the gateway on the same patch calendar as the VPN concentrator and the identity provider, subscribe to the project's security advisories, and re-run this checklist whenever a KEV entry names a component in the stack. The planning side of that discipline, including who owns the gateway and what the patch SLA should be, is in Treating AI Infrastructure as Privileged Infrastructure.