Most enterprises discovered they had an AI problem the same way: someone asked how many models were running in production, and the answer took three weeks to assemble and was wrong.
Models arrived the way cloud did, from the edges, in a hurry, driven by teams with a business case and no appetite for a governance conversation. A support team wired an assistant into the ticketing system. An analytics group gave a model read access to the warehouse. Somebody connected an agent to an internal API because it was faster than filing a request.
None of that was reckless. All of it created connections that nothing was watching.
Table of contents
- What an AI gateway is
- How an AI gateway differs from an API gateway
- What runs through it
- Routing is not enforcement
- The two attackers on model traffic
- What to require of an AI gateway
- Frequently asked questions
Key takeaways
- An AI gateway is a control point every connection to and from a model passes through.
- It differs from an API gateway in what it inspects: intent and content rather than routes and rate limits.
- Visibility alone is insufficient, a gateway that logs but cannot block operates at the speed of whoever reads the log.
- Model traffic faces both autonomous AI attacks today and long-term exposure from harvested encrypted traffic.
- Enforcement, quantum-safe encryption, machine identity governance and audit evidence belong at the same point.
What an AI gateway is
An AI gateway is a control point that every connection between a model and everything else passes through.
That includes prompts going in, completions coming out, calls a model makes to internal APIs, data retrieved from vector stores and databases, tool invocations by agents, and traffic to external model providers. If a model is talking to something, or something is talking to a model, it goes through the gateway.
The reason this shape exists is that models broke the assumptions most enterprise cybersecurity was built on. Traditional controls assume a fixed set of applications with predictable behaviour and human users acting at human speed. A model is a non-deterministic system that can be persuaded through its input to do things nobody specified, acting through machine identities at machine speed, often with permissions granted quickly during a pilot and never revisited.
An AI gateway restores the property that was lost: a single place where what is happening can be seen and, more importantly, stopped.
How an AI gateway differs from an API gateway
The distinction matters because the two are frequently conflated during procurement, and the conflation leaves a gap.
An API gateway manages traffic. It routes requests, terminates TLS, enforces rate limits, checks API keys, and provides a consistent front door to a set of services. It answers questions about shape: is this request well-formed, is this caller authorised to reach this endpoint, are they exceeding their quota.
An AI gateway has to answer questions about substance: what is this request trying to make the model do, does this response contain something that should not leave, is this agent invoking a tool it has no business invoking, is this sequence of otherwise-permitted calls assembling into something that is not permitted.
An API gateway is content-agnostic by design. That neutrality is a virtue when moving structured requests between services. It is a limitation when the payload is natural language and the risk lives inside the meaning.
The OWASP Top 10 for Large Language Model Applications catalogues the categories involved, prompt injection, insecure output handling, excessive agency, sensitive information disclosure, and none of them are visible to a control that inspects routes rather than content.
An API gateway asks whether the request is well-formed. An AI gateway has to ask what the request is trying to do.
What runs through it
In practice an AI gateway sees several distinct traffic types, and each carries different risk.
Inbound prompts, including anything a user or upstream system supplies that becomes part of the model's context. This is where injection attempts arrive.
Outbound completions, which may contain data the model retrieved and should not disclose, or instructions that downstream systems will act on without validation.
Retrieval traffic, where models query vector stores, databases and document repositories. Retrieval permissions are frequently broader than anyone intended, because they were granted to make a prototype work.
Tool and function calls, where an agent invokes real capability, sending mail, writing to a system, making payments, provisioning resources. This is where excessive agency becomes an incident rather than a design flaw.
Provider traffic, where prompts and enterprise data leave for an external model API. Everything in that stream is data in transit, crossing networks the enterprise does not control.
Routing is not enforcement
A significant share of products described as an AI gateway are, functionally, observability with a proxy attached. They log prompts, record completions, produce dashboards and raise alerts.
Visibility is necessary. It is not sufficient, and the reason is the same arithmetic that undermines conventional security operations.
If the gateway logs a prompt injection and alerts a human, the response time is the time it takes a person to read, assess and act. That is tens of minutes on a good day. An autonomous attack chaining through model access to reach internal systems completes in seconds. The log is an accurate record of something that already finished.
Enforcement means the gateway itself decides and acts, blocking the request, refusing the tool call, stripping the disclosure, writing the rule that prevents the next attempt, without a queue in the path. That is a materially harder engineering problem than logging, which is why so many products stop short of it.
The NIST AI Risk Management Framework is useful for structuring which risks matter to an organisation. It is a framework for governing risk, not a mechanism for stopping an attack in progress, and organisations sometimes mistake having adopted one for having deployed the other.
The two attackers on model traffic
Model connections are exposed to both adversaries an enterprise currently faces, and an AI gateway that addresses only one leaves the other open.
Autonomous AI attacks are present tense. The cost of capability has collapsed, what required a skilled team now requires an open-weight model and an afternoon. Model endpoints are an attractive target precisely because they hold broad permissions and sit close to valuable data. These attacks complete faster than any human-mediated response.
Harvest now, decrypt later is the second. Prompts and completions travelling to external providers are data in transit, and adversaries are capturing encrypted traffic today to decrypt once quantum capability matures. Enterprise prompts routinely contain exactly the material with a long confidentiality life, customer records, contracts, source code, internal strategy. CNSA 2.0 mandates post-quantum compliance from 2027, with Canada, the EU, the UK and the UAE following.
An AI gateway that inspects content but leaves connections protected by classical cryptography has secured the meaning and exposed the transmission.
What to require of an AI gateway
Five requirements separate an enforcement point from a proxy with a dashboard.
- Does it block, or does it alert? If the response path terminates in a person, response is human-scale no matter how fast detection runs.
- Does it cover every model, including the ones you have not found yet? Coverage that depends on teams registering their models will always trail reality.
- Does it govern machine identities? Agents act through credentials. Permissions granted during a pilot and never reviewed are the most common excessive-agency finding.
- Does it apply quantum-safe encryption to traffic in transit? Content inspection and transport protection are different problems and both are in scope.
- Does it produce audit evidence automatically? Evidence assembled by hand before an assessment does not scale and does not survive scrutiny.
Conux answers those at one point. Five agents run the loop: one detects, one blocks and writes the rule, one hardens the surface, one logs the evidence, one orchestrates the rest. Detect. Block. Adapt. Prove.
Nothing is rebuilt. Existing cloud, identity, applications and models stay where they are, the cybersecurity gateway sits in the middle of what is already deployed. Weeks, not quarters. Invisible to users.
Related reading: AI infrastructure security, LLM security and why AI governance is not AI security.
The bottom line
An AI gateway exists because models created a class of connection that nothing was watching.
The useful version does more than watch. It decides, it blocks, it encrypts what it forwards, and it produces the record afterwards, because the adversaries on that traffic operate at machine speed today and are archiving what they cannot read yet.
Two attackers. One gateway.
