AI Security Guard with Envoy ext_authz + MobileBERT

Envoy AI Security Guard is a small, self-contained lab that puts a machine-learning security decision directly in the request path — using Envoy Proxy’s native external authorization (ext_authz) filter to inspect every request with a MobileBERT SQL-injection classifier before it can reach your application, and blocking anything it judges unsafe at the edge. If you’ve ever wondered how to add an AI-driven guardrail to an API without rewriting the API, this lab is a working, end-to-end answer you can run on Docker Desktop in a couple of minutes.

The problem: security checks live inside every app

The usual way to stop a malicious request is to validate it inside the application — a filter in each service, in each language, maintained by each team. That works, but it scatters enforcement across your whole estate and raises an uncomfortable question: what happens to the one endpoint whose developer forgot to add the check?

Moving the decision to the proxy turns that scattered logic into a single, consistent control point. Every routed request follows the same inspection path, the guard can look at the URL, headers, query, and body, and a denied request never touches the backend. Just as importantly, the detection engine can evolve — from rules, to a model, to a layered risk score — without changing a line of application code.

Envoy supports external authorization: the filter calls an external gRPC or HTTP service to check whether the request is authorized or not. If the check fails, the request is denied.

— Envoy ext_authz filter documentation

This is not a replacement for secure coding, authentication, or authorization. It’s an additional layer — one that’s especially useful when the threat changes faster than the application can be redeployed.

Why I built it

AI for security is easy to talk about and harder to wire into something real. I wanted a lab where a trained model makes an actual allow-or-deny call on live traffic, not a notebook that scores strings in isolation. The interesting part isn’t the classifier — it’s the architecture around it: a clean boundary where the proxy owns enforcement and a separate service owns analysis.

Starting with a published SQL-injection model let me validate the whole control plane end to end, then reason about the questions that actually matter in production — latency, availability, false positives, and evasion — with a working system in front of me instead of a slide. It sits alongside my other hands-on security labs, like SPIFFE & SPIRE on your laptop for workload identity.

What it does

The lab runs three containers wired together with Docker Compose. Clients connect to Envoy, never directly to the backend. The architecture diagram above shows the full flow; here are the moving parts:

Component	Port	Role
Envoy Proxy	8080 / 8443	Terminates TLS, enforces the authorization decision, routes allowed traffic
AI Security Guard	5000	Loads MobileBERT + prompt rules, returns 200 allow or 403 deny
Web application	8000	A simple backend that only ever sees requests the guard approved

For every request, Envoy runs this sequence via ext_authz:

Step	What happens
1	Client sends an HTTP or TLS request to Envoy
2	Envoy calls the guard with the path, approved headers, and up to 64 KB of body — waiting max 2s
3	Guard scores the input and returns 200 OK (allow) or 403 Forbidden (deny)
4	Allow → Envoy forwards to the app · Deny → 403 returned · Guard unavailable → request blocked

That last row is the important one. The lab sets failure_mode_allow: false, so if the guard can’t return a decision, Envoy fails closed rather than open:

http_filters:
  - name: envoy.filters.http.ext_authz
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
      http_service:
        server_uri:
          cluster: ai_model_service
          timeout: 2.000s
          uri: http://ai_model_service:5000/check
      with_request_body:
        max_request_bytes: 65536
        pack_as_bytes: false
      failure_mode_allow: false
  - name: envoy.filters.http.router
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

Fail-closed is the safer default for an experiment, but it’s also an availability decision — a real deployment has to decide which routes demand strict enforcement and what latency budget is acceptable.

Quick start

The whole environment comes up with one command — the guard image downloads the model at build time, so the first request isn’t blocked on a model pull.

# Clone the lab
git clone https://github.com/jeevanbisht/AISecurityGuard.git
cd AISecurityGuard

# Build and start Envoy, the AI guard, and the backend
docker compose up --build -d
docker compose ps

Send a normal query through the TLS listener — expect 200 OK and the backend response:

curl -k "https://localhost:8443/search?q=laptops"

Now send an obvious SQL-injection string — expect 403 Forbidden, and the backend never processes it:

curl -k "https://localhost:8443/search?q=SELECT%20*%20FROM%20users;--"

A prompt-injection attempt takes the same path over the HTTP listener:

curl "http://localhost:8080/search?q=ignore%20previous%20instructions"

The lab also ships an automated check that verifies one allowed request and two blocked requests:

python test_lab.py

Requirements: Docker Desktop and, for the test script, Python 3. Nothing else — no cloud account, no API keys.

How detection works

The guard is a hybrid: a trained model for SQL injection, deterministic patterns for prompt injection. Each request is URL-decoded and whitespace-normalized, then passed through both layers.

Layer	Engine	What it catches
SQL injection	MobileBERT sequence classifier — blocks when the injection-class probability ≥ threshold	UNION SELECT, boolean tautologies, stacked queries, obfuscated variants
Prompt injection / unsafe payloads	Regex signatures	ignore previous instructions, system prompt, <script>, /etc/passwd

The SQL-injection decision is genuine model inference — PyTorch + Hugging Face Transformers running MobileBERT:

def predict_sql_injection(content: str) -> float:
    tokenizer, model = load_model()
    inputs = tokenizer(content, truncation=True,
                       return_tensors="pt", max_length=512)

    with torch.inference_mode():
        logits = model(**inputs).logits

    probabilities = torch.softmax(logits, dim=-1)
    return probabilities[0, SQL_INJECTION_CLASS_ID].item()

A blocked request comes back with a machine-readable reason header and the model’s confidence — useful in development, though in production I’d keep the detailed reason in protected telemetry and return something terse to an untrusted client:

HTTP/1.1 403 Forbidden
x-ext-auth-reason: unsafe-query-blocked
content-type: application/json

{ "status": "blocked",
  "reason": "SQL injection detected by MobileBERT (confidence: 0.997)" }

Two environment variables tune the behavior without touching code:

Variable	Default	Purpose
SQL_INJECTION_MODEL_ID	cssupport/mobilebert-sql-injection-detect	Swap in any compatible sequence-classification model
SQL_INJECTION_THRESHOLD	0.70	Raise for fewer false positives, lower to be more aggressive

Scope and limitations

This is a lab, and it’s deliberately narrow. It’s a foundation to build on, not a drop-in WAF:

• The model classifies SQL injection only — prompt-injection coverage is a handful of illustrative regexes, not a trained detector.
• The guard is now on the latency and availability path. With fail-closed enabled, a slow or down guard means blocked traffic.
• Regex prompt rules are trivially evaded; real coverage needs normalization against encoding, fragmentation, and alternate syntax.
• No identity, rate limiting, or per-route policy yet — every route is treated the same.

None of that undercuts the point. The lab proves you can insert model-backed inspection in front of an application without modifying it, with a clean split between the enforcement point (Envoy) and the decision engine (the guard).

From a classifier to layered AI security

The natural next step isn’t “replace every rule with an LLM” — it’s a layered decision:

1. Normalize first so equivalent payloads get equivalent analysis.
2. Keep high-confidence rules — fast, cheap, explainable.
3. Use specialized classifiers — MobileBERT scores SQL injection today; other models can own other attack classes.
4. Extract structured signals — route, method, content type, identity, payload size, history — not just raw text.
5. Apply policy to model scores — a confidence value is evidence for a decision, not the entire policy.
6. Close the loop — store reviewed false positives and negatives for evaluation.

Envoy still enforces a binary allow-or-deny, while a richer decision object supports logging, tuning, and future response actions:

{
  "decision": "deny",
  "risk_score": 0.97,
  "category": "sql_injection",
  "signals": ["union_select", "stacked_query"],
  "policy": "high-risk-input-v1"
}

That’s the real value of putting AI at a proper enforcement point: the proxy doesn’t need to understand MobileBERT, tokenization, or confidence scores — it just needs a fast, trustworthy answer, and the application receives only traffic that policy allowed.

FAQ

How do you add AI security to an API without changing the app? Put the decision at the proxy. Envoy’s ext_authz filter calls an external guard for every request and only forwards it on an allow, so the application itself needs no security-specific code.

What does the MobileBERT model actually detect? It classifies SQL injection and returns a probability. The guard blocks the request when that probability crosses a configurable threshold (default 0.70). Prompt injection and other unsafe payloads are handled by separate deterministic rules.

What happens if the security guard is unavailable? Envoy is configured to fail closed (failure_mode_allow: false), so if the guard can’t return a decision within the timeout, the request is blocked rather than allowed through.

Is this a replacement for a Web Application Firewall (WAF)? No — it’s a hands-on lab and architectural foundation. It shows how to run model-backed inspection in front of an app; production use needs broader attack coverage, latency and availability engineering, and threshold tuning.

---

Source: github.com/jeevanbisht/AISecurityGuard — Envoy ext_authz in front of a Flask guard running cssupport/mobilebert-sql-injection-detect on PyTorch + Transformers, packaged with Docker Compose and fail-closed by default.