Layering NGINX for more Inference security

AI inference networking
Part 1	Open WebUI as front door for ChatGPT
Part 2	Inference security using Open WebUI
Part 3	Layering NGINX for more Inference security
Part 4	F5 AI Guardrails out-of-band

Adding NGINX Ingress Controller for mTLS auth.

Why add NGINX to our architecture?

In the previous posts we covered Open WebUI and F5 AI Guardrails. Now we’ll add NGINX ingress controller so that we can do some mTLS auth. After that we’ll experiment with adding other functionality.

Pre-requisites

• Follow blog posts 1 & 2. By now you should be able to deploy and destroy our environment on demand.

Instructions to deploy NGINX Ingress Controller

1. Deploy Open WebUI + CalypsoAI integration on K8s following previous posts.
   • Instructions now include env var for Open WebUI to define a header for trusted auth: WEBUI_AUTH_TRUSTED_EMAIL_HEADER
2. I’ve cloned the repo for NGINX Ingress Controller at version 5.3.1
   • I have added the relevant manifests to the repo accompanying this series.
   • The instructions to deploy should include all manifests in the correct order.
3. Now we should have NGINX I.C. running. You should be able to access Open WebUI via public IP address, and your traffic should be traversing NGINX I.C.

Instructions to deploy cert-manager, server cert, and mTLS certs.

1. Install cert-manager via manifests:

     kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.19.2/cert-manager.yaml
   

2. Create a ClusterIssuer
3. Create a CACert
4. Create a (namespaced) Issuer using the secret from the CACert
5. Create a secret of type nginx.org/ca where the only data field is “ca.crt” and the value is the certificate material.

     # Extract the tls.crt from the ca-cert and create secret of type nginx.org/ca
     # this is so we have a secret in the correct format for NGINX's Policy CRD, which will implement mTLS.
     kubectl get secret open-webui-ca-cert -n open-webui -o json | jq '.type="nginx.org/ca" | .data={"ca.crt": .data."tls.crt"} | .metadata={"name": "open-webui-ca-cert-nginx", "namespace": "open-webui"}' | kubectl apply -f - 
   

6. Create a secret for TLS for NGINX, and multiple mTLS client certs
   • ensure your mTLS client certs include an email address. The modern way to do this is in the Subject Alternative Name (SAN) field. Cert-manager handles this well.

Instructions to configure NGINX Ingress Controller

1. Create a Policy CR that requires mTLS verification. My example.
2. Create a VirtualServer CR that implements TLS with our server cert and references our Policy impelemting mTLS. My example.

Now, test your access to Open WebUI

1. Export CA Cert into trusted root store of your VM (unless using public CA)
2. Export mTLS client cert/key into your VM
3. Access the website, using a DNS name that matches your TLS cert (required by Chrome to prompt for mTLS certs)

If you can login to Open WebUI by **only** providing a mTLS client cert, you have correctly configured NGINX to require and verify client certificates, to pass the value of the email on to Open WebUI via a HTTP Header, and Open WebUI to accept this header value as an authenticated user.

Conclusion

We’ve added mTLS authentication to our environment, by using NGINX Ingress Controller and Open WebUI trusted headers.