F5 AI Guardrails out-of-band
| AI inference networking | |
|---|---|
| Part 1 | Open WebUI as front door for ChatGPT |
| Part 2 | Inference security using Open WebUI |
| Part 3 | Layering NGINX for more Inference security |
| Part 4 | F5 AI Guardrails out-of-band |
Why put AI guardrails out-of-band?
I’m aiming for a model-agnostic approach in AI. Actually I’d like to be agnostic on almost everything. By running F5 AI Guardrails out-of-band, I plan to move some orchestration decisions back to my Open WebUI instance.
For example, I want to be able to do things like this:
- Multiple models
- I may switch models easily from Open WebUI without touching F5 AI Guardrails
- Perhaps if Guardrails flags a prompt or response, I’ll have options at Open WebUI. Examples:
- I could send a “blocked” prompt to an on-prem model
- I could try my hand at redacting offending text from a flagged response
- Policy enforcement centralized
- depending on which scanner blocks a prompt or a response, I may or may not honor the decision of Guardrails
- I could bypass the Guardrails checks selectively
- etc
Pre-requisites
- Follow blog posts 1, 2 & 3. By now you should be able to deploy and destroy our environment on demand.