A typical operational investigation looks like this: one tab for the container orchestrator, one for metrics, one for cost analysis, a terminal running log queries, and a local checkout of the infrastructure code. Multiple environments, two regions. The question "which services are over-provisioned and how much could we save?" costs two to four hours of manual work.
I built an AI agent that answers questions like that in two minutes. This article is not about LLMs being capable — it is about what actually determined whether the operations team was willing to use one.
Trust is designed before intelligence
The first version of the agent had no write permissions at all. Not because write access was infeasible, but because read-only was the price of entry. The principles adopted from day one:
- Read-only by default — every write operation is blocked until explicitly enabled.
- Approval gates — applying infrastructure changes always requires human confirmation. The agent proposes a diff; a person decides.
- Secret masking — credentials are redacted before anything reaches the model.
- Full audit trail — every tool call and every decision is written to DynamoDB. After an incident the entire session can be reconstructed.
- Least-privilege IAM and in-region processing — data does not leave the account or region.
The primary finding: adopting an AI agent in operations is 20% a model problem and 80% a trust problem. Security is not a feature here — it is a precondition.
Tools, not prompts
The agent runs a tool-use loop (up to 30 iterations per query) against a managed in-account model. The tools cover cost optimisation, reliability monitoring, infrastructure drift, log analysis, incident investigation, dependency blast radius, network and queue visibility, and reporting. Representative production queries:
- "Reconstruct the incident from the last 24 hours" — 3 minutes instead of 30–90.
- "Check drift across all services in dev" — 45 seconds instead of 1–2 hours.
- "Generate an executive health report" — 1 minute instead of 1–2 hours.
The tool count has since more than doubled; the current scope is documented in the Infrastructure Intelligence Agent publication.
An agent that improves without fine-tuning
The most interesting part of the system turned out to be the learning mechanism — with no changes to model weights:
- Baseline tracker records typical CPU, memory and cost values per service, so the agent notices deviations by itself ("this service normally sits at 30% CPU; it has been at 70% since this morning").
- Playbook store saves successful multi-step investigations (three or more tools) as reusable templates, so a similar question resolves faster next time.
- Feedback collector learns from accepted and rejected recommendations.
What it costs
The whole system — compute, storage and model calls — costs $7–23 per month, depending on usage intensity. That is less than an hour of engineering time, against a system that saves dozens of such hours monthly (a 95–99% reduction in investigation time across the measured scenarios). Errors in production: zero.
How to start on your own infrastructure
- Start with read-only plus audit — it builds trust and risks nothing.
- Pick three to five questions the team asks every week and build tools for those; do not attempt full coverage first.
- Log everything from day one — the audit trail is also the training data for playbooks.
The full architecture — learning system, security model and production evaluation — is described in the Infrastructure Intelligence Agent publication (PDF available for download).