AIOps

AIOps Self-Healing Loop

Known failures run an approved runbook and are verified; new ones get an AI analysis and an engineer's decision, then become rules.

Telemetry from the workloads, metrics, logs, traces, platform events and recent changes, is collected with one context. Anomalies against baselines and service levels are correlated into one incident and recorded in the ITSM tool. A failure that matches an approved rule runs its runbook, and health checks verify the result or roll it back. A new failure goes to an AI analysis agent that searches runbooks and past incidents; an on-call engineer approves or rejects the proposed fix, and approved fixes are codified as new rules.

signalsknownnew rulenewcause + fixapprovedticketalertAWS CloudNew failure · AI-assisted, an engineer decidesKnown failure · approved runbooksTelemetry · one context across signalsWorkloadsTools the team runsAmazon EKSapps · podsServersOTel collectorone contextMetrics+ AlertmanagerLogsAWS X-RaytracesEvent watchersK8s API eventsChangesdeploys · configDetectbaselines · SLOsCorrelateone incidentKnown failure?rule matchApproved runbookrestart · roll backVerifyhealth · rollbackLearnfix → ruleKnowledge baserunbooks · past incidentsAI analysis agentlikely cause + fixOn-call engineerapproves or rejectsITSMincident recordTeam chaton-call alertsGrafana dashboardsSLO dashboards

Point at a component to light its links; select it for what it does here and the practice that makes it right on AWS.

When to use it

  • The same incidents come back and are fixed by hand each time
  • Alerts are many, context is thin and root cause takes hours to find
  • Operations want automation, with engineers still deciding on anything new or risky

How NGT delivers it

Built from code, reviewed at a gate by an NGT architect, and handed over with runbooks your team can run.

AIOps