Reactive autoscaling has a structural limitation: it reacts. Before a metric crosses its threshold, before the alarm transitions to ALARM, before a new database instance is provisioned (3–5 minutes) — users have already been waiting. The scaling action arrives after the degradation has occurred.
In this system, the mean time from a traffic spike to an additional replica serving requests was 5.2 minutes. This article describes how that was reduced to 1.4 minutes — and why the answer was not faster provisioning, but prediction.
Observation: CPU is a lagging indicator
CPU is the conventional scaling signal, and it is the last metric to move. Before database CPU rises, something else happens first: connection count increases and IOPS increases. In production data the leading correlations were:
- database connections predict CPU load with a correlation of r = 0.73, 5–15 minutes ahead;
- IOPS — r = 0.64, over a comparable window.
That window is precisely the headroom provisioning requires. If the scaling decision is made on leading indicators, the 3–5 minutes of instance startup stops being a problem — capacity is ready before CPU reaches the threshold.
Three mechanisms instead of one threshold
1. Leading indicators
The system classifies the connection and IOPS signal as strong / moderate / stable / decrease based on rate of change. A strong leading signal can trigger scaling on its own, without waiting for CPU.
2. Trend extrapolation with kinematic equations
For CPU the system tracks not only the value but the velocity and acceleration of change. From CPU(t+Δ) = CPU(t) + vΔ + ½aΔ² it derives an estimated time-to-threshold. Predictions run at three horizons (5, 10, 15 minutes) with exponentially decaying confidence — the further the horizon, the less weight it carries in the decision.
3. Temporal patterns
Production traffic has hourly and daily rhythm. The system learns those profiles and scales ahead of an expected peak — the morning ramp, for instance — even when current metrics show nothing yet.
Results after deployment
| Metric | Reactive | Predictive |
|---|---|---|
| Scaling reaction time | 5.2 min | 1.4 min (−73%) |
| SLA compliance | 97.3% | 99.9% |
| Monthly cost | $1,440 (static) | $835 (−42%) |
| Prediction accuracy (5 min) | — | 89.2% |
A notable cost result: the predictive system proved cheaper not only than static provisioning but also than reactive autoscaling ($890/month), because it also scales down more intelligently — it does not hold spare capacity "just in case".
What this taught us
- Do not start with ML. Leading correlations and kinematic equations are secondary-school mathematics, and they delivered the 73% improvement. Models can be layered on later — which is what happened for the container tier, documented in a separate publication.
- Prediction confidence must be part of the decision. A prediction without an uncertainty measure is an invitation to oscillation.
- The whole system runs as an AWS Lambda function (Python 3.11) with DynamoDB and 23 custom CloudWatch metrics — no dedicated infrastructure.
The full system description — mathematical model, algorithms and production evaluation — is in the publication Predictive Autoscaling for Cloud Databases (PDF available for download).