Problem
Reactive autoscaling for Amazon DocumentDB detects threshold breaches after the system is already under stress. With instance provisioning taking 3–5 minutes, users experience elevated latency during traffic spikes — the scaling action arrives when the damage is already done.
Solution: a predictive autoscaling system
The system scales resources before demand spikes occur, built on three key innovations:
- Leading Indicator Analysis — database connections (r = 0.73) and IOPS (r = 0.64) predict CPU load 5–15 minutes ahead.
- Trend Extrapolation — velocity and acceleration metrics estimate time-to-threshold using kinematic equations.
- Pattern-Based Preemptive Scaling — learned hourly and daily workload patterns trigger scaling before expected peaks.
Architecture
CloudWatch metrics feed a multi-metric collector; leading-indicator and trend analysis produce multi-horizon predictions, which a preemptive controller turns into DocumentDB scaling actions:
CloudWatch Metrics → Multi-Metric Collector → Leading Indicators + Trend Analysis → Multi-Horizon Prediction → Preemptive Controller → DocumentDB
Key technical contributions
- Multi-horizon prediction at 5, 10 and 15-minute intervals with exponentially decaying confidence.
- Time-to-threshold estimation using kinematic equations (
CPU(t+Δ) = CPU(t) + vΔ + ½aΔ²). - Lead signal classification — strong / moderate / stable / decrease, based on metric rate-of-change.
- Production implementation as AWS Lambda (Python 3.11) + DynamoDB + CloudWatch with 23 custom metrics.
Production results
| Metric | Reactive baseline | Predictive system |
|---|---|---|
| Scaling response latency | 5.2 min | 1.4 min (−73%) |
| SLA compliance | 97.3% | 99.9% |
| Monthly cost | $1,440 static / $890 reactive | $835 (−42% vs static) |
| Prediction accuracy (5 min) | — | 89.2% |
| Preemptive lead time | — | 8.3 min average |
Technology stack
AWS Lambda (Python 3.11) Amazon DocumentDB 5.0 CloudWatch DynamoDB Boto3
The full paper includes the mathematical model, algorithms, and a detailed production evaluation: download the full paper (PDF) or start with the two-page summary.