Predictive ML-Based Autoscaling for Containerized Microservices (ECS Fargate)

Problem

AWS ECS threshold-based autoscaling operates reactively, introducing a 3–7 minute lag between load increase and new capacity. For containerized microservices this means service degradation during every traffic spike.

Solution: a 6-component ML pipeline

1. Agreement-Based Ensemble Predictor — adaptive exponential smoothing (CV-driven α ∈ [0.1, 0.6]) combined with weighted moving average; divergence-based estimator selection.
2. Linear Trend Extrapolator — EMA-smoothed velocity eliminates the parabolic overshoot of kinematic models.
3. Multi-Signal Leading Indicators — online-learned request-to-CPU elasticity (E = Cov/Var) for ALB; correlation-learned weights for Micrometer JVM metrics.
4. Percentile-Based Burst Detector — P95 + median deviation with a capped 1.5× scale factor.
5. Pattern Learner — hourly/daily temporal load profiles with preemptive scaling triggers.
6. Confidence-Aware Hysteresis Controller — prediction confidence modulates thresholds; provable oscillation bounds.

Key innovations

• Minimum baseline stability guards replacing naïve ε = 10⁻¹⁰ denominators — eliminates false signals at low utilization.
• Online correlation learning for both ALB elasticity and Micrometer signal weights.
• Formal safety guarantees — proven bounded scaling invariant (N_min ≤ N* ≤ N_max) for all execution paths.
• Oscillation bound theorem — at most ⌊T/(K_consec+1)⌋ oscillations in T evaluation periods.
• 9-level decision hierarchy — burst → queue overload → leading indicators → prediction → trend → pattern → reactive → scale-down → no change.

Production results (6 ECS services, 2 environments)

Technology stack

AWS Lambda (Python 3.11) ECS Fargate CloudWatch DynamoDB EventBridge ALB Micrometer

The full paper contains mathematical proofs, theorems, algorithms and complexity analysis: download the full paper (PDF) or the summary.