Amplitude’s Anomaly + Forecast feature takes a metric’s historical trend and projects it forward, flagging points where actual performance deviates significantly from what the model expected. Paired with the platform’s expanding predictive layer — which can also estimate which users are likely to convert or churn — it’s a genuine step up from staring at a line chart and eyeballing whether last week’s dip is normal. For product teams setting quarterly targets or deciding whether a metric drop needs an incident response, having a statistically grounded “expected range” instead of a gut-feel comparison to last month is valuable.
The risk with any forecast feature is that it’s silently only as good as the event data feeding it, and teams tend to trust a confident-looking projection more than they’d trust their own back-of-envelope estimate — even when the underlying data has gaps a human would have caught. A forecast built on three weeks of data that included a broken release, or a chart that mixes an old event schema with a new one after a migration, will still render a clean, confident-looking trend line. Nothing in the UI tells you the forecast quality is only as good as the noisiest week it learned from. Teams that don’t track what feeds the model risk making a real roadmap decision off a prediction the tool never actually had grounds to be confident about.
Data Points to Track
- Training window composition for each forecasted metric — which date range, and whether it spans any known bad-data periods (outages, broken releases, schema migrations)
- Anomaly flag frequency and magnitude over time, so you can tell a genuinely volatile metric from one that’s just noisy
- Forecast-vs-actual variance logged after the fact, to build your own track record of how reliable a given metric’s forecast has been
- Underlying event volume stability for the events that compose the forecasted metric, since a volume drop from an SDK issue looks identical to a real behavioural change
- Segment definition changes applied to a forecasted metric mid-window, which can shift a trend line without any real change in user behaviour
- Who acted on which forecast, so a decision made off a bad prediction can be traced back and the model’s blind spot documented
Setup Steps
- Tag known bad-data periods (incidents, migrations, broken releases) in a shared log before setting up any forecast, so you can exclude or annotate them.
- Export forecast-vs-actual comparisons on a recurring schedule rather than only checking the chart when a number looks alarming.
- Set a minimum event-volume threshold below which a metric shouldn’t be forecasted at all, since sparse data produces confident-looking but unreliable projections.
- Alert on underlying event volume drops separately from the anomaly flag itself, so a real behavioural anomaly isn’t confused with an instrumentation failure.
- Review forecast accuracy quarterly against actuals and adjust which metrics get forecasted based on that track record, not on how appealing the chart looks.
Actionable Insights
A forecast is a hypothesis about the future generated from a specific slice of the past — it’s only as trustworthy as that slice was clean. The most useful discipline here isn’t resisting the feature, it’s building a habit of checking forecast accuracy after the fact and keeping a visible log of when a projection was wrong and why. Over a few quarters that log becomes more valuable than any single forecast: it tells you exactly which metrics your data pipeline supports confident predictions for, and which ones still need a human sanity check before they drive a decision.
Related Resources
Need help tracking this in your app?
Our team sets up analytics pipelines for mobile and web teams every day. Talk to us and get your first events flowing in under an hour.
Talk to an expert