Amplitude’s Compass chart scans historical user data and ranks which early actions correlate most strongly with long-term retention — automating the kind of analysis that used to take an analyst days of manual cohort-slicing to find a product’s “aha moment.” Facebook’s classic finding that users who added seven friends in ten days almost always stuck around is the canonical example of what Compass is built to surface automatically. The appeal is obvious: point it at your event data and get a ranked list of behaviours worth building onboarding around, without a bespoke statistical analysis every time.
The catch is that Compass can only rank the events you’ve actually tracked. If a genuinely predictive action — say, connecting a specific integration, or completing a particular in-app configuration step — was never instrumented as its own event, Compass has no way to surface it; it will simply rank whatever coarser, already-tracked events come closest, and hand back a plausible-looking answer that misses the real signal entirely. A team that trusts the output without checking event coverage first risks building an entire onboarding strategy around a proxy metric instead of the actual driver.
Data Points to Track
- Granular action-level events for every meaningful early-lifecycle behaviour, not just screen views — a user “viewing” a feature and a user “completing” it are different signals, and Compass can only rank what’s split out
- Time-to-action properties, capturing how many days or sessions elapsed between signup and each candidate behaviour, since Compass’s predictive window depends on this being consistent and accurately timestamped
- Retention window definition consistency, logging which N-day or N-week retention definition is feeding the Compass analysis, so results aren’t silently compared across mismatched windows over time
- Cohort and platform metadata (acquisition channel, device type, app version) on every candidate event, so a Compass result can be checked for whether it holds across segments or is being driven by one dominant cohort
- Event volume per candidate action, flagging low-volume events that Compass may rank on statistically thin data, distinct from genuinely high-confidence predictors
Setup Steps
- Inventory every meaningful early-lifecycle action in your app — not just top-level screens — and confirm each fires as its own distinctly named event rather than being folded into a generic “screen viewed” call.
- Backfill or verify timestamp accuracy on early-lifecycle events, since Compass’s time-to-action analysis is only as reliable as the underlying event timing.
- Standardise your retention window definition (e.g. Day 7 or Week 4) across the events feeding Compass, and document it so future analyses stay comparable.
- Run Compass and cross-check the top-ranked behaviour against a manual segment split, confirming the correlation holds across at least two acquisition channels or device types before acting on it.
- Add any newly discovered predictive event to onboarding instrumentation as a named milestone, so its impact can be tracked directly once product changes start nudging users toward it.
Actionable Insights
Compass is a hypothesis generator, not a verdict — its output is only as trustworthy as the event coverage underneath it. Run it after an honest instrumentation audit, not before, and validate any surfaced predictor against a manual cohort check before redesigning onboarding around it. Teams that treat Compass results as ground truth without checking for undertracked candidate events risk optimising for a coincidental correlation instead of the behaviour actually driving retention.
Related Resources
Need help tracking this in your app?
Our team sets up analytics pipelines for mobile and web teams every day. Talk to us and get your first events flowing in under an hour.
Talk to an expert