Docs

Tracking AppsFlyer's Virtual Analyst Queries

AppsFlyer's Virtual Analyst lets marketers query attribution data in plain English — track what it's asked and answered before it drives spend.

Acquisition

AppsFlyer has rolled the Virtual Analyst out inside its AI Assistant, letting marketers ask attribution questions in plain English — “which campaign drove the cheapest paying user last week” — and get an instant answer instead of waiting on an analyst to build a report. That’s a real speed win, but it changes who’s touching attribution data and how. A natural-language layer sits between the raw event stream and the person making a budget call, and unlike a saved dashboard, its output isn’t the same query run the same way every time. Two people asking a similar question with slightly different wording can get answers built on different filters, date ranges, or attribution models without realising it.

The failure mode isn’t the tool being wrong — it’s the tool being unaudited. If a channel gets paused or a budget gets reallocated off a Virtual Analyst answer, and nobody logged the underlying query, you’ve lost the ability to reproduce or challenge that decision later. Worse, if the assistant silently defaults to last-click attribution when a team’s standard is data-driven or position-based, every AI-generated answer quietly disagrees with the dashboards everyone else is using, and nobody notices until the numbers are compared side by side.

Data Points to Track

  • Query volume and query text submitted to the Virtual Analyst, logged per user, so any spend decision traced back to an AI answer can be reproduced
  • Attribution model applied per query, confirmed against your team’s standard model rather than assumed to match
  • Date range and filter defaults the assistant applies when a query doesn’t specify them explicitly
  • Answer-to-action rate — how often a Virtual Analyst response is followed by a budget change, campaign pause, or creative swap within the same session
  • Discrepancy rate between Virtual Analyst answers and the equivalent saved dashboard report, sampled weekly during rollout
  • User role and access level for each query, since natural-language access can surface data outside a marketer’s usual reporting scope

Setup Steps

  1. Enable query logging for the Virtual Analyst before rolling it out beyond a pilot group, so every question and answer pair is retained.
  2. Run a parity check by asking the assistant the same questions your standard dashboards already answer, and compare outputs for attribution model and date range differences.
  3. Document the assistant’s default filters — attribution window, model, currency — and flag anywhere they diverge from your team’s configured standard.
  4. Tag any spend or campaign change made off a Virtual Analyst answer with the query that prompted it, so the decision has a traceable source.
  5. Set a recurring discrepancy review comparing a sample of AI answers against dashboard totals for the same period, at least until the tool has a full quarter of track record.

Actionable Insights

Once query logging is in place, the discrepancy rate becomes the tool you actually manage the rollout against: a low, stable rate means the Virtual Analyst is safe to lean on for faster day-to-day decisions, while a rising or inconsistent rate points to a default (usually attribution model or lookback window) that needs to be pinned centrally rather than left to the assistant’s judgement per query. The answer-to-action rate is the other number worth watching closely — it tells you how much real budget is now moving off natural-language queries instead of reviewed dashboards, which is exactly the population you want covered by the parity checks first.

Expert help

Need help tracking this in your app?

Our team sets up analytics pipelines for mobile and web teams every day. Talk to us and get your first events flowing in under an hour.

Talk to an expert