Product & Strategy ps-4 25 min

AI Impact Tracking Worksheet

Learning Objectives

  • apply the impact tracking worksheet to one active AI initiative
  • construct a complete worksheet entry with baseline metrics and ROI estimates
  • adapt the worksheet format for a small experiment or a multi-team initiative
  • complete a 90-day review cycle using the worksheet as the source of truth

Core Concepts

Baseline metric A quantified measurement of how the process works today, captured before AI is introduced. Without a baseline, ROI is unverifiable. A baseline is not an estimate: it is a recorded observation from a defined time window.

At X-company, the baseline for the PRD drafting initiative is an 8-day average cycle time, measured across all PRDs closed in the six months prior to rollout.

Leading indicator A metric that changes quickly after an initiative launches and predicts whether lagging outcomes will follow. Leading indicators tell you whether adoption is happening; they do not yet confirm business impact.

X-company tracks two leading indicators: the percentage of PRDs started from an AI brief template (target: 70% by day 30) and PM satisfaction scores on draft quality (target: 3.5/5). Both are measurable within the first month.

Lagging indicator A metric that reflects actual business impact but takes longer to move. Lagging indicators confirm whether the leading indicators translated into real outcomes.

X-company's lagging indicators are average PRD cycle time (target: 2.5 days by month 3, down from 8 days) and the number of revision cycles per PRD (target: below 2). Neither is measurable until enough PRDs have completed the full process.

ROI dimension A specific category of return: cost savings, time recovered, quality improvement, or capability expansion. Each dimension needs its own calculation method. Combining them into a single number before validating each one separately produces an unreliable figure.

X-company separates two ROI dimensions. Cost dimension: at 80% adoption, senior PM time saved per PRD multiplied across the team produces $17,280 in recovered capacity per month. Time dimension: 6 days saved per PRD enables 2 additional experiments per quarter, which compounds on velocity independent of cost.

Review cadence A scheduled rhythm for checking whether the initiative is on track. Leading indicators are reviewed monthly (fast feedback, early correction). Lagging indicators and ROI are reviewed quarterly (enough data to be meaningful).

Key Points

  • A baseline must be recorded before launch; it cannot be reconstructed from memory
  • Leading indicators measure adoption and early signal; lagging indicators measure business impact
  • ROI has multiple dimensions: calculate each one separately before combining
  • A review cadence is a commitment, not an intention: schedule it before the initiative launches
  • The worksheet is the source of truth at every review meeting

Tools, Prompts, or Templates

Without a structured template, tracking tends to degrade into a spreadsheet that no one updates. The worksheet below prevents that by giving every initiative a single, consistent record that travels through its full lifecycle: from baseline capture to quarterly ROI review.

Fill in every field before launch. Leave nothing as TBD at the moment you start: if you cannot answer a field, that is a signal to delay the rollout until you can.


AI Impact Tracking Worksheet

Field Your Entry
Initiative name
Initiative owner
Launch date
Business goal
Baseline metric Value + measurement method + time window
Leading indicator 1 Metric name, target value, target date
Leading indicator 2 Metric name, target value, target date
Lagging indicator 1 Metric name, target value, review date
Lagging indicator 2 Metric name, target value, review date
ROI: cost dimension Calculation method + projected monthly value
ROI: time dimension Hours or days recovered + downstream effect
Adoption target % of eligible users at 30 / 60 / 90 days
Monthly check date Scheduled date for leading indicator review
Quarterly review date Scheduled date for lagging + ROI review
Status at 30 days On track / at risk / blocked + one line
Status at 90 days Actual vs. target for each indicator + ROI actuals

X-company: Completed Worksheet (PRD Drafting Initiative)

Field Entry
Initiative name AI-assisted PRD drafting
Initiative owner Head of Product
Launch date Day 0 of Q3
Business goal Reduce time-to-feature-spec to increase experiment velocity
Baseline metric 8-day average PRD cycle time, measured over 6 months of closed PRDs
Leading indicator 1 % of PRDs started from AI brief template; target 70% by day 30
Leading indicator 2 PM satisfaction score on draft quality (1–5); target 3.5 by day 30
Lagging indicator 1 Average PRD cycle time; target 2.5 days by month 3
Lagging indicator 2 PRD revision cycles per document; target below 2 by month 3
ROI: cost dimension (Senior PM hourly rate) × (hours saved per PRD) × (PRDs/month) × 0.80 adoption = $17,280/month
ROI: time dimension 6 days saved per PRD × PRD volume = 2 additional experiments per quarter
Adoption target 50% at 30 days, 70% at 60 days, 85% at 90 days
Monthly check date First Monday of each month, 30-minute PM sync
Quarterly review date End of Q3, leadership review with Head of Product + CTO
Status at 30 days (to be filled at review)
Status at 90 days (to be filled at review)

Prompt: Generate Baseline Metrics for a New Initiative

Use this prompt when you are about to launch an initiative and need to identify what to measure. Run it before filling in the worksheet.

I am launching an AI initiative at [company name].

Context:
- Company: [size, industry, stage]
- Initiative: [what AI is being used for]
- Team affected: [which team, how many people]
- Business goal: [what outcome we are trying to achieve]

Help me identify:
1. The single most important baseline metric I should capture before launch, with the measurement method and a realistic time window for the historical sample
2. Two leading indicators I can track in the first 30 days to detect whether adoption is happening
3. Two lagging indicators that will confirm business impact by month 3
4. The most defensible ROI calculation for this type of initiative, with the formula written out

Be specific. Avoid generic suggestions like "track satisfaction." Give me metrics I can pull from systems we already have.

Prompt: 90-Day Review Preparation

Use this prompt in the week before your quarterly review. It turns the worksheet data into a structured review narrative.

I am preparing for a 90-day review of our AI initiative. Here is our tracking data:

Initiative: [name]
Business goal: [goal]
Baseline: [value]

Leading indicators at 30 days:
- [Indicator 1]: target [X], actual [Y]
- [Indicator 2]: target [X], actual [Y]

Lagging indicators at 90 days:
- [Indicator 1]: target [X], actual [Y]
- [Indicator 2]: target [X], actual [Y]

ROI actuals:
- Cost dimension: projected [X], actual [Y]
- Time dimension: projected [X], actual [Y]

Help me:
1. Write a two-paragraph executive summary of results for a leadership audience
2. Identify the two most important gaps between target and actual, with a likely root cause for each
3. Recommend whether to scale, adjust, or pause the initiative, with one sentence of reasoning
4. Suggest the two highest-priority actions for the next quarter

Actionable Takeaways

  • Before your next AI initiative launches, fill in the baseline metric and the two leading indicators. If you cannot fill those fields, do not launch yet.
  • Schedule the monthly and quarterly review dates in your calendar on the day the initiative starts. Do not leave this to a future version of yourself.
  • Use the cost and time ROI dimensions separately. Do not collapse them into a single number until you have validated each one at the 90-day review.
  • If an initiative is already running without a baseline, do one of two things: reconstruct the closest available historical data from existing systems, or declare the current state as the baseline and start tracking from now. The worst option is continuing without any reference point.
  • Run the baseline metrics prompt before the next initiative kickoff. It takes 10 minutes and prevents 3 months of unverifiable claims.

Practical Examples

Scenario 1: A small experiment (one PM, one feature)

A product manager at X-company wants to test AI-assisted user story generation for a single feature squad. The initiative is small enough that a full worksheet might feel like overhead.

The right approach is a trimmed version, not no version. Use three fields only: baseline metric (current time to write user stories per sprint, measured over the last 4 sprints), one leading indicator (percentage of stories drafted with AI by sprint 3), and one lagging indicator (average story-writing time at sprint 6). Skip the ROI dimensions until there is enough data. Set one review date at sprint 6.

This takes 15 minutes to set up and produces a result you can share. If the experiment succeeds, the data is already in the format needed to justify a wider rollout.

Scenario 2: A multi-team initiative

X-company's CTO wants to roll out AI code review assistance across all three engineering squads simultaneously. The initiative spans 22 engineers and affects both delivery speed and code quality.

The worksheet needs one entry per squad, not a single aggregated entry. Each squad has a different baseline (code review cycle time varies by squad size and review process), different leading indicators (adoption rate per squad), and different lagging indicators (PR cycle time reduction, defect rate change). Roll-up summaries are produced at the quarterly review by aggregating the three entries.

The initiative owner assigns one squad lead as the data owner per entry. The quarterly review uses all three entries to identify which squad is leading and which needs intervention.

Scenario 3: An initiative mid-flight with no tracking

X-company's sales engineering team started using AI to draft customer-facing technical proposals six weeks ago. No baseline was captured. The VP of Sales wants to know if it is working.

Reconstruct what you can. Pull the last 20 proposals from before the AI rollout and measure average cycle time from intake to delivery. That is your reconstructed baseline. From today forward, track cycle time on every new proposal. At 60 days, compare. The comparison will be less precise than a clean baseline, but it is far more useful than anecdote. Document the reconstruction method in the baseline field so reviewers know what they are looking at.


Implementation Workflow

  1. Pick one active or upcoming AI initiative. It does not need to be large. A single-team experiment qualifies. The goal is to complete one worksheet end to end.

  2. Open the blank worksheet template. Make a copy in whatever tool your team uses for documentation (Notion, Confluence, Google Docs). Name the file: [Initiative Name] AI Impact Tracker.

  3. Fill in the initiative name, owner, business goal, and launch date. If the initiative is already running, use today as the baseline capture date and write "retroactive baseline" in the measurement method field.

  4. Define the baseline metric. Write the metric name, the measurement method, and the time window for the historical sample. Pull the actual number from your existing data. Enter it in the worksheet. If you cannot get a number today, set a date within 5 business days to retrieve it and do not launch until it is filled.

  5. Run the baseline metrics prompt from the Tools section. Use the output to pressure-test your metric choices. Adjust if the prompt surfaces a more precise or more available metric than the one you chose.

  6. Define two leading indicators. Each one needs a target value and a target date within the first 30 days. Write the measurement method: where will this data come from? Who is responsible for pulling it?

  7. Define two lagging indicators. Each one needs a target value and a review date at approximately 90 days post-launch.

  8. Calculate the ROI dimensions. Use the cost dimension formula: (hourly rate) × (hours saved per instance) × (volume per month) × (adoption rate). Write the time dimension as a downstream effect: what does recovering that time enable? Enter both in the worksheet.

  9. Schedule the review dates. Monthly leading indicator check: put it in the calendar of the initiative owner and at least one stakeholder. Quarterly lagging and ROI review: put it in the calendar of the person who approved the initiative budget or headcount.

  10. Share the worksheet with one stakeholder before the initiative launches. This step converts the worksheet from a private tracking document into a shared accountability artifact. The stakeholder does not need to do anything with it: they just need to know it exists and where to find it.

  11. At 30 days, run the monthly check. Pull the actual values for both leading indicators. Write one line of status (on track, at risk, blocked) in the Status at 30 days field. If either indicator is at risk, identify one corrective action before the meeting ends.

  12. At 90 days, run the 90-day review preparation prompt. Use the output to structure the quarterly review. Fill in the Status at 90 days field with actuals vs. targets and a one-paragraph summary. Present the worksheet as the source of truth in the review meeting.