Would you expand this pilot yet?

Observations

• 48 users were invited to the beta.
• 18 users (37.5 %) are active on a weekly basis.
• 7 users (14.6 %) completed an interview.
• 5 of those interviewees (71 %) requested a CSV export.
• The pilot has run for only three weeks; no data exist on retention beyond that period.

Inference

• The active‑user rate is modest, and the interview‑completion rate is low relative to the invitation pool.
• The high proportion of interviewees who asked for CSV export suggests that the feature is useful for a subset of engaged users.
• Without longer‑term retention data, it is unclear whether the pilot is gaining traction or merely attracting a small, highly motivated group.

Recommendation
Pause the pilot for now. Extend the trial to at least 6–8 weeks to capture retention and usage depth, and gather more qualitative feedback from both active and inactive participants.

Next evidence to collect

1. Retention metrics (e.g., % of users still active after 4, 6, and 8 weeks).
2. Frequency and depth of feature usage (e.g., number of CSV exports per user, time spent).
3. Qualitative feedback from users who did not become active or who did not complete interviews.
4. Conversion data (if applicable) to see whether active users move toward paid plans or higher engagement.

Observations:

• 48 users invited
• 18 weekly active users
• 7 completed interviews
• 5 interviewees requested CSV export
• Pilot has run for 3 weeks
• No long-term retention evidence

Inference:

• Engagement appears low (18 out of 48 active weekly)
• Limited feedback (only 7 interviews)
• No data on long-term retention

Recommendation: Pause the pilot.

Next evidence to collect: Long-term retention metrics and additional user feedback.

Observations:

• 48 invited users
• 18 weekly active users
• 7 completed interviews
• 5 interviewees requested CSV export
• Pilot run for three weeks

Inference:

• It is uncertain whether to expand, pause, or stop the pilot. The data is insufficient to make a clear recommendation.

Next evidence to collect:

• Longer-term retention data
• User feedback from the remaining invited users
• Specific reasons for requesting CSV export
• Any reported issues or bugs from users