I’m Taige, founder of Fairytrail, a freemium travel buddy app for remote workers. We launched our new monetization feature a month ago. Shoutout to Mixpanel for their startup friendly free plan.
Key Question
The question was how much of a free trial should we give to users? If we don’t give enough of a trial, people won’t trust us and won’t see value. If we give too much, people may naturally churn and we won’t be able to monetize.
Since we are a startup, user growth is more important than monetization. However, charging can be great for growth. In Elad Gil’s High Growth Handbook, Marc Andreessen says , “Companies that charge more can better fund both their distribution efforts and their ongoing R&D efforts. Charging more is a key lever to be able to grow. And the companies that charge more therefore tend to grow faster.”
Experiment
When we launched the feature, we setup an A/B test to measure
- Conversion to paid
- Engagement (i.e. matches, chats)
- Retention
- Quality (e.g. profiles reported)
We split users into two groups: 50% of users were able to send 3 connect requests before we ask them to pay. 50% of users were able to send 30 free connect requests. Note: There’s still a verify with friends option so technically users never have to pay to use this app. They just need to do the work of inviting a remote worker friend to the app to verify them.
Verify Screen
Test Group A : 3 free connects before being asked to verify
Test Group B : 30 free connects before being asked to verify
My hypothesis was that sending 3 connect requests would be enough to build trust with our users and offering more would not help much.
We’d see higher conversion rate to paid. We’d see less engagement due to users being restricted. We’d see a lower retention rate as some users would delete their accounts because of the friction and they have a lower chance to start a chat. We’d see a higher quality user base as paying users are more intentional.
Expected Results
| Metric | 3 Free Connects | 30 Free Connects |
|---|---|---|
| Conversion to Paid | Baseline +5% | Baseline |
| Engagement | Baseline -5% | Baseline |
| Retention | Baseline -5% | Baseline |
| Reported Profiles | Baseline -5% | Baseline |
Findings
After running our experiment for 30 days, here’s what we found.
Conversion To Paid
3 Free Connects group had a 10% higher conversion rate. Time to convert was 1.5 days instead of significantly 2.6 days. Asking for payment sooner, let to more payment and payment.
Paid Conversion Rate
Engagement
On the other hand, the 30 Free Connects group had a much higher match rate as measured by users having at least one match, 87% vs. 60%. Higher match is better!
Match Rate
30 Free Connects group had 7% more engagement events per day per user than the 3 Free Connects group. Higher engagement is good, so that’s another win for 30 free connects.
Daily Avg Events Per User
Retention
Both test groups had a weekly retention of around 70% at the 4 week mark The 30 Free Connects group had a 5% higher retention after 4 weeks… which could add up. But you can already see the 3 Free Connects group retention rate start to flatten out which is exciting for long term retention.
Weekly Retention
Quality
Users in 3 Free Connect Group surprisingly had a higher % of being blocked/reported at least once but this is under 1% and not stat sig.
Actual Results
| Metric | 3 Free Connects | 30 Free Connects |
|---|---|---|
| Conversion to Paid (Signed Up to Paid) | + 26% (+10%) | 22% |
| Engagement ( Daily Core Events / User ) | 10.6 (-7%) | 11.3 |
| Retention (4 weeks) | 67% (- 4%) | 71% |
| Reported Profiles (% of profiles blocked/flagged) | 0.5% (+0.3%) | 0.2% |
Conclusion
When we allowed sending only 3 free connect requests, the conversion to paid increased while retention dropped as expected. Since engagement and retention are more important for us than monetization, it makes sense for us to give 30 free connects instead of 3. The tradeoff is worth it as it seems we don’t need to monetize immediately and we haven’t found a good way to use our revenue to grow. We tried some ads and it was $30+ per install.
Another interesting observation is users in the 30 free connects group were more willing to subscribe to a premium plan. The 4.6% of the 3 Free Connects group were on a premium plan, while the 5.5% of the 30 Free Connects group were on a premium plan.
Perhaps the extra free connects allowed them to trust us more and better understand the value. Perhaps it’s the reciprocity effect.
Tier Distribution
For clarity, free means the user paid $9.99 for a one-time verification (or verified with friends) and is on the free plan. Gated means the user cannot send more connect requests until they verify. Silver and gold mean the user is paying a monthly subscription for a premium plan.
We’ll be doing more fun experiments in the future. Follow me on LinkedIn or Twitter to read the next one.
Thanks for reading!
Taige
