How we reduced our cancellation rate by 87.5%

45 Replies

I was stressing.

We had seen 40% of our customers cancel, and it was eating me up inside.

Each of those (former) customers had:

  1. decided they had the problem SocialWOD solves (workout tracking for every gym member at your gym, just by snapping and emailing a photo of your results whiteboard)
  2. decided they needed to solve that problem
  3. done research on solutions that might solve their problem
  4. decided that SocialWOD was compelling enough to try
  5. decided to take out their credit card to pay SocialWOD
  6. used and promoted SocialWOD to their members

After investing that much mental and emotional energy, they then they decided to cancel.

Argh.

We were trying to understand what the key differences were between our customers that cancelled, and our customers that didn’t.

We hop on the phone every week or two to call up customers, so qualitative data was no problem.

But we wanted to see what we could learn from our data.

Enter the Cohort Analysis

A cohort analysis helps you see how behavior varies across different cohorts of customers.

In this case, we were interested in the behavior of the cohort of customers who HAD cancelled, vs the behavior of those that hadn’t. What differences were there in each cohort’s usage of our product, and could we steer more gyms to do things like those gyms that had stayed on as customers?

I had read high-level stuff about cohort analyses, but it was this 90 minute cohort analysis class in NY with genius marketer Cassie Lancellotti-Young that really opened my eyes to how to run a proper one.

What Tools to use

Cassie recommended using Excel, and provided a template to use as the basis for a cohort analysis.

She mentioned tools like Kissmetrics and Mixpanel, but they were harder to use, less flexible, and required more overhead than Excel.

Given that likely all of the useful data you’ll want to analyze lives in your application’s database, her suggestion made a ton of sense to me.

How to start

We started by asking questions.

In this case, we were trying to answer this question:

“What are the major differences between customers that cancel, and customers that don’t?”

Pull some data

The next step was to write a ginormous SQL query pulling out all the data that could possibly help us answer this question.

Here’s some of the relevant data we pulled:

  • The month a customer joined in
  • The amount a customer paid per month
  • Free trial length
  • The number of a gym’s members that claimed their SocialWOD profiles
  • # days after paying before canceling their service
  • whether a gym uses SocialWOD to post their workouts to their gym’s Facebook page (where the gym’s members hang out)

All in all, there were 51 different fields of data that we pulled.

Analyze the raw data

I ran pivot tables comparing the difference between gyms that cancelled and gyms that didn’t.

I learned that customers who cancelled:

  1. had about half as many of their members claim a SocialWOD profile (so they could engage with our product)
  2. used the “post our workout results to my gym’s Facebook page” feature about half as much as gyms who remained customers
  3. paid about 23% more than gyms who stayed
  4. cancelled their subscription after 61 days on average

Square the quantitative with the qualitative

We have a spreadsheet that details cancellation reasons for every customer, and I’d estimate we’ve talked to at least half of the customers who quit.

The two most common reasons we kept hearing about why gyms quit were:

  1. our athletes aren’t using it
  2. it’s too expensive. SocialWOD is a nice-to-have (we’re working on that 😉 ). A member CRM is essentially a must-have for a gym over a certain size. The price for a member CRM is lower that it should be imho, and it sets a strong price anchor in the mind of a customer (i.e. “I *need* a CRM and it costs $X. I don’t *need* SocialWOD, but it costs $1.5X.”)

Given that the quantitative data suggested high member usage and lower price were big drivers of customer retention, the qualitative and quantitative data told a pretty strong story about what to do to cut down on cancellations.

Acting on the results

Here’s what we decided to do about each learning:

1. A cancelled customer had about half as many of their members claim a SocialWOD profile (so they could engage with our product).

Previously, our customers would tell their members to sign up for SocialWOD by posting an announcement about SocialWOD on their gym’s Facebook page, writing on their gym’s blog (read by most members), talking about SocialWOD at the gym, etc.

We figured there’s nothing easier to act on than an email describing benefits and a call-to-action. So we improved our onboarding to help a gym owner export a CSV of their members’ email addresses to send to us.

Once they do, we email each member a description of how SocialWOD benefits them, along with a link to click to sign up.

2. A cancelled customer used the “post our workout results to my gym’s Facebook page” feature about half as much as gyms who remained customers.

We improved our onboarding so that customers who haven’t set up posting to Facebook receive an email detailing the benefits, providing a video, and a strong CTA to do so.

3. A cancelled customer paid about 23% more than gyms who stayed

We built some technology than makes processing photos cheaper for larger gyms, and passed those savings on to both our existing and new customers. Those changes enabled us to drop prices by 15-60%, depending on price plan.

4. A customer cancelled their subscription after 61 days on average

We haven’t acted on this yet, but I can imagine sending gyms that meet some “likely to cancel” criteria an offer around day 45 of their membership to lock in for a year at a heavy discount.

The Outcome

Since implementing changes 1-3 two months ago, we’ve seen our cancellation rate drop from 40% to 5% – an 87.5% decrease. We’re going to run another cohort analysis in a couple months to isolate the impact of each change as it’s still too early to know the long-term impact of these changes, but in the short-term both Ryan and I are stoked about stopping the hemorrhaging.

—

Interested in learning how a cohort analysis can help your business grow? Get in touch – I work with select clients to help identify growth and retention opportunities, and build features to realize those opportunities.

—

Update (9/2/2012): You can now learn how to do a cohort analysis in this downloadable class.

Get more actionable tactics to grow your business

       

45 thoughts on “How we reduced our cancellation rate by 87.5%”

  1. Pingback: How we reduced our cancellation rate by 87.5% | Inbound.org

  3. Ben

    Great play-by-play. A few questions though:

    “Those changes enabled us drop prices 15-60%”
    >> How has that effected your gross revenue?

    And, did you go to any lengths to isolate the variables that you changed? In other words, perhaps the lower price-point doesn’t matter as much as those other components, so dropping your price by 15-60% may be unnecessary and is cannibalizing your revenue. Interested in your thoughts.

    Reply ↓
    1. Brandon

      In this case revenue isn’t the most valuable metric, but profit is (or profit per customer or revenue:profit ratio). By reducing costs, reducing the price may necessarily reduce revenue but doesn’t necessarily eat into profit margins. Of course profits margins would be smaller than if they had maintained the same price and reduced their costs.

      Still, I agree that this is an interesting statistic worth measuring and seeing.

      Reply ↓
    2. kareem Post author

      as you might imagine, gross revenue declined but has recovered to pre-price drop levels (we saw an increase in signups). Also, we didn’t isolate the variables. Our hypothesis is that price was the biggest factor in seeing cancellation rate drop. If the answer could have a meaningful impact on our biz, I’ll run another cohort analysis several months from now to see if I can find some answers.

      Reply ↓
  4. Guy Nirpaz

    Kareem, great insightful post!

    This is an excellent process you’ve followed to reduce churn which is killing businesses.

    What I liked most, is that you’ve first identified the problem you have, gathered data in order to act on it.

    Guy Nirpaz,
    Founder of Totango
    Customer Success Software

    Reply ↓
    1. kareem Post author

      Charles Kettering said “a problem well stated is a problem half solved.” I try and keep that in mind before writing SQL 🙂

      Reply ↓
      1. Jojo

        Hi Kareem,

        I’m interested in your bootstrap cohort analysis pkg … When will you run the 50% off deal again?

        Best,
        JJ

        Reply ↓

  5. Pingback: Flow » Blog Archive » Daily Digest for June 17th - The zeitgeist daily

  6. Rohan Singh

    Perhaps for #4, instead of just offering a straight-up discount, you could call customers around the 45-day mark and ask them how they like the product and whether there’s anything more they would like?

    Reply ↓
    1. kareem Post author

      @rohan yep, that’s a possibility too. we usually call our customers around 20-30 days in (which is 35-45 including the free trial period) though.

      Reply ↓
  8. Santos Halper

    I have to say, as impressed as I am with the scientific and methodical approach you took with analyzing the problem. I am a bit disappointed in your rather unscientific “shotgun” approach to fixing it. Perhaps you felt that time was of the essence and you have to stop the bleeding now, but it would have been interesting to solve one problem at a time and isolate the impacts.

    Still… kudos to you for running your business on legitimate analysis.

    Reply ↓
    1. kareem Post author

      thanks. while we’re curious about cause and effect of changes we make, our priority is building a profitable business right now. we can always try and isolate the effects if it will help us be more profitable later.

      Reply ↓
  9. Ed Hallen

    Kareem – Fascinating article. The 45 day offer and the common characteristics of cancelling customers is something I’ve seen at each software company I’ve worked at as well as something I’ve heard alot lately while talking to potential Klaviyo customers.

    One thought on how you address it. First, perform the same cohort analysis you’ve already done, but look at the cancelling customers vs retained customers at day 1, day 15, day 30 and day 45, then use this analysis to figure out your triggers (things like # of Facebook posts needed by day 15, % of profiles claimed by day 30, etc). Once you have your triggers, you can make proactively calling / emailing problematic customers a key part of your daily routine. While discounts might still be the way to go, this trigger based approach is one we’ve seen work well for our clients.

    Glad to discuss it in more detail.

    Reply ↓

  11. Pingback: Audience and Product Experimentation « Tinker On

    1. kareem Post author

      Thanks Tyler. I’m likely going to write another post about how to run a cohort analysis soon, and include the Excel templates with that. Stay tuned!

      Reply ↓

  14. Pingback: Can you provide examples of following a data driven approach to product management in consumer web products? - Quora

  15. Pingback: Monday Links from the Bench vol. CXXIX

  16. Pingback: 192: TZ Discussion – Trillions and Trillions of Stars

  17. Bill MacEwen

    Nice post. We’ve been using Kissmetrics but it’s tough to get a specific as you have. Have you thought of writing some analytics into the app so you can see how your cohorts are doing on the fly?

    Reply ↓
    1. kareem Post author

      At most we’d find interesting changes week over week, but depending on what we’re looking for it’s month over month. Once the SQL query is written, it’s trivial to run and import into Excel… so I’d rather spend the time writing new features or growing SocialWOD than by building in-app analytics right now.

      Reply ↓

  21. Pingback: Your Audience: To Lead or Listen? | Just Sherpas Blog

  22. Pingback: Management Improvement Blog Carnival #170 » Curious Cat Management Improvement Blog

  24. Pingback: Viv Taylor » How we reduced our cancellation rate by 87.5%

  25. Pingback: “This Is How We Built It” Case Studies (Part 2)

  26. Pingback: 75 Instructive Case Studies - This Is How We Built It – Smashing Magazine

  27. Pingback: 75 Instructive Design Case Studies | Design Guru