We believe that data for growth companies has two fundamental elements: analytics and experimentation. Analytics is there to help you understand your users, your marketing, and your product. And experimentation is there to understand how changes to your product and marketing impact business outcomes, usually by running controlled tests.
Typically when we talk to clients about experimentation, they have in mind a northstar metric that they would like to improve, and more often than not that is something related to either revenue or user engagement.
But for Climaider the challenge was different. Their mission is to help individuals to reduce their carbon emissions, either through carbon offsetting, or through challenges that the user undertakes to change their lifestyle in an emission-reducing way, for example by buying second-hand clothing, or pledging not to fly.
They wanted our help to construct a sequence of experiments, that would identify how changes to their product and their growth activities impacted on the total CO2 reduction of their userbase.
Experimentation typically follows a four step process:
Below are a few highlights from running that process with the Climaider team.
We used two primary methods for ideation: delving into analytics data to find insights and opportunities, and applying concepts seen succeed in other verticals. With these approaches, we came up with some pretty interesting experiment ideas, including:
We used an adapted method of the RICE prioritization technique to score our experiment ideas, and run controlled A/B tests on their react app, analyzing the results with Amplitude funnels. This has enabled the Climaider team to see how their changes are impacting conversion rates and ultimately CO2 reduction.
We've still got lots of experiment ideas we want to run on the Climaider product, and are delighted that our work has such a meaningful impact.