Status: LIVE

Experiment Guide

• Set a hypothesis, e.g. "By forcing all people to sign up before they can use our app, more users will end up as active buyers on our platform, because we expect that the conversion loss during signup is less than the conversion gain during buying".

• Set a measurable prediction "I expect signup conversions to drop from 90% to 30%. I expect selling conversions to rise from 1% to 4%. Therefore I expect a 33.3% percent increase in active buyers."

• Implement appropriate analytics.

• Be scientific. Track a cohort. Eliminate potential confounding influences. Understand the statistical significance of you sample.

• Nominate an experiment lead who will ensure that it gets done. This is basically the AL: Activity Lead for the experiment.

• Set an experiment debrief meeting where the results are presented and discussed.

• Write up an experiment conclusion that includes the data that we measured. The data must be directly comparable with the predictions we made. There may be additional data as well. It should also include a summary of what we learnt.

• Every experiment is documented into our knowledge base.

• If the experiment contains valuable insights, invite the Core Team to a 30-min presentation and share the knowledge. We'd all love to hear it.