Experiments
You can compare the success of variants in featureflow using Experimental A/B testing.
Experiments are only available for enterprise plans.
You may create experiments to test the effect of a new feature variant. Featureflow will determine which variant is the winner given a defined goal.
1. Define a goal.
A goal is any event that you wish to track. For example a payment or signup submit button. Using the javascript api simple add the following:
featureflow.goal('goal-button-clicked');
2. Configure your feature experiment cohorts
Using feature targeting rules define the cohorts that will be sent to each variant. This may be a simple 50/50 split or more advanced such as sending only a subset to the experiment variants. In your project environment, click 'Update Rules' for your desired feature and set the cohort rules. In the example below we have set a 50/50 split for all users who have the 'ab_cohort' attribute. Everyone else will see the 'off variant'.
The more traffic you send to a variant, the quicker your experiment can derive a meaningful conclusion, of course you may want to balance this to a trusted or beta set of users.
3. Create an experiment
An experiment links a given feature to your desired goal. Under the Experiments menu, click 'New Experiment
Set an experiment name, description, goal key, feature and desired variants.
4. Enable the experiment
Simply toggle the experiment on.
5. View your experiment statistics
Experiments will log impressions and goal conversions. When we reach a 95% confidence level with over 1000 samples we can declare a winner.
