An Experiment is Born

🧑‍🔬 Talk Data to Me

We've got our feature flag, our metrics, our hypothesis, now all that's left is to create our experiment!

A bit about hypotheses
A well-formed hypothesis is the foundation for a great experiment. We're not all data scientists though, and LaunchDarkly is built to work for the data scientists and for the rest of us. In LaunchDarkly, it's easy to iterate on an experiment and evolve the parameters as you learn! LaunchDarkly has got your back too, if there's anything major wrong with your experiment, LaunchDarkly will raise the red flag for you and let you know.

Step 1: In order to create the Experiment, navigate to the Experiments screen in LaunchDarkly.

Step 2: Click the "Create experiment" button:

Step 3: Fill out the Experiment Details with the following:

Name: "Determine how the wording on the accent label affects the customer's decision to purchase"
Hypothesis: "Given the right wording to accent featured products, customers will be more likely to click "Add to Cart" indicating their decision to purchase and ultimately, this will translate to more sales."
Randomization unit: "user"
Primary metric: "Add to Cart Click"
Secondary metric: "Checkout Click"
Choose flag variations: Flag: "8 - Featured Product Label"
Set audience: Set the value for each variation to "25%", including "None" 0% of user contexts remain and will be served: Control: "None"

Step 4: If your experiment settings look like this, save your new experiment

🧬 It's Alive!

We're now ready to launch this experiment and see which variation will work the best for our application!

Step 1: After saving you should see this at the top of your experiment detail screen:

Step 2: Go turn the 8 - Featured Product Label feature flag on, it should serve "None" as the default and "None" when the flag is off.

Step 3: Come back to the experiment screen and click start. ▶️

🧑‍🏫 Let's Get Learning

Experience is our best teacher and nothing beats real-world data direct from our users. The thing with users is, they can tell you what they like and don't like, but they can't tell you what will change their behavior to get the outcomes you want, and avoid the outcomes you don't.

Everyone here at Toggle Outfitters has a different idea about what should work well. We appreciate everyone's opinions and ideas, but getting this real-world data to share with everyone is going to get us all on the same page about what really resonates with our users.

Experimentation in Action!

Step 1: If things are working as expected, you should be able to visit your Toggle Outfitters app on your machine, refresh a few times and see the different variations in action.

Here are the variations in action:

The Classic "NEW"	The Bargain "SALE"	Who doesn't want something "SPECIAL"?	The Control "No Label"

Step 2: With our variations in place, we can start collecting real world data! You could simply add a few to the cart to see the metrics change, but we have a better solution!