What is a Marketing Experiment?
From the marketing stand of view, per hubspot “an experiment is a form of market research in which your goal is to discover new strategies for future campaigns or validate existing ones.”
Differences between a survey and an experiment
In a survey, you ask questions to a selected audience to obtain feedback about, for example, a service provided by a company. Based on the survey results, you can improve the quality of the service, discontinue the service or improve the current service. A survey is suitable when looking for feedback on a specific topic, it shows you a result from variables you cannot control.
In an experiment, you test a campaign to see which performs better. For example, you may want to test a landing page performance by changing a “call to action” for another one, so with this experiment, you may discover that the current call to action is better than the one you want to implement or maybe a different results. An experiment is good to test the validity of a hypothesis and predict what happens with one variable when you manipulate other.
Steps to perform a marketing experiment
- What is the problem? identify the problem.
- Brainstorm your ideas and focus on one idea.
- Make a hypothesis.
- Collect the data research.
- Select your KPIs or metrics.
- Execute the experiment.
- Analyze your results.
Running a Landing Page Experiment
Imagine you are a car body shop in South Florida, running a Google ads campaign targeting people who have been involved in a car accident and need car repairs. You are running your campaign, but you have experienced a decrease in conversions or people calling you to get a free quote to repair their car. You don’t know what is happening, so ask the PPC agency to help you investigate. The agency proposed to run an AB testing and make a small change in one element in the landing page that has been used in the campaign.
- What is the problem?
The first step you need to do is to identify your problem. In this case, is a decrease in conversions. What is causing the problem? the color of your call-to-action? the hero image you have on the top? page speed? mobile responsiveness? Pick one presumed issue and start the test. - What is the proposed solution?
Once you have identified the problem, you must determine how to fix it. Remember to pick one issue to test. - What will be the result?
What do you want the outcome will be? You have to specify your KPIs. Define the results you expect to obtain by experimenting. - Write your hypothesis
Assume that you want to change the Call to Action from Call for a Free quote to Get a Free Quote Now. So the hypothesis should be like this: Changing the call to action on the landing page from “call for a free quote” to “get a free quote now” will draw more attention to consumers and increase the number of leads. - The next step will be to generate an alternative page changing only the proposed CTA, then set up the experiment in Google ads where the campaign has been running.
- KPIs. Establishing which metrics you will use to measure the results is important. For this case, the issue was a decrease in conversions, so we should establish a baseline using: cost, impressions, clicks, ctr, bounce rate, and conversions, and, after executing the experiment, compare these KPIs to see if there are any improvements or not.
- If the experiment doesn’t show a statistical difference between pages A and B, you should pick another page element, re-formulate your hypothesis, and test again. To compare results, I will suggest performing a chi-square test.
In this example experiment, our independent variable is the call to action button. This is the only change we will test. Upon the results, we will decide to change the variable for the hero image, background color, or any other variable. We can control this. The dependable variable is the conversions, which we want to improve, but we don’t have any control over this. In the case of South Florida, the extraneous variable may be a Hurricane hitting the area, people are more worried about getting supplies than repairing a car, the inflation and cost of living, etc. There are extraneous variables we cannot control but we need to observe because they can affect the results or be the cause of the decrease of conversions.
Digital marketing experiments like the one I just gave, are easy and cheap to implement and can help us to improve our campaign performance, so as I mentioned in other blog post, this is an endless AB testing process.
