Before you start: what do you know about SEO split-testing? If you're unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we're suggesting you start here or request a demo of SplitSignal.
First, we asked our Twitter followers to vote:
Only 31.7% of our followers guessed it right, the test result was negative.
Read the full case study to find out why.
The Case Study
If you want to be successful with SEO in 2021 (and beyond), you need to be the most relevant result for a search query. Creating content that matches the user’s search intent is becoming increasingly important. On the one hand, Google has improved its algorithm over the years, with updates like Hummingbird, RankBrain, and Bert able to determine people's search intent and reward pages that best fit that particular intent. On the other hand, you need to convince your target audience to choose your result instead of the 9 other search results you are competing with.
In this case study for a major medical site, we will show that search intent is very powerful and that understanding search intent and successfully responding to that search intent are two very different things.
Before moving to the case study, let's take a brief look at what search intent is. Search intent (or user intent) is why a user performs a specific search. What are they looking for? Anyone who searches online wants to find something. What they are searching for varies enormously, but we distinguish 4 common search intentions:
Informational. The user is looking for specific information and wants to know something about a certain topic. Navigational. The user wants to find a specific page or website. Instead of going directly to the site via the browser bar, they use the search engine as an intermediate step. Commercial. The user is looking to convert but hasn’t made a final decision and is still investigating. Transactional. The user wants to complete an action, like completing a purchase. Note that this is not just about buying.To learn more about search intent, check out this complete guide written a little while ago.
Now, let’s dive into the case
The Hypothesis
This particular website had the following title tag:
They hypothesized that replacing the text after the colon with the question “what are they?”, which reflects the user's actual search intent, would have a positive effect on organic traffic to the (informational) side effect pages.
The Test
We used SplitSignal to easily set up and run the test. The test involved a page title change with a focus on search intent. 350 pages were selected as either variant or control. We've changed the variant group page title to include the question:
The Result
We ran the test with SplitSignal for 21 days and saw the following results:
Almost immediately after the test started, traffic from our modeled control group outperformed the actual traffic from the variant group. This means that the traffic to the variant group performs worse than predicted and that the result of the test is negative.
The cumulative view shows not only that the change has a negative effect on clicks but also that we performed a significant test. When all three curves perform below (negative) or above (positive) the y=0 (cumulative gradient) axis, the test is statistically significant. At the end of the test, the confidence level was 99%. This means we can be 99% sure that adding a question that resembles the user's search intent will negatively affect the organic traffic to the website. “But how is that possible?” you may wonder. What about the fact that search intent would matter so much? This is where the difference between understanding search intent and how to deal with it comes into play.
Why?
As mentioned, understanding a user's search intent is not enough. You will also need to think about how to respond to their intent in the best possible way to be the most relevant answer in the search results. For this medical website, it is very clear that users have an informational search intent, and answering the user's intent with a question does not match the intent of the target audience. We think that (certainly in this industry) people attach great value to authoritative information publishers. Placing a question in the page title rather than actually stating what the user will find on the website's page (answering the “why”) is not a sign of authority or even trustworthiness. It may be worthwhile for the organization to investigate this further through qualitative research.
Over the course of the test, we saw that the CTR dropped dramatically. Compared to our modeled control group, rankings and impressions remained fairly stable. We do think that if this was tested for a longer period of time, they would experience a loss in rankings because their search results are less relevant. The search results show that Google believes they're responding to the user's search intent, but the actual search result provided just doesn't fit what a user expects when looking for answers to these types of medical questions.
This test shows not only the immense power of search intent but also the importance of testing these types of changes. We know from experience that these kinds of tests can be very positive.
Keep in mind that results can differ per site, industry, and target audience. Given the impact this can have on your overall search performance, it's a good idea to not only research your target audience's search intent but also to thoroughly test these optimizations. In this case, the organization was lucky to learn more about what their target audience expects, giving them the necessary information to better fine-tune optimizations and their journey, providing users with the best possible answer.
Have your next SEO split-test analyzed by OrangeValley Agency.
Innovative SEO services
SEO is a patience game; no secret there. We`ll work with you to develop a Search strategy focused on producing increased traffic rankings in as early as 3-months.
A proven Allinclusive. SEO services for measuring, executing, and optimizing for Search Engine success. We say what we do and do what we say.
Our company as Semrush Agency Partner has designed a search engine optimization service that is both ethical and result-driven. We use the latest tools, strategies, and trends to help you move up in the search engines for the right keywords to get noticed by the right audience.
Today, you can schedule a Discovery call with us about your company needs.
Source:
