Semrush Keyword Difficulty: Now More Accurate Than Any Other Tool

In the spring of 2021, we updated the formula that calculates a search term’s keyword difficulty and relaunched the metric.

These changes were made to make sure Semrush offers the best keyword research software possible.

The score is now more accurate for all types of keywords across all of our regional databases.

Especially when it comes to lower volume keywords, lower competition keywords, and local keywords in less-populated countries and regions, Semrush now provides much more consistency and accuracy than any other SEO tool.

I took a look at the new KD scores for our keywords - amazing job! They are much more accurate now.

Bartosz Góralewisz, CEO at Onely

What is Semrush Keyword Difficulty?

Keyword Difficulty (KD) is a metric that tells you how much SEO effort it might take to organically rank a page in the top 10 results on Google for a certain keyword.

You can use KD% to plan out your SEO campaigns and set priorities.

Looking for keywords with low KD% scores will help you find long-tail keywords and low hanging fruit for your campaigns.

The higher the percentage is, the harder the competition will be, and the more work you will need to do before seeing results.

Let’s say you’re considering two keywords for an SEO campaign:

Keyword A, which has a 40% KD, and keyword B, which has an 80% KD.

Focusing on keyword A gives you a better chance of ranking on the first page faster. If you try to rank for keyword B, it would be more difficult and take more time.

This difficulty percentage is estimated based on the competitive factors that impact the challenge of SEO, which we’ll talk about below in this post.

Learn More: Keyword Difficulty: How to Measure Ranking Possibility

Where is Keyword Difficulty in Semrush?

To find the keyword difficulty of any keyword, go to Keyword Overview and enter the keyword into the search bar. Then, locate the keyword difficulty widget.

Below, we can see that “seo” is rated as 88%.

You’ll also see a column titled KD% in any Semrush report that has keywords. You can filter or sort these columns to find your best opportunities.

The full list of Semrush reports where you’ll see keyword difficulty includes:

Keyword Magic Tool Keyword Manager Keyword Overview Keyword Gap Organic Research Organic Traffic Insights Advertising Research

Need to find KD scores for your list of target keywords?

You can check up to 100 keywords at a time via the Keyword Overview.

You can also filter any of our keyword reports by KD% range to hone in on a specific range of difficulty.

For very low volume keywords that have little customer interest, we might not calculate KD% until it is requested by a customer. All you have to do is add your keyword to a list in the Keyword Manager and find the refresh metrics button.

After hitting this refresh you should see a live calculation for this keyword’s KD score along with other metrics like click potential and SERP Features.

Keyword Difficulty vs. Competitive Density

One thing to note is that you don’t want to confuse keyword difficulty with competitive density.

While both metrics estimate how hard it is to target a keyword;

Keyword Difficulty (KD%) represents the difficulty of ranking in organic search (SEO) Competitive density (Com.) measures the difficulty of ranking via paid search (PPC).

Competitive density is based on the advertisers we see running PPC ads on the SERPs.

What is a Good Keyword Difficulty Score?

After the updated formula, we now can categorize the score into 6 main buckets. The first three are attainable and great opportunities for new sites, and the other three are more difficult.

0-14 = Very easy. These are the best opportunities to start ranking new webpages on Google as soon as possible without backlinks. 15-29 = Easy. These keywords have some competition but are still possible to rank for when you’re starting out. To be able to rank for these, you’ll need quality content focused on the keyword’s intent. 30-49 = Possible. Slightly more competition. You’ll need well-structured and unique content appropriately optimized for your keywords.

The top three buckets are going to require more effort in order to rank, but they could still be acquired over time.

50-69 = Difficult. You’ll need to have some backlinks in addition to your well-structured, helpful and optimized content in order to compete here. 70-84 = Hard. Even stiffer competition. These keywords will demand more effort in terms of getting higher authority referring domains in order to rank your well-optimized and helpful content among the top pages. 85-100 = Very hard. The absolute hardest keywords to compete for, especially for a new website. These will demand a lot of on page SEO, link building, and content promotion efforts to eventually rank and acquire traffic.

The calculation that takes into account the median # of backlinks & Authority Score PLUS the relative weight of the links really helps nail down a more precise score. Thanks for putting the time and effort into this.

Franco Valentino, Founder at Narrative SEO, LLC

How to Use Keyword Difficulty

Keyword difficulty should be taken into consideration when planning search campaigns around keywords.

On every report on Semrush, you can set a KD filter to find the best opportunities for you. This can be helpful in a number of instances.

For example:

When building a new campaign, use it to find low competition keywords for the keyword list In a competitive analysis, run a Keyword Gap to discover keywords in your competitor’s SEO strategy that could be good targets for you Audit an existing site and set keyword priorities for SEO vs PPC Prioritize long-tail keywords and low hanging fruit for quicker SEO wins Filter away high KD% keywords that would just waste your time/budget Target the more difficult keywords with PPC instead of/in addition to SEO Seo services san diego 6

What’s the Semrush Keyword Difficulty Formula?

The new formula for Keyword Difficulty on Semrush can be explained in three main steps.

Step 1: Analyzing the SERP.

First, we look at the top 10 ranking pages on Google’s SERP and identify the quality of these pages. If you want to outrank your competition, you’ll need to know how strong their SEO is.

We check how many referring domains are pointing to each page, as well as the follow/nofollow link ratio and overall Authority Score of the domains.

Then, we identify the median value of each and use that for the formula.

Considering the median values within the top 10 helps avoid any inflated keyword difficulty scores that could discourage you.

For example, if a certain keyword has one super strong page ranked #1 on the SERP, but the rest of the results are not as strong, you could still compete and get traffic from a top-3 spot.

Step 2: Analyzing the keyword.

Next we look into the keyword’s search volume, word count, SERP Features, whether it’s branded or not, and a few more characteristics.

These parameters all carry different weights based on how much they are likely to impact the challenge of ranking on the first page.

These parameters (and how much weight they carry) include:

Parameter

Weight Share

Median # of referring domains for ranking URLs

41.22%

Median Authority Score for ranking domains

16.99%

Search volume

9.47%

Median follow/no-follow ratio for ranking URLs

9.17%

Featured snippet

4.74%

Instant answer

4.07%

Branded keyword

3.50%

Local pack

3.39%

Knowledge panel

2.71%

Methodology

To inform the changes to our formula, our data scientists and engineers ran extensive research into patterns of SERP movement among over 100,000 keywords in our database.

We then took this data and analyzed volatility and movement among SERPs to identify which were most difficult to break into the first page via SEO.

Next, we identified patterns and correlations between 100+ parameters for each keyword and used only the most important out of them.

As a result, we finalized the list of parameters into the most important ones and calculated their weight above.

We've had our KD metric for about 7 years now, and our team has been listening to user feedback over these years. This way we were able to see an opportunity to improve this metric even further to be more accurate for your needs. In the past 6 months, our development team dedicated over 4,500 hours into the research and re-formulation of this metric. We considered many parameters related to the difficulty of ranking and geography. We’re now confident in saying that our new formula leads to the most accurate keyword difficulty score on the market. We even ran a study to prove this fact. Thank you for trusting us and please always feel free to share your feedback to keyword-research-feedback@semrush.com

Pavel Kolesnikov, Product Owner at Semrush

Why Semrush has the Best Keyword Difficulty Software

There are plenty of similar keyword metrics provided by other SEO tools, but we believe that our KD formula is now the most accurate and consistent.

How do we know that?

Our formula is more robust and factors in way more parameters than any other metric.

This is especially important in the modern SEO era where instant answers, knowledge graphs, and other SERP features by Google are adding nuance to each and every SERP.

Our formula is weighted per country. What this means is that we factor in the size and scope of the websites and web traffic in any given country when calculating the KD%.

For example, without weighing these factors, you could see a keyword like “google” having a KD score of 99% in the US, but something like 60% in a less populated country such as Finland.

In reality, it would still not be easy to rank a website for the query “google” in Finland, so we had to make sure our formula would take this into account.

The updated formula takes into account a country’s size and the number of websites and this leads to more accurate and consistent scores across different countries.

For this reason, we can say that we have the best keyword difficulty tool for local SEO and international SEO research.

The new and improved @semrush KD metric definitely looks more realistic based on the difficulty of actually ranking for a given keyword. I’ve analyzed it based on websites that I’ve ranked and assessed my effort:rank ratio. Higher SV doesn’t always mean higher KD. Great work!

Itamar Blauer, SEO Consultant

Keyword Difficulty Use Case

Cristian Drucioc at Mention.com gave us this testimonial after we asked him to compare the new score to the old score.

Cristian had a project around the ‘best time to post on social media’ and selected a bunch of keywords last year.

This was helpful because he had the history of the old KD and was able to export it to compare with the new KD and tried to make some comparisons.

Among the things he noticed were no more 0 KD scores compared to the old version, and the higher scores for terms he targeted matched up with his experience of difficulties while optimizing for them. These were his words:

No more 0 KD scores compared to the old version. We can see in the old version it's either 70-79 or 0. Now the average KD makes more sense because there are no more 0’s. That's really nice. I have a bunch of keywords related to "best time to post on" +twitter +day AND +facebook +day AND +linkedin +day. From my experience related to blog posts, the same content related to Twitter will be easier to rank on than Facebook or Instagram. And according to the new KD that's the case. Overall, the new KD makes more sense than the previous one which I was never using.

New Keyword Difficulty! Thanks @semrush for this MEANINGFUL KD%.

Peter Meade, Wordpress SEO Consultant

Data Study: Comparing Semrush Keyword Difficulty to Other SEO Tools

To prove our score is the most accurate on the market, we ran a study between our metric and four other keyword research tools (Ahrefs, Mangools, Serpstat, and Moz).

From this research, we made the following conclusions:

Semrush’s KD is the most accurate on the market because it has the most accurate correlations with backlink profile strength and keyword parameters. Semrush’s KD performs accurately across different regions and databases. Semrush’s KD works especially well for long-tail and local keywords. Our competitors had very low values for long-tail and regional keywords. Our competitors’ KD scores were either lower than ours or absent, especially in countries where backlink profiles are generally smaller than those in the US.

Data Study Methodology:

We gathered 30,000 keywords in the following four regions (US, DE, IT, FI) with a wide and equal range of high, medium, and low volume, CPC, and word length. We collected the keyword difficulty scores of these 120,000 keywords from Ahrefs, Mangools, Serpstat, and Semrush. Looking in the US database, we analyzed the correlations between each KD score and four factors that contribute to the difficulty of SEO (search volume, keyword word count, Authority Score, and referring domains). Looking in all four regional databases, we compared how KD scores compared between Semrush and our competitors for the complete list of keywords.

Correlations with Key SEO Factors

We took a set of keywords and observed how one parameter behaved in relation to the other, in order to detect dependencies.

For example, we expected that, with the volume’s growth, KD will grow as well, meaning that the greater the correlation the more accurate KD will be.

Correlations with volume (US)

Correlations with word count (US)

Correlations with referring domains per URL (US)

Correlations with Authority Score (US)

From the graph above, we can see that Moz KD correlates with Authority Score stronger than others. This could mean that Moz KD may strongly depend on a domain’s strength from the top 10 results.

A stronger domain in the top leads to a significant growth of KD, while the URL itself can be weak. This can provoke a situation when, for example, an unpopular Amazon shop got into the top, making KD grow significantly.

Comparison of Semrush KD with competitors (US)

Here we looked at how closely the KD scores of our competitors mirrored the KD scores from Semrush.

As you can see, we found many instances of our competitors’ scores being far off from our new score.

Notice all of the borderline values where you see the plotted points topping off at the top and bottom of the Y axis. These points indicate one of our competitor’s scores reported 0 (bottom of Y axis) or 100 (top of Y axis) while we reported an actual score.

What this means is that you can find more opportunities using keyword difficulty in Semrush than any other tool.

What We Discovered About Ahrefs, Mangools, Moz, and Serpstat

Ahrefs has a lot of keywords showing 0 and 100 KD. The analysis showed that Ahrefs shows 0 KD for keywords that correlated with small backlink profiles among competitors.

If a keyword is popular and quite difficult to optimize for, but competitors have weak backlink profiles, Ahrefs KD will turn out to be low.

Some keyword examples where Ahrefs showed KD=0 (US) include:

home goods near me, csl bonus, car alignment near me, smoothies near me, coffee shops near me, data analyst salary, coffee shops

These are mostly high-volume keywords with transactional intent, which should probably have a higher KD.

All in all, Ahrefs KD is suitable for the needs of a strong domain, when the goal is to look for keywords where competitors simply have weak backlink profiles.

For any other goal, Ahrefs KD won’t provide accurate insights. This is less evident for the US, where backlink profiles are generally larger. But if we look at smaller countries and databases, abnormalities of Ahrefs KD become more obvious.

For example, in Germany, Ahrefs shows significantly low KD for some high volume queries:

Keyword (DE)

semrush_KD

ahrefs_KD

audi rs6

zara home

volvo xc90

titanic film

lego star wars

nike air max

nissan gtr

dyson v10

coronavirus news

100

restaurant in der nähe

hotel in stuttgart

küchenmesser test

Mangools' KD appeared slightly more accurate than Ahrefs KD, yet Mangools is not super precise either.See below their undervalued scores for a sample of highly difficult terms (US):

Keyword (US)

semrush_KD

mangools_KD

lakers

100

gmail

100

youtube music

www google com

100

white house

firefox

100

google hangouts

100

coronavirus usa

Below are some undervalued KD estimations from Mangools for Germany.

Keyword (DE)

semrush_KD

mangools_KD

weather

bbc

coronavirus news

100

covid-19

100

google com

100

risikogebiete

100

münchen

100

As we mentioned above, Moz KD again shows strong signs of dependence on a domain’s Authority Score from the top 10 results, which leads to some aberrations when calculating KD.

Keyword (US)

semrush_KD

moz_KD

chiefs

bears

dodgers

patriots

aapl stock

trump news

american airlines

youtube downloader

amzn

cheap flights

google hangouts

100

For Germany, Moz would show a significantly low KD for this selection of highly competitive keywords.

Keyword (DE)

semrush_KD

moz_KD

aldi

bmw

borussia mönchengladbach

währungsrechner

coronavirus aktuell

aldi online shop

deutsche bank

formel 1

bmw aktie

google

coronavirus deutschland

amazon aktie

apple aktie

übersetzter

fc bayern news aktuell

Serpstat’s KD showed the most aberrations both for the US and other regions. Below are some results for high-volume keywords in the United States:

Keyword (US)

semrush_KD

serpstat_KD

ups tracking

gmail com

new york times

100

adobe acrobat reader

manchester united

lewis hamilton

weather com

Below are the results for Germany, where Serpstat KD shows “easier” values for highly competitive keywords:

Keyword (DE)

semrush_KD

serpstat_KD

ikea

15.65

englisch deutsch

15.82

google übersetzer

11.08

bayern münchen

14.23

coronavirus

13.13

munich weather

9.35

fc union

6.65

What this tells us is that in regions other than the US, our competitors are missing a lot of the opportunities you’ll find in Semrush.

Try out the new keyword difficulty score today and feel free to always let us know your thoughts and feedback at keyword-research-feedback@semrush.com.

If you want a quick PDF with the details of our KD score, you can download this PDF.

Innovative SEO services

SEO is a patience game; no secret there. We`ll work with you to develop a Search strategy focused on producing increased traffic rankings in as early as 3-months.

A proven Allinclusive. SEO services for measuring, executing, and optimizing for Search Engine success. We say what we do and do what we say.

Our company as Semrush Agency Partner has designed a search engine optimization service that is both ethical and result-driven. We use the latest tools, strategies, and trends to help you move up in the search engines for the right keywords to get noticed by the right audience.

Today, you can schedule a Discovery call with us about your company needs.

Source:

Blog