The Google Files, Part II

Where we last left off, I showed you a chart. It’s from a tool called SEMRush, which SEO professionals swear by. 

We don’t have access to Google’s internal documents, but we have access to something even better: a historical snapshot of organic search rankings over time.

How to Read Historical SEO Rankings

Here’s how SEMRush and other tools like it work. It does millions of Google searches every day on all kinds of keywords and captures what the Google organic search results page looked like on that day and captures all the sites ranking in the top 100 for that search term. Over the years they’ve built a huge database of data.

Is this legal? Of course, because SEMRush never access Google’s internal data. It gets all of its data from the same public search pages that you and I and millions of others see.

The result? You can see how any individual site has performed in organic search for any keyword over any period of time (going back to around 2015).

SEO Performance of Conservative News Sites from 2016 to 2020

This particular graph shows the SEO performance of Breitbart.com, one of the leading conservative news sites. The blue line doesn’t show SEO traffic, it shows the number of keywords that Breitbart.com ranked for in the top 100 of Google results over time.

Right away, you can see a few clear trends.

1. In October 2016, prior to the 2016 Presidential Election, Breitbart.com ranked on SEO for over 1,024,489 keywords.
   1. 49.2% of these keywords were “branded”, meaning about 500,000 people searched for phrases containing the word “breitbart”. For example, “breitbart news”, “breitbart website”, etc.
      
      The other 500,000 searched for phrases that did not (“non-branded”), such as “big government”, “benghazi coverup” and “hillary clinton latest news”.

2. By October 2020, prior to the 2020 Presidential Election, Breitbart.com ranked on only 64,027 keywords.
   • 99.7% of the terms were branded, 0.3% were non-branded. Unlike in 2016, Breitbart.com was essentially invisible for any other Google term with sizable interest. 

Prior to 2018 I’d chalk this up to bad SEO. Normally in the SEO space when you see a collapse like this, it means one of two things:

1. You messed up something in your technical SEO
2. Someone in Google changed the algorithm.

Heaven knows that I’ve seen hundreds of sites in my day where the site owner would complain of Google punishing them, when all they had to do was make simple technical changes.

And when I look at Breitbart.com and most conservative news sites, yes, I do see SEO improvements they can be making. But nothing that would warrant a 94% collapse in search visibility.

The story becomes more clear when you look at the bigger picture. Something catastrophic happened to conservative news sites between the 2016 and the 2020 elections. Here’s a sample of other sites and what happened to them and their Google visibility:

Newsmax.com

October 2016: 958,325 keywords, 21.9% branded, 78.1% non-branded

October 2020: 134,513 keywords, 85.7% branded, 14.3% non-branded

TheBlaze.com

October 2016: 601,042 keywords, 63.2% branded, 36.8% non-branded

October 2020: 102,849 keywords, 95.1% branded, 4.9% non-branded

DailyCaller.com

October 2016: 822,804 keywords, 18.2% branded, 81.8% non-branded

October 2020: 79,905 keywords. 97.7% branded, 2.3% non-branded

DailyWire.com

October 2016: 143,630 keywords (as a new site in 2016, the Daily Wire had no branded searches yet)

October 2020: 36,102 keywords, 98.3% branded, 1.7% non-branded

Some people might point to the rise of keywords from 2021 to 2024 and claim, “see, Google isn’t biased”. But today 86% of the Daily Wire’s SEO continues to be from branded search. In other words, that rise is entirely caused by The Daily Wire’s effectiveness as building their own brand awareness, not from Google allowing them to rank for non-branded searches.

TheGatewayPundit.com

October 2016: 109,670 keywords, nearly 100% non-branded

October 2020: 36,102 keywords, 98.3% branded, 1.7% non-branded

I could go on (and will be happy to update this page with other sites if you leave them in the comments). But the pattern is clear:

1. At the time of the 2016 election, conservative political sites looked very similar to all other sites when it comes to organic search rankings. I’ll explain how in the next post.
   
2. By the time of the 2020 election, conservative voices were all but invisible in Google for any phrase. Sites could only survive by building up their brand names in other channels like radio, social media, and first-party Web sites, but organic search was completely dead.
   
3. All sites saw a radical shift in the ratio of branded to non-branded terms. Put another way, an undecided voter who was researching by entering normal keywords had near zero visibility of conservative news sites.

SEO Performance of Conservative NGOs and Advocacy Groups from 2016 to 2020

Let’s try a couple of right-leaning NGOs and advocacy groups.

JudicialWatch.org

October 2016: 23,662 keywords, 31.1% branded, 68.9% non-branded

October 2020: 3,876 keywords., 96.5% branded, 3.5% non-branded

NewsBusters.org

October 2016: 6,532 keywords, 1% branded, 99% non-branded

October 2020: 3,094 keywords, 83% branded, 18% non-branded

LiveAction.org

October 2016: 6,532 keywords, 99% non-branded, 1% branded

October 2020: 4,674 keywords, 82.4% branded, 17.4% non-branded

Notice a similar pattern. These sites were growing, some impressively, up to the 2016 election. But by the 2020 election, they were mostly invisible to Google Organic search, unless someone specifically heard about them and typed in their name.

Where did all the traffic go?

When you look at all these numbers (and I can provide hundreds of other examples) you can see clearly that many, many conservative Web sites systematically lost SEO visibility in Google. Sites that once attracted thousands of visitors suddenly went silent.

So where did all that traffic go?

Let’s see what was happening to mainstream news sites during that same time for many of these same terms:

CNN.com

October 2016: 8,197,919 keywords, 41.7% branded, 58.3% non-branded

October 2020: 27,216,646 keywords, 33.7% branded, 66.3 non-branded

NYTimes.com

October 2016: 15,981,426 keywords, 8.7% branded, 91.3% non-branded

October 2020: 50,404,384 keywords, 9.1% branded, 90.9% non-branded

NBCNews.com

October 2016: 2,281.945 keywords, 26.9% branded, 73.1% non-branded

October 2020: 5,190,110 keywords, 9.3% branded, 90.7% non-branded

NPR.org

October 2016: 3,509,559 keywords, 12.9% branded, 87.1% non-branded

October 2020: 6,860,077 keywords, 8.3% branded, 91.7% non-branded

WashingtonPost.com

October 2016: 5,034,698 keywords, 10.8% branded, 89.2% non-branded

October 2020: 12,563,051 keywords, 11.1% branded, 88.9% non-branded

There are countless others, of course, including CBS News. ABC News. Reuters, and AP, each with exactly the same patterns, which were:

1. Each site enjoyed a tremendous increase in SEO visibility between 2016 and 2020 off an already gargantuan base.
2. Each site had a disproportionate amount of “non-branded” keywords. Undecided voters who Googled generic phrases in 2016 got a diversity of opinion with which to make decisions. By 2020, they were shown a monolithic set of opinions.

How did this happen? We’ll explore in Part 3.