Were Britons really googling, "What is the EU" after they voted on the referendum? Here's how a misuse of search data and a misunderstanding of search intent formed a perfect media storm.
Disclaimer: I'm an Englishman resident in the United States who happens to practice the dark arts of SEO for a living. The misinterpretation of search data and misunderstanding of search intent is the bane of my existence.
At 7:25am (11:25am UTC/GMT) on Friday morning, Google Trends tweeted the following:
Across the globe, online media ran with a similar narrative, namely that the UK had voted to leave the EU without knowing what the EU was. See these two articles from two of the most recognized media outlets:
First, The Washington Post:
The WAPO article, running with the Google Trends tweet, pontificated: "And although leaders of the campaign to exit Europe are crowing over their victory, it seems many Britons may not even know what they had actually voted for."
Again, journalists ran with the Google Trends data and wrote: "But if you judge a country's interests only by prevalent Google searches, it was after the polls closed when British voters started to think seriously about the implications of their choice."
Those are pretty high indictments being levied against a population.
And, I'd say, those indictments were balderdash.
This is a classic case of the misuse of Google Trend data combined with a misunderstanding of search data. The result is wildly audacious claims coming from reputable sources. That's a big problem.
So, What Was The UK Actually Googling?
If we look at Google Trends for 'What is the EU' for most of Friday (6/24) and compare data from the UK, US, France, Germany, and Italy—the search would appear to trend higher in the States than the UK.
By WAPO's logic, this would mean America is more clueless than the UK. By NPR's logic, it would mean that the US was more concerned about the implications of the vote than Britain was.
Google Trends Data Does Not Equal Search Volume
The first thing we need to understand is that Google Trends is not an indicator of how many people have searched for something. According to the Google Trends documentation, GT:
- Analyzes a percentage of Google web searches to figure out how many searches were done over a certain period of time
- Adjusts search data to make comparisons between terms easier
While two regions show the same number of searches for a term, that doesn't mean that their total search volumes are the same.
Is there a way to find out Google Trends search volume?
Not exactly, but we can look at Google Keyword Planner to get a sense of historical search volume.
In the UK, there have been 3,600 'what is the eu' exact match searches per month over the past year, with 9,900 in May 2016. I'll let you decide if that's a lot for a country of nearly 70 million.
I decided to take a look at comparative search volumes for the month of May in other English speaking countries:
New Zealand: 70
United States: 3,600
Now, look at what we get if we compare "what is the eu" for the last 30 days:
Exact Match Search
This is an exact match search—meaning people are searching for 'what is the eu' as a complete phrase—with maybe a word before or after the phrase. New Zealand has the biggest relative mindshare, but that certainly has nothing to do with actual volume.
Broad Match Search
This is 'What is the EU?' searched for over the same time frame against the same set of countries in Google Trends without double quotation marks, meaning the words can be split up and include other terms as well.
What this tells us is that the UK was actually searching for broader terms than merely "What is the EU." For example, "What is the EU result" or "What is the EU referendum result" would also be included in a broader trend search.
No other country trends for 'result' modified searches:
In fact, this shows us nothing more than a nation waking up and wanting to know if they'd left the EU.
By Google's own definition of ways to filter searches in Trends, 'What is the EU?' searches could include additional terms depending on how the phrase is searched for in the tool, which is exactly what we see in the data.
Google as Self-Fulfilling Search Prophecy
Finally, it's extremely interesting over a seven-day period to see the rising and breakout (+5,000%) topics globally. We see a 1,800% increase in 'What is the EU?' traffic related to topical Google Trends along with The Washington Post.
This means that interest in the story drove Google Trends spikes for 'what is the eu' searches after the original tweet. For the UK in particular, the top two breakout queries (+5,000%) were people searching for direct references to the search itself. Google Trends as a topical entity relating to the search is seeing a 2,000% increase.
The TL;DR Cliff NotesThat is all quite a bit to absorb, I know. And perhaps too far in the weeds. Here are the core takeaways:
- 'What is the EU?' exact match searches were trending less in the UK than other English speaking countries
- When factoring in broader match searches with modifiers such as 'What is the EU result?', 'What is the EU referendum result', etc., there is a considerable uptick in search interest around 'What is the EU?'
- Is it unethical for Google to drive search trends? Or, is online media to blame for not being able to understand data?
I want to be clear that I'm not suggesting Google fudged any numbers. What I'm saying is: the numbers are sufficiently small for the article to be seen as a considerable driving force of search interest for the term.
To understand search, you need to understand the intent of the query. In this instance it seems easier to understand the intent of a search engine currently embroiled in three antitrust complaints with the EU.
So, the real question is, "What is the EU going to do with Google?"