This blog is about the light that the cumulative searches of hundreds of millions of individuals can shine on the world in a way that traditional sources of insight cannot.

So what makes keyword research better than other research methodologies? It’s primary strength lies in it’s lack of bias. This impartiality is born of the intimacy that exists between a searcher and their search box that simply can’t be replicated at scale any other way.

For example it’s unlikely that if asked in a survey, many of the 165,000 global searchers using Google to find information about ‘flatulence’ in July 2012 would admit that it was their primary concern. Perhaps they might instead choose to align themselves with the more socially concerned (and fragrant) 60,500 people searching for ‘cure for cancer’ in the same month.

When a user enters their search they are speaking to a machine, they have a need and, as best they are able, they clearly and explicitly state that need.

These searches range from the mundane; “where can I buy Nespresso capsules” to the hilarious: “why does my mom smell”, to the potentially tragic: “test for aids”.

Whatever a searchers intention, every time a search is made it is added to aggregate statistics for the informed researcher to mine.

The strength of this new source of understanding is not only in it’s candour, it is also unprecedented in terms of it’s scale. Google with around 66% of the search engine market is queried 400 million times per day. Extrapolated to the whole search market that’s around 600 million searches, a sample size that few other research methodologies can hope to match.

Search data versus social data (Part 1)

Social networks such as (in Anglo-Saxon countries) Twitter, Facebook and LinkedIn are often portrayed as the modern mirror of the people.

The immense data held, particularly by Facebook, is often quoted as having the key to understanding people on a macro and individual level.

For example, Facebook knows where you live, who your friends, colleagues, family are, where you go for fun, where you go on holiday and your favourite TV shows. Surely this is the ultimate data set for understanding humanity on a grand scale?

Well, no, and here’s why.

Perception versus reality

When an individual creates content on a social network, particularly those where real names are encouraged such as Facebook or Google+, they are typically at least as conscious of the impact this will have on others perception of them as they would be talking in person with people they know.

The reason for this is that the average Facebook user has around 130 friends, but 7 close ‘real life’ friends, therefore any statement on Facebook is likely to reach a much wider and more diverse audience than one made in person.

As such, most social media users will screen themselves, conscious that their content may reach the eyes of family, co-workers, less close acquaintances and quite probably strangers and that each group of people may react in different ways.

For example political opinion expressed to 4 or 5 close friends is less likely to be challenged than one made to a diverse group of more than a hundred people from separate parts of one’s life.

Beyond the user’s direct connections, the ability for a particular piece of content to be shared is virtually limitless as a number of individuals writing indiscreet Twitter updates or posting Facebook photos have found.

Instead, individuals conduct themselves on social networks in the way they wish to be perceived by this broad community of people, rather than as they truly are.

In social, users update with socially acceptable facets of their life.
“I’m on the train”
“I’m looking forward to my holiday”
“My cat is adorable”.

It would be an unusual breach of convention for users of social media to ask where they can find at some good pornography, and yet that demand clearly exists, there are 277 million porn related searches from the comfortable anonymity of the search box every month.

And it’s not only sexual interests and personal hygiene problems that are directed at search and not social. If you are in need of information regarding a specific purchase, let’s say the purchase of a lamp or refrigerator, you are much more likely to start your search with a Google search rather than ask your friends who are relatively unlikely to have specialist knowledge about specific products.

If the average Facebook user has 160 friends, compared to tens of billions of indexed pages in Google and Bing, many of them written by niche experts and specialist retailers, it’s clear that online search is a more effective way of researching your needs.

To be continued…