Parsing the Google referral string in a post (not provided) world

Note: As March 2016 Google is no longer passing this information in the referral string.

 

As of September 2013 Google prevented site owners from seeing all organic referring keyword data in the referral string.

However there is still plenty of data to be gleaned from the string. For quick testing the HttpFox Firefox plugin is excellent. Systematically capturing the data is easily done in any web analytics tool or server log parser using simple Regex.

It’s important to note that this data appears not to be passed from mobile searches which may somewhat skew any conclusions.

1) The rank of the link that the user clicked

To understand the rank of the result the user clicked to arrive at your site, look at the ‘cd’ key / value pair. e.g.

http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CC8QFjAA&url=http%3A%2F%2Fwww.ebay.co.uk%2F&ei=oe9GUtLDMJPT7AbkZw&usg=AFQjCNGEltn-KekW3pKDE9fUDb2NKpKbWw&sig2=BecHxquQThrLWIGjk9VEMA

cd=1 indicates the clicked listing was in first place, cd=3 third place etc.

It does however get more complex when authority links and universal search are included on the Search Engine Result Page (‘SERP’), which will happen in most cases.

In this case the universal search results are counted in the SERP and must be considered e.g. in this case it’s possible to have up to a cd value of 16 on page 1.

CD variable by SERP result
Orange numbers represent the ‘cd’ value

2) The type of link clicked (search, news, image etc)

The ‘ved’ parameter indicates what type of result has referred a visitor to your site.

http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CC8QFjAA&url=http%3A%2F%2Fwww.ebay.co.uk%2F&ei=oe9GUtLDMJPT7AbkZw&usg=AFQjCNGEltn-KekW3pKDE9fUDb2NKpKbWw&sig2=BecHxquQThrLWIGjk9VEMA

This has been well documented by Tim Resnik in this excellent post on moz.com.

Here’s a marginally more verbose version of Tim’s table. Note these are substrings of the total value;

VED Value This means
QFj A normal organic search result
QqQIw A news OneBox link (e.g. 11, 12 & 13 above)
QpwI A news OneBox image (e.g. 11 above)
Q9QEw Video OneBox link
Qtw1w Video OneBox image
QjB An authority link (e.g. #2 – 4 on the screenshot)
BEPwd Knowledge graph image
BEP4d A secondary Knowledge Graph image

 

3) The local version of Google searched by the user

This is straightforward, you can clearly see the Top Level Domain (TLD) of the Google search that referred the visitor. In this example you can see Google UK;

http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CC8QFjAA&url=http%3A%2F%2Fwww.ebay.co.uk%2F&ei=oe9GUtLDMJPT7AbkZw&usg=AFQjCNGEltn-KekW3pKDE9fUDb2NKpKbWw&sig2=BecHxquQThrLWIGjk9VEMA

To simulate this quickly, try the Search Latte international search tool.

4) The landing page URL

The ‘url’ variable is another nice easy one to decipher;

http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CC8QFjAA&url=http%3A%2F%2Fwww.ebay.co.uk%2F&ei=oe9GUtLDMJPT7AbkZw&usg=AFQjCNGEltn-KekW3pKDE9fUDb2NKpKbWw&sig2=BecHxquQThrLWIGjk9VEMA

Note the address itself is character encoded hence; http%3A%2F%2 represents
http://.

5) Is the user logged in to Google?

Finally the ‘sig2’ parameter only appears whe a users is logged in to Google, therefore you can determine the proportion of users arriving at your site authenticated with Google.

http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CC8QFjAA&url=http%3A%2F%2Fwww.ebay.co.uk%2F&ei=oe9GUtLDMJPT7AbkZw&usg=AFQjCNGEltn-KekW3pKDE9fUDb2NKpKbWw&sig2=BecHxquQThrLWIGjk9VEMA

 

What does any of this mean?

Obviously the loss of the referring keyword is a blow to the accuracy of any SEO reporting. But the above will at least allow site owners to answer questions like;

  • Does traffic from different ranks convert at different rates?
  • Does traffic from different types of search result behave differently?
  • What proportion of visitors arrive at your site from different local versions of Google?

Tracking a brand with keyword research

Historically brands have invested heavily in understanding the ‘awareness’ of their product. This is a time consuming, expensive and often statistically dubious process.

Querying search engines for the number of times a brand is searched for can provide a significantly better gauge of a brand’s awareness among it’s target population.

Let’s take as a simple example the Canadian airline market. Canada’s commercial airways are a two-horse race between Air Canada & WestJet who vie for the domestic market.

In June 2012 there were 368,000 Google searches from within Canada containing ‘westjet’ versus 2,740,000 containing ‘air canada’. So Air Canada appears a clear winner.

But a closer look reveals that there were 1,830,000 additional searches containing ‘west jet’. Not the brand’s approved name but nearly 5 times more popular.

So if we compare again but this time including the additional ‘west jet’ searches the score is 2,198,000 for WestJet versus 2,740,000 for Air Canada, still lower but much closer.

The lesson here is that consumers can’t be relied upon to search for your approved brand term, always consider including misspellings. This is particularly the case if you have a name which can be easily misunderstood.

For example mobile phone retailer Phones4U has a weird and wonderful presence in the mind’s of consumers.

Phones4U brand terms UK August 2012 Broad Match Google Searches
Phones4U brand terms Search volume
phones for u 823,000
phones 4 u 673,000
phones 4u 673,000
phone 4 u 550,000
phones for you 246,000
phones 4 you 74,000
phone4u 33,100
fones 4 u 6,600

 

Or if you have spent years educating your market about a different name, witness the 5,500 searches per month for ‘midland bank’ despite former bank ‘ The Midland Bank’ being wholly subsumed by the HSBC in 1992.

Geography

Every time a search is made it comes from an IP address which gives a very approximate indication of the searchers location.

Armed with this Google can also represent the interest in a brand, relative to all the other searches being made, geographically.

July 2012 Google Phrase Match searches for ‘west jet

A heat map showing July 2012 Google Phrase Match searches for 'west jet'
This heat map, perhaps unsurprisingly, shows how WestJet is stronger in the West of Canada, it’s home market.

Caveats

An important caveat is that many people will have cause to return to your website for transactional reasons e.g. an online backing customer checking their account, or an airline customer checking in.

Thus brand term searches don’t always accurately represent your brand’s popularity versus a brand with out similar online functionality.

Because of this lack of visibility into the ‘frequency’ per member of the searching population the number of brand term searches is best used as an abstract measurement between similar businesses.

Also, as ever, it is also important to consider synonyms. To use the hackneyed example – while ‘apple’ might represent the consumer technology behemoth, it may also be a search for a record company, a flavor, or even a humble piece of fruit.

Conclusion

Brand term search is, used carefully an incredibly useful, statistically valid and near free indicator of your brand’s popularity versus that of a competitor.