There’s a very interesting post by Robin Goad over on the Hitwise blog about the change in distribution of search terms on search engines. As more and more people are searching online, two things are happening to the range of search terms, one intuitively obvious, the other somewhat counter-intuitive.
The obvious change is that the total number of different search terms used is going up. If you think of the distribution of search terms as a head/tail curve, this means that the tail is getting longer. The non-obvious change is that the number of terms that make up the top 5, 10 or 20% of all searches (i.e. the most popular terms, or the “head”) is going down.
Robin’s charts are a little tough to parse, so I’ve taken a crack at simplifying them a little. Firstly, the chart below shows the change in breadth of search terms in the top 5%, 10% and 20% of search traffic. I’ve normalized the vertical axis (2005 = 100) to highlight the proportional change. You can see that in the top 5% group, the overall number of search terms fallen by over 80%.
This implies that a few very popular search terms are really starting to dominate the traffic. Robin goes a stage further and separates out “navigational” search terms, to produce the following chart. Here the drop-off in diversity is even more marked (note that the buckets in the chart below are different to those in the one above), with an average drop-off of around 80% even up to the 10% point.
What do we mean by navigational search? Searches for sites by their own name; for example, people trying to find the British Airways site by searching on “British Airways”.
What this data tells us is that these brand or navigational search terms are starting to crowd the top of the leaderboard for searches, with people using them as proxies for remembering the URL of the site itself.
The reason this is interesting to online marketers is that sites are increasingly having to pay attention to these search terms – and some sites are choosing to buy their own company name as paid search to ensure that visitors click through to their site when they search on their company name. Look again at the search results (linked above) for “British Airways”. The first sponsored results is… an ad for British Airways, despite the fact that BA’s site is first in the Organic results, just a couple of lines down.
Apart from the fact that having to do this is likely costing BA a fair amount of money (which they could instead be spending on me when I fly to London at the weekend), it’s likely to skew BA’s picture of how their online marketing mix is really working for them. A lot of users who have been researching online (especially for something like flights) will use a navigational search term to return to the site where they’ve decided to purchase. Because most analytics tools use a “last-click” attribution model for conversions, BA’s reporting on marketing effectiveness is likely to overstate the relative importance of those keywords, when it may really have been other keywords (or other kinds of marketing altogether, such as e-mail) which drove the visitor to the site in the first place.
So what are brand owners to do in this situation? They don’t want to drop their navigational search keyword campaigns because they’ll lose clicks and business, but on the other hand, buying navigational terms seems like a bit of a tax for these kind of sites, and distorts the numbers. Part of the answer lies in the rules the search engines impose about bidding on other companies’ brand names (though this has caused all sorts of misery with Live Search and adCenter), but the true answer lies in a smarter attribution model for the sites involved.
In addition, sites should group branded or navigational search into a separate bucket, and take conversions that are attributed to those terms with a pinch of salt. I would even recommend that these conversions be not included when calculating the overall ROI of paid search, and instead be thought of as part of the cost of more brand-focused marketing activities such as TV. What do you think?
Interesting post.
1) Yes, increased concentration of top search terms may not be intuitive. On the other hand:
If people are “perfectly” networked, we should expect spikes in behaviors — everybody will search for the term-of-the-day. We should also expect a high churn rate. As the data is computed for 4-week intervals, yes, it does make perfect sense that concentration is increasing.
2) I totally agree on the tactical importance of “navigational” search terms. A few months ago I had (indirectly) reacted to one of your posts, by questioning the wisdom of making a distinction between “organic” vs “paid” search. I argued that for consumers, organic links are qualitatively different from paid links. Organics appears to be more legitimate, and paid more biased.
Based on what we think we know about consumer behavior, I would suggest investing is SEO for broad search, and paid links for specific search. Consumers using broad search terms are presumably at the opinion formation stage, and much more likely to value information from a presumed independent source. Consumers using specific search terms are probably in a “hunting mode” and should be presented with our value proposition, even if that proposition was not the one they had in mind.
To illustrate, a consumer wants to buy a car. This consumer will search here and there for alternatives, suggestions and so on (e.g. best compact car). I would not advise that a manufacturer invest massively in paid search at this point. Ideally, you make shure that your brand has good reviews and do your best to put these reviews in front of your prospects’ eyes. You may achieve this goal by massively pointing at those favorable reviews to make them rise above the rest.
Consumers reach a conclusion and then search for a specific brand to get pricing info. That would be the time to rely on paid search, to draw those consumers to your retail site, or to propose strong alternatives (eg. autonation, or “hybrids help the planet”).
And yes BA has better buy navigational links because otherwise I am a very good candidate to poaching by competitors (who may even promote discounted prices through paid links triggered by the british airway (typo intended) query).
Thanks for the pointer: it’s very useful to see some evidence to confirm this trend and get some idea of the scale.
Not so long ago I can remember people sneering at this behaviour and suggesting that only naive users would type a prominent brand name, or even the url itself, into a search engine. But it’s always been a fairly sensible approach — let Google and co sort out any spelling mistakes and typos. They’re really good at it.
I think you’re absolutely right to encourage people to treat ‘brand’ searches as a special category. I urge the people I work with to try to formalise the process by, for example, re-writing all the brand keyword variations as just ‘brand’ in most views of the data. That way they get three wins: the useful (often huge) summary total, a much clearer view of the generic terms, and the most important — a clear reminder that this group is in some way different.
And should be ‘taken with a pinch of salt’ in your words.
But trying to go beyond that general health warning to get at the likely multi-session history of ‘broad generic search’ then ‘slightly more specific refined search’ to the conversion on the ultimate ‘navigational/brand search’ still seems a bit of a dream to me. I use Indextools on one site, and they have a serious go at it. But the real problem comes back to user behaviour. In this case the absence of the cookies to connect the visits — not because of deletion but because of the increasing use of multiple devices to make those multiple visits.
So what I’d love to see from somewhere is a serious attempt to measure THAT theoretical change in behaviour. The increased likelihood that an individual person will use several different machines for their browsing during any given week.
That sounds like an impossible challenge. How on earth to do such a thing? But meanwhile we continue to clutch at the opposite straw in the form of faith in our ‘new vs returning’ reports and skewed ideas about the conversion rates for generic and brand searches.
It occurs to me that maybe Hitwise themselves could come up with at least one indicator based around patterns of use suggesting multiple machines behind one domestic connection. Even a trend indication of the decline in families all sharing one home computer would help measure out that pinch of salt.
Great post, very insightful.
In the past I have watched what I would consider “less savvy” users typing in the brand name into Google as a way of getting to the site. I too scoffed at this approach as elementary. But as options online increase–and website URLs become less obvious–I too find searching the brand on Google the easiest route.
I especially appreciate your insight about inflated reporting on brand keywords. We just got a report back from our advertising partner that our brand keywords are working best and we should continue buying them. Your post has given me some additional background that can help me more holistically consider their recommendation and our focus moving forward.
THANKS!
Quintin Snelson
Enjoyed every bit of your blog.Really thank you! Want more.