History is recorded moments in life. People’s thoughts, either great among others or just great in their own shadow. Every glimpse of life at that just moment, deserves to be treasured. So just “Keep That Thought”…
By Richard Kirk, November 7, 2012
Keyword research is hugely important. The success of your entire SEO engagement depends on getting this piece of work right.
Why? Because an informed client will judge you on how visible their website becomes for the terms in their SEO keyword list, and that list is the product of your keyword research.
To improve keyword research, start with a really tight definition of what successful keyword research looks like. In order to say you’re a keyword research ninja, you should be able to look over your completed keyword list and say the following:
Kind of sound like you? Here’s some further acid tests:
Another prerequisite of improving keyword research is to face up to what you’re currently doing that is holding it back from being the best it can be. So much of keyword research relies on “gut feeling” so the term “dirty keyword research” is a fitting term for any process that involves making decisions in your lower digestive tract.
The classic example of a “dirty keyword research” process would be something like this:
This process is bad. It doesn’t look at anything other than potential audience, it certainly doesn’t favor the long tail or secondary keywords where we all know massive gains can be made, and fundamentally it sets you up for a big fall further down the road when your client doesn’t get the results they may have expected.
Here are some of the other major pitfalls of dirty keyword research:
It’s understandable why keyword research is compromised in many instances; you need to get it right but can’t spend forever on it – a client is unlikely to spend large amounts of money which enables you to spend 16 hours on keyword research for a normal site. Giving junior staff keyword research projects that will take them forever is pretty boring – the longer someone spends on something they aren’t enjoying, the more the quality drops off.
Imagine a keyword research process that:
This is important, because you can’t sell something your customer doesn’t understand.
Keyword research should always be broken down into two phases:
There is lots of information on keyword discovery; people love posting about this stuff as soon as new tools become available. However, keyword list refinement is all too often neglected, or it’s undertaken subjectively. The key to building a great keyword list is applying the same rigorous tests evenly to every single keyword in order to truly understand their worth.
The light bulb moment: Great keyword research involves making keyword list refinement robust and efficient through the use of an algorithm based on several keyword metrics.
Keyword research is usually required in two client scenarios:
The light bulb moment 2: You don’t have to have just one keyword research methodology. Different approaches should be used to get to the same great quality keyword list depending on the type of site you’re working with.
It’s so simple you can sum it up in one diagram!
Here’s a downloadable Excel template to help you execute the process described in the article. Here’s the process in a diagram.
How do you build a keyword list for a relatively normal site with limited amounts of template content? In these engagements you build a keyword list and then optimize individual pages for individual terms. Brand websites are a good candidate for this kind of keyword research.
Search volume data from the Google keyword tool is bullshit. You know it, I know it, the whole industry knows it. You look up the search volume for a term, then log in to the tool and do the same thing again – the search volume is completely different!
The simple truth is: you shouldn’t rely on individual datapoints coming out of Google’s keyword tool. It’s really there for illustrative purposes only.
On the other hand, it’s the only real source of data on any keyword; Analytics is only going to give you data on stuff you already rank for. So you’re stuck with the Google keyword tool as your comprehensive and easily-accessible data source.
You can use a simple equation to try and reduce our reliance on individual data points that come from the Google keyword tool; we call it “relative search volume”.
relative search volume = individual keyword search volume / total search volume for all keywords
By dividing the search volume for the individual term by the total volume, you get down to focusing on the proportion of the total audience rather than the raw number of searches. It allows us to place the search volume in context, which is crucial for properly judging how important it is to target a keyword.
Please note, in both forms of keyword research, it’s assumed that you have already been through the keyword discovery phase; as stated previously there are tons of good resources on this topic – check them out! Discovery can generally be broken down into discovery through the client or discovery through tools.
Discovery Through Your Account Manager/Client
Who better than the client themselves to help kick off the keyword research process? Just as we know our SEO, clients know their websites and the products and services they offer. They also know their brand and hopefully their customers and what they’re looking for when they visit the site.
In a discovery session you could cover these potential topics to aid keyword research:
Discovery Through Tools
You know the form here, we’re talking about the usual sources: Google Instant searches, Google Adwords Keyword Tool, paid search data, paid-for discovery tools (e.g. SearchMetrics Essentials).
Once we have the big-ass list we can get to work refining our keyword list to a useful size, with a priority order built in.
Standard keyword research is all about subjecting your long list of keywords to a series of tests based on keyword metrics. Tests have various scoring systems, the scoring of which can be weighted to reflect your opinion of how important a factor is.
When coming up with keyword metrics there is one golden rule: you have to be able to get all the necessary data for all your keywords in one sheet. If you have to query each keyword in turn to generate the data then it’s not a suitable keyword metric.
For instance, “difficulty of ranking on page 1” would be a great metric, available from several sources, the most well-known being SEOmoz. Unfortunately this data can only be fetched by querying each keyword in turn. This disqualifies this metric on the grounds of it being too manual, until such time that a tool is developed for getting this data for keywords in bulk (hint hint, developers!)
Here are some examples of tests we can run. Does the client:
And finally, does the keyword in question make up a large proportion of the total search audience?
The beauty of running several tests on your keywords is that it if a keyword isn’t suitable from one point of view, it can be from another – but everything is done in a consistent manner so you don’t need to argue with your team or your client over which keywords you’re going to target; they just rise to the top of the list naturally.
Even better, the structure of the table doesn’t need to change if you believe the system isn’t working, you can simply change the scoring system.
Ranking and Opportunity Based on Ranking
Setup: In a seperate tab paste your alphabetical keyword list in Column A, and the corresponding rank in Column B. If it’s not ranking, make sure the rank = 0.
If you’re ranking for a term, that’s a good thing! It means your site has a page that’s relevant for the term – so you don’t need to spend time creating something from scratch, and you may well be able to improve ranking through some simple optimization.
In this test we reward terms that rank. This is done through a simple if statement. If ranking > 0, score 1, if not, score 0.
We also run another test on this data – how much of an opportunity is there to get major traffic? We know traffic from a keyword increases exponentially as ranking approaches the top positions above the fold – spots 1, 2 and 3.
Therefore, we should prioritize keywords ranking just outside these spots; a small ranking improvement here could generate a big traffic increase. Using nested if statements, you can score keywords; 5 points for ranking 4 – 10, 4 for 11 – 15, 3 for 16 – 20, 2 for 21-30 and 1 for beyond position 30. All other positions get 0 points – including those in positions 1 – 3, as we feel there isn’t a huge opportunity to move the needle on these terms; either it’s too competitive to change rank or there are bigger gains to be made from getting above the fold for other terms.
Setup: exactly the same as for your own ranking data above.
In this scenario, we look at the competitor’s ranking and apply a negative score if a competitor is present on the first page of results. This is because we consider a keyword more of an opportunity if a direct competitor doesn’t rank on the first page. We usually choose the most visible site in the client’s sector as the competitor, so you know that in general you’re going to have more of a struggle on your hands if they’re on the first page.
In this scenario, if competitor rank > 0, we score the keyword -1, if not we score it 0.
Existing Anchor Text
Setup: paste a list of all the anchor text phrases used in external links to your site into a column in a separate sheet.
If someone is already linking to you using the exact same anchor text as the keyword you’re assessing, that’s a great signal that your site is relevant for that term, and that as a result it will be relatively easy to get more links with that keyword as the anchor text. Use a match formula to check if the keyword you’re assessing is an existing anchor text phrase. If it is, award 3 points, if not, the keyword gets 0 points.
vs. On-Site Search Terms or Paid Search Conversions
Setup: Same as for anchor text – grab the top terms which people searched within your site, or the terms from your paid account that drive conversions (you made need to exclude really low values) and paste them into new sheets.
If a user searches for a term once they have arrived at your site using internal search, that’s a pretty good indication the term is relevant. If one of our potential keywords appears in the top internal search terms, we award it 1 point.
It’s vitally important keywords that drive conversion are prioritized – if any of our long list keywords also appear in the top converting terms from paid search (exact match basis) we award 3 points. You could improve this further by penalising terms with a -1 score if the conversion rate for that term is low, as this would indicate the user’s searching for the keyword are not particularly relevant.
vs. Time on Site
Setup: Enable the non-paid search segment in your analytics. Download the top 500 keywords driving visits, and make a note of average time on site for the non-paid segment.
If the average time spent on site for a keyword is above the site-wide average for organic traffic, we score the keyword 1 point.
To find if the keyword you are testing exists within your user behaviour data and return the corresponding time on site or bounce rate figure use a vlookup formula, then in an adjacent column use an if statement to determine the keyword’s score based on its value relative to the average values you noted earlier.
If the keyword isn’t present in the user behaviour data, we score the keyword 0.
total keyword score = ((sum of scores from tests)-CPC from paid search)*relative search volume
By multiplying all the previous by the relative search volume you ensure that the most important factor in your keyword refinement algorithm is audience size. The great thing is though, you’re not focussing on absolute number of searches, but on the proportion of your total audience that will search the term in question.
The final step in producing your table should be to order all of the data by total score, and there you have it; a prioritized keyword list for your client. You can proudly state to your client that the reason you want to focus on the terms at the top of your list is because they are:
Even better, you can point to a robust methodology to back this up.
If you need to do a keyword list refresh after a few months, another benefit of this process appears: all you need to do is fetch fresh keyword metric data and rerun the formulas in your mastersheet – you’ll find the order of terms in your list has changed naturally as your algorithm reflects the fact that you have managed to get some terms into a great position.