Roughly speaking, search keywords are often divided into two camps – ‘top-level’ (or ‘head’) keywords and ‘longtail’ keywords.
In reality the distinction is not so clear of course, with there actually being a fine spectrum of keywords running from the very generic (often one-word) terms, to the very specific (arguably unlimited-word) search terms. We are often encouraged, or prompted, to loiter in the top-level vicinity of the keyword spectrum, wary of venturing too far into the infinite forest of longtail.
And there is good reason why people are drawn towards top-level keywords. They can often provide the highest levels of traffic and revenue in relation to the number of keywords targeted; the perceived authority - and with it trust - that can be gained by ranking for competitive generic terms.
But there is a great deal of value to be found in that dark longtail forest.
Here are a selection of reasons why I find longtail more attractive:
- The conversion rates for longtail keywords tend to be much higher than more top-level keyword as they are more targeted and the searcher is usually already further along the ‘Purchase Process’
- There are almost unlimited longtail keywords that can be targeted and can thus drive traffic to your site
- Competition for top-level keywords is getting harder at a much faster rate than longtail
- Longtail data can often tell you much more about site performance and about opportunities in the market. It stands to reason that the more keywords you can gather data on (such as CTR, conversion rate, bounce rate etc), the more informed you will regarding the performance of your landing pages and potential opportunities based on search trends.
Here’s a graph that you’ll have likely come across before showing the traffic comparisons between top-level and longtail search terms: 
For the sake of argument I have provided a clear separation between top level and longtail keywords, although as we know this doesn’t exist in reality. On this particular graph (Van Gogh I am not) the total potential traffic values (the red and blue bits) for top-level and longtail keywords look to be roughly the same. However, bear in mind that the longtail search terms are likely to be better converting, easier to rank for, and continue off the graph indefinitely, whereas there are finite top level terms. With this in mind I would argue that out of the two areas, the longtail is where most opportunity lies. So how do I tap into that opportunity? Or to continue with the forest metaphor - how do I draw the sap from the trees?
(this advice assumes you are already getting some traffic to your site and making at least some conversions)
1. Set up Ecommerce Tracking. Ecommerce tracking, in my opinion, is this most important element of keyword research. It doesn’t matter what analytics package you are using or what you are selling, you need to know which keywords lead to conversions otherwise you will cost yourself a great deal of unnecessary time and money. If you don’t sell anything directly on your site you need to track enquiry forms, signups or even offline leads, (whatever it is you attribute value basically)
2. Collect Data. After a few months of collecting ecommerce data you need to generate a report that shows all the keywords that sent traffic to your site along with their conversion rates, and better yet total revenue for each keyword.
3. Semantically Group Keywords. Next you need to semantically group the keywords together, i.e. group together similar keywords along with their conversion rates and revenue figures. I have customised software to do this for me, but you can also use Wordstream’s keyword grouper or the new keyword research tool from SEO Gadget to do this.
4. Highlight Top Groups. Look at the average conversion rates and revenue figures to work out which areas or groups of longtail keywords convert the best and bring in the most money. The reason we take averages across semantically linked keywords is because this improves trend accuracy and eliminates conversion anomalies.
5. Collate Your Next Target Keyword List. Extract and target those keywords from high performing groups that you do not already rank top for in the major search engines. Additionally, manually generate (and target) keywords that are closely linked to the best performing groups, but that you do not have data for.
6. Repeat from point 2.
This should be an ongoing process that over time will become increasingly refined and point you towards the keywords and keyword groups that your site converts best for, and that bring in the most money. Make sure you’re keyword-mining in the direction of the best performing keywords; target these, review your conversion stats, and then start the process again. This should also help you stop wasting time targeting keyword areas that do not provide good ROI.
The advice given above is admittedly not overly advanced stuff, but sometimes all it takes is a good methodology and encouragement that you’re on the right path, to help you begin to see the wood amongst the trees.