Keywords that click by Barry Welford inspired an exchange between Ruud and I resulting in Ruud emailing me and asking me to expand on my thoughts on the audience being less a part of the keywords discovery process. I think of it more of a case of SEO's loosing touch with their audience.
By the way it's not that I think tools like Google Insights and others that provide behavioral data or concepts as Barry put it are not useful, quite the contrary, I just need a reason to use them. Generally if I feel uncomfortable with the resulting strategy from the keyword discovery I will try other tools. At that point I still prefer to go to the source, the audience.
Old School Keyword Research
Before SEOs had a gazillion keyword research tools our only available sources for keyword discovery information were the audience, what we had between our ears, referrer logs (with every request listed) and a few search engines had "suggestions" which were indicators of the query space.
I learned audience search behavior because with nothing else I had no other choice. Second generation SEO's had a keyword research tool via Overture so seldom or never were exposed to the need for observing the user search process. Data drives their campaigns with often ill conceived initial keyword targets.
I relied heavily on watching how people search to understand the query space and how their choices affected the query suggestions the Search Engines were providing above results. As I watched them I asked why they chose the input they did which indicated the thought process while refining the search.
I asked the client and if possible a person in "the trade" and lastly the least internet savvy person I knew how they would search for the product etc. we were marketing.
Keywords Research Today
SEO has become too fixated on data during the keyword discovery phase to truly understand the audience, engage it (landing page) and guide the customer towards a conversion. Today SEO to some in the industry is somewhat like telemarketing and junk (e)mail/response/direct marketing, just another numbers game. I can say that because, other than UCE email, I've played all those games.
The line between Social and Search is blurred with the majors implementing Real Time Spam, errrr Search. With the addition of the Twitter Stream the lines separating Social and Search may be erased entirely.
Social Media & Search
Social Media is about engaging an audience of real people, SEO is about crawlers, indexing, people searching and Search Engines providing a variety of content in various media types from many content pipes or search verticals to hopefully satisfy the searcher's needs.
Now when customers search where they are in the conversion funnel/purchase process may have changed from where it was a few years ago. These are the unknowns that become clearer with time and study. SEO's need to re-learn the user search process.
The Social Soup
It is not just Facebook and Twitter but Social networks in general. To confuse things even further smarter internet aware devices have added "ubiquitous connectivity" to the chaos. Stir in local and a little mobile search and you have a hyper local soup of Twitter/Social, local and mobile looking more viable than ever!
Now I often use keyword tools to seed my keyword discovery. One of my favorites is Google's Adwords keyword tool which includes a crawler option which is very useful for SEO.
I will be reviewing the new addition to my tool box the Wordstream free keyword research tool at the end of this post.
I do less of the the audience "one on one", however, when I do it is usually because I'm not comfortable with my understanding of the business/service/site or query space and I am double checking to be certain my strategy is sound.
I don't have to do this as much now because I understand the customer search/thought process, therefore, I understand that a "top term" in total queries may not be a top converter justifying top priority in the SEO process. That is what the audience contact was for. Learning the query refining approach of the audience.
In the past I have used Aaron Wall's keyword tool, the Google tool mentioned earlier and the Wordstream free Keyword Tool. I was fortunate enough to be a beta tester for two new tools by Wordstream developers.
In the name of full disclosure I have been contracted to do blog posts for Wordstream, and at this time am in the process of fulfilling that obligation. Wordstream had no input into my review beyond providing access to the tools. Whether I reviewed the tools was left entirely up to me.
As I mentioned I do use a variety of keyword research tools in the discovery phase. I use the Wordstream tools to prioritize the keywords list based on my belief in the terms value to the site, the sites ability to meet the needs of the searcher/customer which is heavily influenced by product mix and price point and of course the number of queries.
To some degree that is difficult because the Wordstream tool doesn't show actual query numbers but rather representations. If I had actual numbers I'd take a page from Dan Thies book and weight the keywords using percentages based on the terms value to the site. I have two spreadsheets with my macros in place to provide an SEO and PPC worksheet.
The Niche Finder is simply one of the best tools I've seen for quite some time. One thing I truly love about the free keywords Tool is it provides far more keywords then any other tool I've used outside of the Google tools.
The Niche Finder takes this huge list and breaks it into many lists that are great for wireframe and Information Architecture modeling and building PPC campaign and AdGroups. The later would have saved me hours of work back when I was managing 000's of keywords in PPC campaigns.
The Keyword Grouper takes a comma or tab delimited list and makes new terms from the words in the list. This tool was clumsy to work with and quite simply the terms often made no sense at all.