Finding your Search Engine keywords / phrases

January 20, 2006 3:30 PM

I'm looking for a way to find out what keywords/keyphrases are being used on all search engines for my website. My stats program shows a few but there must be loads that I haven't found yet. Thanks for reading.
posted by spinko to Technology (5 answers total)

If you have access to your raw log files, you can try another Log Analysis Program. The log files will contain a 'referer' field that gives the url of the referring page. In the case of most search engines, the url might include a bit like "?q=stuff+I+searched+for" this is the query text. Log analysis software finds these bits for known search engines, and tallies them up. If you know how to program, you could write a small application that just extracts that information from the log files.

Here's the Apache log file format.

If you don't have access to your log files, and your web hosting service won't provide them, then you are going to have to live with the existing data, use a 3rd party traffic counter service like sitemeter which may offer this info, or you mat be able to install a script on your own site (if you are using blog software there may be a statistics plugin).
posted by i love cheese at 5:10 PM on January 20, 2006

On Dreamhost, I had to log in to my stats section and tell it to give me the complete list of keywords in my daily reporting, not just the "top 10." Oh man, it's worth it, if anything just to see what people are searching for these days.
posted by visual mechanic at 7:12 PM on January 20, 2006

visual mechanic: I didn't even know I could customize that! Thanks!
posted by rhapsodie at 10:41 PM on January 20, 2006

If you'd like more stats from Google, you can make a Google Sitemap. I generate one (using this method) and then I can let Google show me which search terms people use.

Doesn't work for other search engines, of course, but then people tend to use the same terms everywhere.
posted by anildash at 1:13 AM on January 21, 2006

Response by poster: I currently have AWstats in my Cpanel software. It shows the keywords and phrases.

It would be a great idea for google to show key phrases on their custom domain. Like you can do link:www.yoursite.com or site:www.yoursite.com. C'mon Google do a keyword:www.yoursite.com.
posted by spinko at 3:39 AM on January 21, 2006