Google search

January 31, 2007 2:53 PM

How do you know what Googlers were searching for?

I often see blogs saying things like 'people searching for ["Malcolm Fraser"+trousers] arrived here'. How do they know that?

posted by tellurian to computers & internet (10 comments total)

Trackers such as Site Meter tell them.
posted by randomination at 2:54 PM on January 31, 2007

The browser sends a variety information to the web site. It is rather benign to all but the most paranoid and includes the site you were previously on. A search from Google will produce the following HTTP string:

http://www.google.com/search?q=metafilter&ie=utf-8&oe=utf-8&rls=org.mozilla:en-US:official&client=firefox-a

It is obvious that "metafilter" is the query term. Because of the uniformity of these strings it is a fairly academic practice to create a variety of programs to sort and present these in a more accessible manner.
posted by geoff. at 3:02 PM on January 31, 2007

Every time your browser makes a request for a page, it sends a bunch of data to the server. One of the lines that is often sent is the Referer: header. This should contain the URL of the page that linked to the new page.

So, they just parse out the Referer: headers that look like google search results pages.
posted by Leon at 3:05 PM on January 31, 2007

At a more basic level, if you do a search on most search engines, the search terms are part of the URL of the page showing search results.

So, a search for ["Malcolm Fraser"+trousers] has a results page with a URL that looks like this:
http://www.google.com/search?hl=en&q=%22Malcolm+Fraser%22%2Btrousers&btnG=Google+Search

When someone clicks on a link in the search results, their browser reports the URL of the page containing the link to the server holding the new page as the REFERRER.

The server can analyze the referrer URL to extract the search terms involved. It's also possible for javascript in the page to extract that information and submit it to a 3rd party server like Sitemeter.
posted by Good Brain at 3:09 PM on January 31, 2007

Trackers such as Google Analytics tell them.
posted by allkindsoftime at 4:04 PM on January 31, 2007

allkindsoftime, I use Google Analytics on my blog and the reports I look at let me know how many visitors came from Google but not what they searched for. Is there another report I should be looking at?
posted by tellurian at 4:17 PM on January 31, 2007

The google webmaster tools tell you what terms people used to get to your site, and how high you ranked for those terms when they did use them to reach you.
posted by mathowie at 4:50 PM on January 31, 2007

A whole bunch of firefox extensions allow you to block the referer (that spelling is in the HTTP standard) being sent.

https://addons.mozilla.org/firefox/953/
https://addons.mozilla.org/firefox/667/
https://addons.mozilla.org/firefox/1093/
https://addons.mozilla.org/firefox/1306/

Currently, none of them automatically replace it with a Google search URL with randomly generated scurrilous search terms, sure to make any blogger reading his or her logs worry about the searches for which he or she ranks highly.

Yet.
posted by Zed_Lopez at 5:07 PM on January 31, 2007

Is there another report I should be looking at?

They refer to this kind of search as an "organic search". So it's under Marketing Optimization > Search Engine Marketing > CPC vs Organic Conversion. If you expand the "google [organic]" it shows you keywords. There's probably another way to get the data too. Analytics is complicated.
posted by smackfu at 5:17 PM on January 31, 2007

This shows up in my site logs, which I can find by logging on to my server. The logs show every referrer to my web site. These days my web site generally accessed via a couple of links, a moderate number of RSS readers, a buttload of random google and yahoo searches, a buttload of search engine robots, and a small number of people who know my URL by heart. The searches generally result in a referrer URL that lets me know what was being searched for.
posted by croutonsupafreak at 7:21 AM on February 1, 2007