How can I get Google to re-index my site and how did it find it in the first place?

AmbroseChapel — Mon, 21 Aug 2006 20:28:47 -0800

How can I get Google to re-index my site and how did it find it in the first place? Way, way before it was ready, a site I was building somehow got crawled by Google.

The version that got crawled has "lorem ipsum" text everywhere and stuff like "catchphrase goes here" in place of actual content.

[I've learned my lesson now -- I will password-protect sites in future, or use robots.txt, or whatever. There's no point lecturing me on this aspect.]

So, question one -- for some reason it got into Google. How? It wasn't linked from anywhere, and we certainly didn't go to the form on Google which says "please list my website".

The only thing we did was change hosting companies. Did Google sense a disturbance in the force when the DNS records clicked over? It seems unlikely.

Question two -- how can I get Google to come back and re-crawl the site? I've joined their SiteMaps program in the hope that would help, but it hasn't. I've used the "please crawl me" form three times now, and more than a month has gone by, and still nothing.

I assumed it would happen after a couple of weeks, but it's getting a little embarassing. When you search for the site name you get "sitename.com -- lorem ipsum blah blah tagline goes here" in the Google results.

Is there any SEO black magic which I can employ to help, or should I just wait?

Two technical details which someone said might be affecting it, although I'm not sure I believe them:

The front page which got indexed is longer the front page, it's blank with a redirect to the actual front page (for futureproofing reasons, long story).
The page which got indexed is "index.html" but has server-side includes (I tweaked the server, I like it that way); but Google doesn't know that, the page is never referred to by name, only with a trailing slash.

Ask MetaFilter questions tagged with disturbanceintheforce

How can I get Google to re-index my site and how did it find it in the first place?