I want the opposite of search engine optimization for my blog

February 2, 2019 3:56 PM

I need to set up a blog and would like to avoid it being indexed by search engines. How can I accomplish this with minimal hassle and expense?

This blog is a student project and I'm not very up on web technologies (though I did create a simple self-hosted web site ages ago). If the service could be free or cheap, quick to set up, and have settings that make it pretty simple to prevent robots from indexing my blog that would be ideal. What's my best option? I've googled around, but not come up with much. Password protection might also be helpful, if preventing the web robots from indexing my page isn't possible.

posted by reren to Technology (7 answers total)

I’m not sure if you’re aware of this but by definition all ethical web crawlers respect robots.txt.

I’m not sure how to deal with the darker side of unscrupulous indexers, but this is a standard first step for staying under the radar. No service or product to buy, just write a little text file and leave it in the right place.
posted by SaltySalticid at 4:01 PM on February 2, 2019

In addition to a robots.txt file, add tags to discourage indexing. So, in your header, use:
<META NAME="robots" CONTENT="noindex,nofollow">
posted by pipeski at 4:02 PM on February 2, 2019

I don't want to threadsit, but to clarify: I know there are options for preventing indexing on a web page you've created, but I was hoping not to have to create a web site from scratch or purchase hosting or write code, etc. Could I use, say, blogger.com or any service like that, but prevent indexing? The class is about writing, not tech. I'd like to throw as little energy as possible into setting this blog up because I frankly don't care to have online readership for my student work.
posted by reren at 4:10 PM on February 2, 2019

Could I use, say, blogger.com or any service like that, but prevent indexing?

Blogger - Search Engine Optimization (SEO) tips for your blog

Hide pages from searches
You can hide pages or posts that you don’t want search engines to find.

Sign in to Blogger.
Click the blog you want to work on.
In the left menu, click Settingsand thenSearch preferences.
Under "Crawlers and Indexing," next to "Custom robots header tags," click Edit.
Choose Yes.
Check noindex.
Click Save changes

posted by zamboni at 4:25 PM on February 2, 2019

Yes, Blogger will do exactly what you are asking and for free.
posted by AugustWest at 5:08 PM on February 2, 2019

You could also create a private blog at wordpress.com for free. You would have to individually invite people to view it. The "hidden" option lets people view it but tells robots not to index it.
posted by belladonna at 6:38 PM on February 2, 2019

Thanks so much, everyone, for pointing out several good directions. I’m a little bashful that I wasn’t able to figure this out myself but very grateful!
posted by reren at 11:51 AM on February 4, 2019