How to handle Googlebot during site downtime

by Patrick Altoft on October 26, 2007

This week one of my clients was carrying out some web design work that required part of the site to be taken offline for several hours.
Google normally spiders the site quite a lot so the client didn’t want to mess up any rankings during the downtime.
The best way to manage this according to Google is to give spiders a 503 (Service unavailable) header code when they visit. This tells search engines that the problem is temporary and they will come back another time.

Serving the error message is easy if its just a static php page and you can even choose the number of seconds Google should wait before coming back.

<?php
header("HTTP/1.1 503 Service Temporarily Unavailable");
header("Status: 503 Service Temporarily Unavailable");
header("Retry-After: 3600");
php?>

Add a nice error message here

However sometimes you need to apply the code to all pages in the site, in this case you need to add this code to your .htaccess file

Options +FollowSymLinks
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} ^.*google.* [NC]
RewriteRule .* /errorpage.php

You should make sure your error page is giving the 503 header as shown in the php snippet above and most importantly test your code as soon as it goes live to check for problems.

Patrick Altoft is Director of Search at Leeds based digital & SEO agency Branded3. Patrick also runs Blogstorm.

You can get our blog posts delivered for free by email every day - simply add your email address to the box below or alternatively grab the RSS feed.

Read some similar posts

{ 4 comments… read them below or add one }

andy 27 Oct 2007 at 8:21 pm

Nice tip here. Thanks a lot. I had never thought about doing individual page downtime messages programatically. Nice work!

Anthony Williams 30 Oct 2007 at 9:43 am

Nice tip. So many websites return the wrong HTTP codes, especially when the pages are generated through a CMS. 503 is definitely much better than 404 — it says “come back later” rather than screaming “ERROR!!! BAD LINK!!!”.

This is not just good for Google, it’s good for visitors too, especially if you’ve written a decent explanation in the errorpage.

FileGets 31 Oct 2007 at 11:48 am

Nice tip! Very nice! I’ll save it.
But unfortunately this can’t solve problem then your host goes down and you can’t access it…

Dave 26 Jan 2008 at 10:34 am

I have always used the rewrite rule and holding page trick but it has never occurred to me to use a 503 header to really send the right message.

Apache will also send a 503 response when it reaches max_clients. It’s good to know that this won’t affect Google.

I’ve always wanted to use 410 Gone but I have never found the right time to use it. If I ever move content then I know where it is and send a 301 response… I don’t ever just delete something.

{ 1 trackback }

How to manage the website of a company in administration
01.20.09 at 10:59 am

Leave a Comment (registration is optional)

Registration is free, takes about 5 seconds and is worth doing.

You can use these HTML tags and attributes:
<a href=""> <b> <blockquote> <code> <em> <i> <strike> <strong>