Over the years we’ve tried dozens of ways to monitor rankings on the major search engines and none of them have ever been able to deliver accurate results, apart from Advanced Web Ranking.

I have very little patience with rank checkers that don’t show accurate search engine rankings because there is no point in reporting on incorrect information. We used to have an in-house tool for monitoring rankings but Google managed to spot all the IP addresses and started to return strange results. We also tried Raven and a few other hosted systems but none of them showed the same rankings we could see in the office. We decided that rather than try to reinvent the wheel we would just move everything onto a server version of AWR.

Read more >>

Some of you may have noticed that your link counts in Google Webmaster Tools have increased somewhat recently, we are seeing amazing numbers up to 1000 times higher than previous figures.

The reason for this is very interesting, following the roll out of the new Caffeine infrastructure Google is able to spider sites far deeper than before and they are now reporting this increased activity in Webmaster Tools. You can see this by looking at your internal links – if you have 50,000 internal links to your homepage then it’s a fair assumption you have 10,000 pages on your site. This number is probably a lot higher than it was last month.

When you take into account the increase in internal links being reported it’s quite clear why Google is now reporting on a lot more external links too, especially when you think about sitewide links. Read more >>

For anybody interested in news sites the interactive graphic below is an interesting breakdown and shows just how powerful some of the social news sites are. The Drudge Report especially sends a huge amount of traffic as does the BBC at nearly 2m per month.

Interestingly the BBC has committed to increasing the traffic they send to other sites:

BBC Online will be transformed into a window on the web with, by 2012, an external link on every page and at least double the current rate of ‘click-throughs’ to external sites.

Adding links to content is a time consuming business so we’ve contacted the BBC with an offer for some of our SEO team to help placing these links but we have yet to hear back from them.

Read more >>

In a major cost cutting exercise the government is to close up to 600 websites in a bid to save over £100m per year in running costs. Some of the numbers being quoted are quite amazing, the Business Link apparently cost £35m last year.

A report from the Central Office for Information, published today, found that £94 million was spent on the construction and set up and running costs of just 46 sites. The Government also spent £32 million on staff costs for those sites in 2009-10.
The most expensive websites were uktradeinvest.gov.uk which costs £11.78 per visit and businesslink.gov.uk which costs £2.15 per visit, according to the COI.

Clearly this is going to cause a huge shakeup in the UK search results, firstly due to the fact that 600 sites with a lot of traffic will disappear (we can be quite sure that 301 redirects won’t be applied) but secondly due to the sudden loss of the most trusted links available.

Sites that rely on lots of .gov.uk links could suddenly face large drops in rankings once these sites stop working.

What will be interesting is to see how many SEO companies start replicating the content of these doomed websites for clients and then contacting everybody who has linked to a particular .gov.uk site suggesting they link to their sites instead. it could be a very big opportunity to get links.

1. Using a Non-SEO to Manage the Campaign

Kind of obvious really right? Maybe, maybe not. Although “hard” SEO skills may not be necessary, but an understanding of “basic” SEO definitely is for any serious brand. Doesn’t matter if the actual SEO is being carried out by an external agency – in order to align the brand and it’s strategies you need someone who understands both – an SEO Manager needs to understand both. Lack of understanding, or even lack of interest in knowing can not only slow the businesses SEO strategy, it can have negative effects.

An intuitive or intelligent individual can go a long way in improving your campaign – and be able to tell all the bullshit apart from real advice – as Rae so decently pointed out.

2. Assuming That They Can Do All SEO in-House – on Budget

SEO isn’t not a fixed skill, which once you have done it for a couple of years, then you know it all. It is an ever changing, ever moving set of goal posts. In house teams that are tiny, or haven’t been built organically based on skill, seldom have the time to research or play with unrelated categories. At the same time, unless you are a pure online player, the access to resources and broad skill bases may be limited.

If you are a big brand, and want to do real SEO in house on a budget, I wouldn’t advise it – you could hire a full time SEO agency for the cost of 2-3 staff – and have access to advanced skills and the benefit of their experience day to day with other campaigns. Not saying that it is impossible to have a full in-house SEO team – all I am saying is if you have never done it – plan this strategy with care.

Back in 2007 High Rankings Interviewed Danny Sullivan on the subject – many of the responses still hold true in my opinion.  If you are intent on running an inhouse team – at least set up some decent processes for SEO .

3. Not Treating SEO Like a Real Revenue Channel

Sounds surprising right? In my experience, 7 out of the 10 Big Brands I have worked with didn’t have SEO KPIs, Strategies or Plans in place. It was something delegated to an in house tech, or to an agency, often with dismal results. This happens when you don’t give the channel the respect it deserves.

SEO is actually more than a series of link builders and content writers, it is a full blown Marketing Discipline, that needs strategy, thought, research and focus on.

4. Not Separating Brand Traffic and Sales from Generic Keyword Sets

To me this is the biggest failure. Looking at “SEO” revenue as a whole and then assuming that the channel is healthy is quite myopic. Brand traffic in most cases is a given including those sales as part of the ROI is not the best way to judge the return on investment. A clued up Big Brand would value the revenue gained from Generic Keywords.

Some brands (especially those in highly competitive industries) do this really well – while more traditional Big Brands don’t in my experience.

5. Not Involving The SEO Team

This a massive process issue in many businesses. From full marketing plans to press releases, from product release to PR disaster, the SEO team should be involved or kept aware of at every stage. There are often first mover advantages in SEO, especially PR. Not giving your own team the edge means you are actually working in detriment of the brand in the long run. For example, one brand I worked with had a policy of distributing their Press Releases before they were published on the site “PR” Section. The result? A third party site that auto publishes press releases got into Google News first and made it next to impossible for this brand to get in that space.

I’ve been thinking for a while now that, as an industry, SEO agencies are really suffering because of the word “optimisation”. It’s just not the right word.

The Google algorithm is based around 2 things, the easy bit which is on-site optimisation and the hard bit which is link-building. Anybody can optimise a website but in most industries link-building is 90% of the algorithm.

There are of course some very complex SEO issues to consider when building a site and planning architecture but once there are done then most agencies have very little scope for changing things. Anybody who has worked with a large company understands that getting architecture and code changes is a 6-18 month timescale for all but the smallest sites.

When you think about optimisation you think of small tweaks and changes, you don’t think of 10-15 days per month researching quality link sources and phoning/emailing hundreds of link prospects in the hope of converting some into live links.

Reading a thread over at b3ta you can see how most people assume that SEO is something a good developer should get right at the build stage but not many people seem to understand the sheer amount of effort required to build enough good quality links to get rankings.

In an ideal world like the one Google thinks we live in you could rely on just creating good content and links would suddenly come flooding in. This simply isn’t the case, unless you have perhaps 10 years to wait before you get visitors.

Now that Google Caffeine is 100% live & following up from the recent May Day update I thought it would be good to talk about some of more advanced aspects of long tail SEO.

First of all Caffeine is a new infrastructure rather than an algorithm update so it’s not related to the May Day changes. What Caffeine does is increase the freshness of the Google index by increasing crawl capacity and also decreasing the time it takes to get the crawled pages live and searchable in the index. For bloggers this might not be a big change for new pages because blog platforms normally ping Google and get indexed in a couple of minutes anyway – for the rest of the web this should make a big difference and open up the door for much fresher long tail results.

The May Day update basically means that some sites with thin content and a lack of internal links are no longer getting the authority benefits they used to. Google is showing more relevant pages instead which is certainly a positive step.

What is the Long Tail?

Long tail SEO describes the thousands or millions of search terms that individually generate very little traffic but collectively generate a large percentage (perhaps 70%) of a sites overall search traffic. Long tail doesn’t mean keyphrases with 4, 5 or 6 words in the phrase – these may fall into the long tail group but that isn’t always the case. We have some very large 4 word phrases that send thousands of visitors per month and they are classed as short tail terms.

The best way to classify the terms is to look at the chart below from SEOmoz which breaks search traffic into 3 buckets:

• Short tail – 18.5%
• Mid tail – 11.5%
• Long tail – 70%

These figures are approximate but as long as we are consistent it doesn’t matter too much what we choose.

The next step is to do some analysis to measure your current short, mid & long tail traffic numbers so that you can monitor each month how things improve. We set this up as an advanced segment in Google Analytics as well as an Excel chart and find that the following figures tend to give the percentages we want for most websites.

• Short tail – keyphrase with 100 or more visits per month
• Mid tail – keyphrase with 6 to 99 visits per month
• Long tail – keyphrase with 5 or less visits per month

You need to run some figures for your site until you get the percentages in the chart above – don’t forget to remove brand searches.

Segmenting long tail traffic

Visualising millions of keywords that each send a handful of visitors every month is an impossible task so we need to try and segment the data in order to try to improve the numbers. The best way to do this is to split the site into the same sections we have create for our multiple sitemaps above and for each section analyse & monitor the following:

• Number of pages indexed
• Number of landing pages receiving > 1 visit per month
• Number of keywords sending visitors to the section each month

Long & short phrases

Having said that long tail doesn’t necessarily correspond to the number of words in a keyphrase it is still very important to track and monitor the distribution of words in your keyphrases every month. You should do this in two ways, by setting up filters in Google Analytics but also by exporting all your keyword data to Excel and running a pivot table query to show figures such as conversion rate vs keyphrase length and visitor or revenue numbers vs keyphrase length.

Measuring Indexation

The best way to measure indexation on large sites is to split the site into sections and create a different xml sitemap for each section. By doing this in Webmaster Tools you can quickly visualise what pages are getting indexed and which are not.

If you find that a particular section has an indexation issue then we need to diagnose what’s going wrong. This get’s a bit technical but the best method we have found is to create a script to check the indexation status of each page as follows:

• Check to see if the page is indexed using the info:http://www.site.com/page.htm command on Google
• Check server logs to see how many times the URL has been spidered in last 30 days
• Use SEOmoz API to find total links to the page & mozRank

Once we have this data we can look into what might be going on & try to fix it.

Measuring number of landing pages

Again, this needs to be done by splitting your site down into different sections, you can do it in bulk but that doesn’t give the right data. The key is to use this method but to add a filter to only show the landing pages from the sub-folder or category you want to analyse.

The final result of your analysis should be a chart that looks something like the one below, taken from one of our ecommerce clients.

Most people have probably heard about the May Day update which has either dramatically increased or decreased traffic depending who you talk to. Initially the talk was all about how sites had lost traffic but there are now a number of people coming out to say that traffic has had a big increase.

We’re seeing either no change or a dramatic increase in long tail visitors since the update.

The update seems to have been quite straightforward, Google has increased the emphasis on quality and is giving smaller sites a chance. Large sites with thin content and a lot of trust no longer get a free ride just because they have a keyword match.

I’m heading off to Sweden today for the EPiServer Partner Summit so don’t have much time to write – however I strongly suggest you read this post about the update to find out what’s going on and how you can benefit.

Ever since the famous AOL data leak of 2006 people have been endlessly quoting the same old figures when asked for amount of clicks a particular ranking gets.

Recently the data has become a lot more accurate thanks to the release of click data in Google Webmaster Tools and a new study by the Chitika ad network.

From the AOL data we can see that first place gets 42% of traffic, compared to just 11.9% for second place. That seems too high to me.

Read more >>