The main goal of all search engines is to return results from the sites that are the most useful and relevant to a user query. Sites with country-coded top-level domains, such as .ie, are associated with a geographic region, in this case Ireland. By default, and all other things being equal, sites with a .ie domain are interpreted as more relevant for users in Ireland. For generic top-level domains such as .com Google allows site owners to manually assign a preferred country in their Webmaster Tools.

As our .com domain’s content covers many countries we can’t use this particular feature to optimize the site for a particular region. Instead, we were hoping that by redirecting our Irish and UK pages to their local domains that those pages would benefit from a boost in search results positioning on Google.ie and Google.co.uk as being targeted at their local audiences.

The initial results were terrible. The redirected pages dropped in the SERPs by about 3 positions. As our Irish pages previously ranked better than our UK ones they suffered even more. (Ireland was our first local area of interest and those pages were in the index much earlier.)

After a few weeks our results moved slightly in the right direction but were still behind the control group. We decided to let the experiment run for longer, almost four months in total, to be sure that the conclusion drawn was correct. In the end we turned off the redirections, just last week in fact. The experiment was successful in terms of achieving measurable and unambiguous results, but unfortunately they were very disappointing.

Our redirected .com pages disappeared from the index and over time their corresponding .ie and .co.uk pages were indexed instead. However, they never reached the search results positions we had had before with our .com pages. We speculated that our .com domain had more trust and authority than our country specific domains and that’s why pages from RevaHealth.com ranked better even in Google’s country specific search engines.

Now, just one week after undoing the redirects, most of the .com pages have regained their positions from before the experiment, which we think goes some way to validating our speculation. Our generic .com domain, which is crawled much more frequently than our local domains are, does seem to be a more trusted site, not only by Google.com but also by Google.ie and Google.co.uk.

Have you run any similar experiments that you can share with us?

SPF (Sender Policy Framework) is a protocol that is built on top of email. When a message arrives at the server receiving your email (Google, Yahoo, or your own company’s email server) the email claims to be from person@domain.com, but it might not really. All we know for sure is that it definitely comes from some IP address.

Using SPF, your email server can check with the domain that the email purports to be from to see if the IP address you got the email from is one that they use to send email. If domain.com says no, they don’t send emails from that IP address, then the email may be spam. If the domain says yes, then it’s probably not.

Probably; however, this isn’t certain. Not all email senders support SPF. It is voluntary but very widely used. So, just because the sender doesn’t support it doesn’t mean all emails from that domain are spam. Conversely, people send spam from perfectly respectable domain names all the time, so just because you do get a valid SPF record back that matches the “from” domain, doesn’t mean it’s definitely not spam. Still, it is a good indication, and some email servers and ISPs will mark your email as spam if the SPF record doesn’t match, or isn’t present.

So how can you avoid this fate?

First check if you already have a valid SPF record. Go to http://www.kitterman.com/spf/validate.html and enter the domain where you send email from. If your domain returns a valid SPF record, everything is fine. If not, you may find some email servers you send emails to may block them as spam.

Here’s what Gmail was showing for us:

Received-SPF: neutral (google.com: XX.XXX.XX.XXX is neither permitted nor denied by best guess record for domain of info@revahealth.com) client-ip= XX.XXX.XX.XXX;

Authentication-Results: mx.google.com; spf=neutral (google.com: XX.XXX.XX.XXX is neither permitted nor denied by best guess record for domain of info@revahealth.com) smtp.mail=info@revahealth.com

(To see this, simply send email to a Gmail account, and then select ‘See Original’ in the little menu at the top of the email message. You get to see all the headers for the email.)

What this is saying is that when they check the IP address we’re sending from, they get back neither a “confirm” nor a “deny” message. That is, there is no SPF record at all.

We used to have our SPF record for RevaHealth.com set correctly. I know, because I did it. I also could tell when it stopped working – when we moved our front end box from one server to another a few months ago. Of course, I couldn’t remember just what I’d done to set it correctly nearly three years ago.

The key to the answer is not who or what sends your email, but who owns the domain. The email receiver doesn’t check with the domain, but with the DNS (Domain Name Server). In the example above, Google GMail isn’t checking with RevaHealth.com, but with the Domain Name Server for RevaHealth.com. In our case, that’s Go Daddy.

Of course, I didn’t remember that at first. Thinking our own front end box would have the SPF record I looked in its own DNS entries and added it there. There’s a very handy SPF setup wizard here to help you to create your SPF record and save it in your DNS. However since our DNS is Go Daddy, this did me no good at all.

So, after going back and reading the very helpful SPF FAQ again, I realised that I should use our DNS to create the SPF record. And that’s when I realised what had gone wrong. When we moved our servers, we updated our DNS entry for RevaHealth.com and lost the SPF record in the process.

Editing your SPF record on a domain register depends on their interface. Thankfully for us, there is a helpful guide to creating  SPF records for domains hosted with Go Daddy.

I quickly added the SPF record to our RevaHealth.com domain entry, but this wasn’t the end of the story.  We send email from our hosting server. This looks like mail.si-svXXXX.com and that’s what an IP address lookup returns. When I entered this domain as an allowed domain to send email, I got nothing. Running the validation check failed.  However, this was because the SPF record should return only the domains it supports, not the sub-domains. Dropping the “mail.” and changing the record to just si-svXXXX.com brought our SPF records back to normal.

Now, Google reports itself happy with us again.

Received-SPF: pass (google.com: domain of support@revahealth.com designates XX.XXX.XX.XXX as permitted sender) client-ip=XX.XXX.XX.XXX;

Authentication-Results: mx.google.com; spf=pass (google.com: domain of support@revahealth.com designates XX.XXX.XX.XXX as permitted sender) smtp.mail=support@revahealth.com

Did it make a difference? Is this worth bothering about?

Yes. Several of our customers had not been receiving emails from us because their ISP was blocking anything without a valid SPF record, and these emails are now getting through okay. It probably reduces the overall spam score for emails we’re sending too, but that was always very low anyway. It’s definitely worth doing, it costs nothing, and the business cost of emails not arriving where they are supposed to can be massive.

An A/B test that uses a sample size that is too small is worse than no test at all. The figures just aren’t statistically significant, and if you had run the test longer the results may have reversed. This means that not only might you have the wrong result but you feel that you have proof in that wrong result. This results in in you defending your position later on despite contradictory evidence.

So how big does your sample size need to be? Unfortunately the answer is – ‘it depends’. Luckily it doesn’t get very complicated. The only thing it depends on is the gap between the winner and the loser. In general the smaller the gap the bigger of a sample size that you need.

Specifically divide the gap between your winner and loser in half and then square the result. The result needs to be bigger than your sample size to be statistically significant?

Below is a worked example where there is a 10% difference between A and B

So with a 10% difference between A and B you need a sample size of about 1500 to be statistically significant.

Let’s see how this works with a 2% difference between A & B

Now you can see that you need a sample size that is around 25 times bigger.

Tip: If you find your test needs a very high sample size (above about 20,000) then your users don’t really care much about your changes. Give up and test something more significant.

Creating an Facebook ad really is very easy, and is split up into three simple steps.

1. Designing Your Ad

Designing a Facebook Ad

Designing your ad couldn’t really be simpler. Put in a destination URL, a headline, some ad text, and an optional picture and you’re good to go. There are some useful links to best practices, reasons for rejection and a design FAQ on the page also.

For some reason their preview of the ad has a line saying “Phil Boyle likes this” with a thumbs up icon beside it, but this hasn’t appeared any time we’ve seen the ad displayed.

2. Targeting

Targeting A Facebook Ad

Facebook offers some great targeting options, but also leaves some basic ones out. For instance, you can easily target just men or women, just those in a relationship, or even just those who are engaged. This is a great service for anyone in the wedding industry looking to drum up some business!

However, it only allows you to geographically target by country, which will be too coarse for many businesses, especially smaller local businesses.

Similarly, their other targeting filters will be great for certain business types but all but useless for others.

3. Campaign Pricing

Pricing A Facebook Ad

All the usual options are available when it comes to controlling the cost of your campaign, but there are also some things that we find a little strange. You have to set a daily limit, as they bill you every day. There are also no VAT invoices available for your campaigns at the time of writing.

You can choose either CPM or CPC payment for you ad. We went with CPC ourselves. The suggested bids seem to be picked out of thin air, and they are quickly creeping up too, presumably as more advertisers buy up slots. Pay what you want to pay and don’t be afraid to cut your bids. They do offer a decent set of reports so you can analyse what effect changing your bids has.

Our intention with Facebook advertising was primarily to get our brand in front of as many people as possible with as little cost as possible, and to that end it has been pretty successful. We’ve had coming up to 9,000,000 impressions on our ads and spent less than €900, and that was just targeting Ireland!

So, in summary, if Facebook’s targeting can identify any niche that your business is interested in, it could well be a useful advertising platform for you to explore. For the rest of us it’s probably just a good way to get some cheap name recognition going.

We don’t have a product or sale as a goal of our Facebook advertising, but if you do we’d love to hear how you have gotten on with it and whether you think it has been a success or not, and any other thoughts people have on social network advertising.

When looking at this it is easy to get caught up in looking at your main landing pages to the exclusion of the other pages. At RevaHealth.com we have historically focused on our search results pages to the exclusion of pages deeper in the site.

Old Page

However, times change and we have over 1.5 million pages indexed by Google. Every month Google sends traffic to a about 250,000 different pages including a large number of deep and seemingly unimportant pages.  The vast majority of landings are still on our main search results pages; however there are an increasing number of visitors landing on individual clinic’s profiles.

These profile pages all had a disproportionally large bounce rate when compare to the rest of the site and looking at the pages it should come as no surprise.  For an engaged audience they present the information in a reasonably structured fashion, however for a visitor who has just landed on the page it’s a bit difficult to see why they should stick around.

The most obvious way to reduce bounce on these pages would be to eliminate the advertising. However these pages rely on advertising for revenue generation, so we elected to see if we could reduce the bounce rate while attempting to keep advertising revenue static.

We introduced a navigation panel on the right hand side. The purpose of this panel is to inform the visitor about relevant content in the rest of the site.  The key points are:

New Page

1. The number of relevant clinics in the immediate area that they might be interested in.

2. Up to three sample clinics in the immediate area.

3. The number of relevant clinics in a broader geography with a means of navigating to them.

Other changes we made to the page were:

1. A new call to action within the address field – “Get Phone Number”. About 20% of the visitors to our site are actively looking for a clinic’s telephone number. We don’t want to immediately display this number because we want to track it, however we absolutely want visitors to know that it is available.

2. Introduction of default Description Text where we do not have a description of the clinic along with a call to action.

After making these changes, the bounce rate of the profile pages dropped by nearly 7%, causing an overall drop in the site’s bounce rate of just over 2%.

We’d love to hear your experiences with bounce reduction, especially any unexpected results you had.

Warning – Not for the casual non-technical reader!

The Problem

RevaHealth.com is made up of 10’s of millions of pages, organised as ‘pretty’ URLs such as

• /dentsist/ireland
• /dentists/ireland/dublin
• /dentists/ ireland/dublin/crowns
• /dentists/ ireland/dublin/crowns/the-big-clinic

We cache each page in an asp.net data cache, and this works for frequently requested pages as they have a high cache hit rate. This works by holding the data you need to construct a page in memory. However, there is a fairly heavy code hit which results in a Time to First Byte of 1.2 to 1.5 seconds.

This wasn’t providing the user experience that we wanted and we were determined to lower it, so we added asp.net output page caching with a time to live of an hour. This holds the fully constructed page in the web server memory so it can be close to instantly returned to the user. This resulted in a Time to First Byte of 0.5 seconds.

This was great.

Or so we thought. Regular testing revealed that even frequently requested pages were rarely in our cache. Why? In fact only about one in ten pages were in the cache. This wasn’t good. Was the output page caching not working?

Why?

The answer wasn’t so simple. Firstly, RevaHealth.com is very broad and flat. Search results are divided by the type of clinic (dental, cosmetic, etc) and then further subdivided by multiple levels of location (country, county, city & neighbourhood). To make matters worse there are options for further treatment and/or specialization sub-subdivisions. A typical landing URL might well look like this:

http://www.revahealth.com/dentists/uk/west-midlands/birmingham/erdington/implants

Landing pages are almost all ‘long tail’, and the tail is very, very long. With over 50,000 locations in over 200 countries, several dozen clinic types and hundreds of procedures, we have millions of search pages and over 100,000 clinics. We knew our landing pages covered a very broad range, but only when we looked into the figures more closely did we realise just how broad and how flat the site was.

In a typical period 152,072 visitors entered the site through 36,357 pages. Only 66 of those pages had more than 100 hits and 30,000 had five or less hits. So in a typical one hour period only a few hundred pages were getting a hit in the output cache. The huge bulk of pages requested were not in the cache when requested.

Looking for Solutions

Clearly a simple remedy would be to extend the cache life beyond an hour. But this has business implications. Firstly, when our customers update their profile, they want to see that change reflected as soon as possible. Asking them to wait more than an hour would not be good.

More importantly, for a site like RevaHealth.com, search ordering and the appearance of results is critical, and search results order updates happen dynamically as patients contact clinics, review clinics and generally interact with the site. So, extending cache time to a level where we get more of the tail into the cache would be very problematic.

We decided to simulate traffic to the site, and to force the most frequently requested pages into the cache on an hourly cycle.

cURL seemed to be the obvious tool to use, as we had some experience with it and it is widely accepted. We generated a list of the top 100,000 most frequently requested URLs and created a cURL script to fetch them all.

Our experiences with cURL

cURL is a feature rich tool, but we wanted to use it in a pretty simple way – fetch all the pages on the list on an hourly cycle.

The first problem we encountered was that from the command line there is no way to limit the rate of page fetches. We knew we wanted to fetch them at a rate just above 12/second to ensure that the script would complete in an hour. But curl will only set a speed limit in kb/sec. Since our page size varies greatly, this made fixing that speed a case of trial and error. Obviously we didn’t want to fetch too fast and strain the server unnecessarily, or fetch too slow and not complete the list in an hour.

We could have used libCurl in our own server code and set a rate per second there, but we were keen not to have to write code for this, and instead use the command line tool to keep it simple.

Some relatively straightforward trail-and-error tests revealed a rate which would enable the script to finish within the cache time available (one hour).

What was frustrating during this process was that there is no way for cURL to send the actual file data fetched to nul and to save  the normal stdout output to a file or even send it to the screen. We didn’t want to save the actual output files which could get potentially very large, but sending them to nul meant normal output was sent to nul too. Equally frustrating when testing was that the normal (non verbose) output does not show the URL of the page being fetched.

The progress meter shows bytes downloaded, percentage completed, etc, but rather strangely, not the file being fetched, so there’s no easy way to tell your progress through the list of pages you are fetching.

In the end though we got past all these problems and had a script that worked – or so we thought. In fact, our first run through made no difference to the cache at all. This caused a lot of head scratching until someone looked at the fetched files and we realised that, of course, they were not compressed.

We always return compressed dynamic pages. Since the output file is gzipped, and cached as a compressed file, we were only having non-zipped pages cached.

Helpfully, cURL allows the http request headers to be set on the command line, so simply adding  –header “Accept-Encoding: gzip,deflate” fetched our zipped pages into the cache and testing in Firebug showed that they were being requested by our script.

We watched memory usage during the build up of the cache, and made some adjustments to allow larger physical memory to be used. At a certain number of pages requested we began to see large page usage, so we scaled back the number of pages being requested and all returned to normal.

Browsers Browser Browsers

We thought we were done, but one of the oddest things was yet to bite us. Like most developers, we love Firebug and we were checking everything using Firefox, but before we push changes live we do a fairly rigorous check in other browsers. Disaster. Firefox and Chrome were receiving our new cached page but Internet Explorer wasn’t.

Internet explorer was simply bypassing the cached pages and hitting the code. This was exactly what we were trying to avoid.

The problem was that we were also using GZIP to compress the HTML. It turns out that  IE passes a different parameter for the ‘accept-encoding: gzip’ than Firefox or Chrome does. Even though they all accepted exactly the same encoding the web server wouldn’t serve it up.

• FF: Accept-Encoding: gzip,deflate
• Chrome: Accept-Encoding: bzip2
• IE: Accept-Encoding: : gzip, deflate  (note the space)

Essentially because the browsers were each requesting the same file using very slightly different parameters it resulted in the web server thinking they were different files.

The choice was simple, either:

1. Cut the size of the cache to 33% and increase the length of it 3x
2. Only support some browsers

Unfortunately the commercial reality of choices like this is – ‘Provide the greatest good to the greatest number of users’. This meant only providing cached pages for IE. As a result Firefox and Chrome users have a slightly degraded experience compared to IE users, however this degradation is largely compensated by faster JavaScript engines.

Note: IIS 7 introduces some control that solves this particular issue.

Your War Stories

We’d love to hear about your  trials and tribulations getting time to first byte down. Leave a comment below.

What’s worse is that they frequently don’t know what the problem is. They then turn to yet more designers thinking that the colour scheme or graphic design is at fault, when they should be focusing on the core proposition of communicating their information efficiently to their visitors.

The following are the top 10 most frequent mistakes that we see clinics make when they first build a website.

1. Creating a splash page. This is a page that appears before the main content that ‘introduces the tone’ of the clinic. This just gets in the way of the visitor and the information they are looking for.  RevaHealth.com usability studies have shown that the majority of visitors don’t progress to a second page.  Get the visitor to the content that matters as soon as possible.
2. Background music. You may think it is soothing – it’s not. Most people find background music irritating, especially if this is a repeat visit to your website.  Audio can be used effectively especially in the areas of treatment descriptions, however this is very much an advanced technique.
3. Original navigation. We have seen everything, even using the teeth in a jaw as way to visually navigate a site. This just confuses the visitor. As a general rule, use tabs.
4. Clinical photography of procedures on prominent pages.  Your patient is interested in results – in general it’s only other professionals that are interested in photographic details. It can be good to include these photographs but only include them on deep pages that are clearly linked to with text similar to “view detailed photographs of this procedure’.
5. Expensive, flashy design. Your site needs to look professional; however it doesn’t need to be flashy. It is very easy for clinic owners to get caught up in the design of a website and lose focus on the whole purpose, which is to inform the visitor.
6. Hit counter. You may want to boast about how much traffic your site has, however why does the visitor care. They want to be treated by your clinic, not take out advertising on your site.
7. Not doing the SEO basics. The basics of search engine optimization are simple and given that the search engine should be the #1 distributor of your content there is no excuse not to do them. Give each page a unique title; make sure your website doesn’t require JavaScript and sign up for Google Webmaster Tools.
8. If you are using an agency to create your website many will offer to supply you with stock content, such as treatment descriptions. Don’t use them. The search engines will recognise them as duplicate content and will not rank you for these pages. Worse still they may think you’re a spam site and remove you entirely. Write your own content.
9. Not including basic information that a visitor may want – like a map or opening hours. We will publish a list of the most frequently requested information by visitors in an upcoming post.
10. No call to action. Why do you want a website in the first place? Is it because you want more online enquiries? If so then there needs to be a large and effective call to action – A big red button with ‘Click Here to Contact the Clinic’ or somthing similar on it.

Let us know any common mistakes or omissions from websites that you’ve come across in the comments.

For instance, here is the snippet of code to show a map of general practice doctors in Brighton.

<script type="text/javascript" language="javascript">

document.write("<iframe src='http://www.revahealth.com/doctors/uk/east-sussex/brighton/externalmap' width='600' height='500' frameborder='0'></iframe>");

document.write("<span>Data provided by <a title='RevaHealth.com' href='http://www.revahealth.com'>RevaHealth.com</a><span>");

</script>

And here is how the map would appear on your page.

The map pins show the locations of the clinics. The prices shown are for a standard doctor consultation in the practice. You can pan around and zoom in and out to see more detail about the location of each practice, and click on each pin to see more practice information.

Using the maps on your own website is completely free and easy to do. You just need to add a small snippet of code to your page which pulls in the map and data from the RevaHealth.com server. You don’t need to be a programmer at all; anyone who can edit their own web page can do it easily.

The snippet can easily be changed to show any of the different kinds of clinics in the thousands of locations in Ireland and the UK which are covered in the RevaHealth database. You can contact us at the address below to see what types of clinics are available. For example, you could show Laser eye clinics in Stratford, Dental Clinics in Prestwick or GPs in Cork.

To do it yourself, just search for the URL of any set of clinics on RevaHealth.com as normal, and when you find the list you want, add /externalmap to the end and replace the URL in the example snippet above with the URL of your choice. Hey presto!

The clinic data is constantly being refreshed and updated by the team at RevaHealth.com and users can look up phone numbers or contact the clinics on-line.

The API is free to use, although we do ask you to show the source of the data beside the map on your site with a link to RevaHealth.com. The code snippet above includes the link:

Data provided by RevaHealth.com

which you can change if needs be. If you are interested in putting these maps on your site or have any further enquiries, please contact us at support@revahealth.com.

The whole reason why search engines work as well as they do is because of the huge amount of effort that publishers put into SEO daily. That’s right – without SEO the search engines wouldn’t work.

Why?

Well, what a search engine tries to do is understand the meaning of a page and subsequently display it in the search results when someone types in a query that indicates that they are looking for that content. This is phenomenally difficult to do; even with the best brains on the planet it is currently impossible.

The reason for this is that machine learning isn’t nearly advanced enough to be able to understand at the levels that humans are able to understand.

So the search engines rely on publishers to shape their content so that they can better understand its meaning. Back when the search engines where pretty unsophisticated this literally involved telling the search engine using the meta keywords and meta description tags.

As the search engines and spammers have become more advanced the means of conveying meaning to the search engines has become less overt, but no less important.

RevaHealth.com definition of Search Engine Optimization:

SEO is the science of narrowing the gap between the search engines’ understanding of a page and a human’s understanding.

You will notice from the above definition that this doesn’t include link building. That’s because we regard link building as PR. Just as old school PR was about getting your company correctly positioned in newspapers and on radio, new school PR is the same thing online.

So what can happen if you build a website that solely targets users and not search engines?

The following examples are completely legitimate ways of constructing your website that will have no negative effects on your end users, but will completely ruin your chances of being distributed through the search engines.

Let’s say you are building a listing site similar to RevaHealth.com

1. You use parameters instead of hard URLs.  If you construct your parameters without thinking about SEO  you can easily create a website that the search engines simply won’t index.
2. You use JavaScript to dynamically call in the content similar to Kayak. This can provide a very useful end user feature. However, if coded without reference to SEO it will result in the content not being indexed.
3. You use a template for each page that results in the search engines thinking that each page is a duplicate of another page.
4. You include useful additional information for the user (for example from Wikipedia) that is not original.   At best this will result in the search engine marking down your pages and at worst they might list you as a spam site and exclude your entire site from the index.
5. You have a infinite number of combinations of search results available resulting the search engine giving up before they have crawled all of the relevant content

In Summary

Search engines are the number one means of distributing your content. If you develop a website without considering the SEO implications of your decisions, you are effectively giving the search engines two fingers. Don’t be surprised if they give you two fingers straight back.

We often get asked about our experiences with SEO, usually about what worked for us and what didn’t. Keywords are a topic that come up again and again. The video above comes from Matt Cutts, the head of Google’s Webspam team, and it neatly summarises the fact that when it comes to Google’s main search product, meta keywords are completely ignored.

That should be the end of that then. But it isn’t.

A lot of people seem to get mixed up between keywords and meta keywords, thinking they are exactly the same thing, so the video above might be taken by some people to mean that ALL keywords are ignored by Google. That just isn’t the case.

The word “keywords” can mean many things interchangeably, but for the purposes of this blog post, let’s  refer to a set of keywords as the most important words, phrases or acronyms that your potential visitors will use to find a particular page of content on your site.

For these potential visitors to find your page, the keywords they use, or their synonyms or related keywords will have to appear on or pointing to the page somewhere. These keywords can  appear in:

• the body text of your page
• the URL of the page, including the domain name
• the page title of the page
• the meta description of the page
• the H1 tag on the page
• the alt text of images on the page
• the links to the page

And that’s just off the top of my head. They can also still appear in your meta keywords, even if Google currently chooses to ignore them THERE.

You should still be doing your keyword research for each page (or set of pages) and using those keywords you identify as being the most important in the page elements listed in the paragraph above. Google even offer some good free tools to help you find out what keywords to use. I’d recommend Google’s Search Insights, Google Analytics, their free AdWords Keyword Tool, and even Google Trends. Add your own favourite keyword research tools in the comments below.