Testing times ahead… or maybe not

Web site testing is one of those things, like sticking to the speed limit on motorways, that everyone says they’ll do but, in reality, most don’t. Like speeding you have the potential to get somewhere more quickly but to do so you must take risks that can be very costly.

Almost without fail, testing will be the one area of a project that’s cut by a client if money or time get tight. In a lot of cases a developer will be told “We don’t need external testers, we’ll just get the guys in the office to give it the once over” or worse still “We’ll fix any problems after we launch”.

Why is testing given such a low priority?

There are four possible answers to this question:

1. the project team will be able to find all problems before launch
2. if problems surface, that they can be dealt with sufficiently quickly that they will only affect a few users
3. testing might reveal some fundamental problems that will be costly to fix
4. Web 2.0 and the “perpetual beta” is cool

Unfortunately none of the above represent a sensible strategy to adopt on your Web site so let’s take a look at each in turn.

The project team will be able to find all problems

No matter how good the team is they won’t find all the problems before launch. The team knows the product too closely to be objective during testing. They won’t try really unusual actions because it won’t occur to them that a user might do that. From a ‘fresh’ user’s perspective they have no pre-conceived ideas about the site so will do what is most intuitive to them.

Problems will only affect a few users

This flawed logic – do you really want any user to have a less than satisfactory experience when visiting your Web site?

The first people to visit a new site (or new area of a site) will be the people who are most interested in what’s on offer. These people are the natural promoters of your site and you can’t afford to turn them from vocal proponents to loud critics on the basis of their first experience. Even the smallest typo decreases the value of your Web site and damages the reputation of your organisation.

Problems will be costly to fix

Problems always cost more to fix if they’re discovered in a production system than if they’re found in a system still under development. It’s also not just the simple fiscal cost that you should consider. If users experience problems with your Web site then this is costly to your reputation and visible issues (like typos) may end up in screen shots or cached by search engines for some time.

Testing is something that should be done throughout the development process. The earlier a problem is identified the easier it is to fix. For example, the terms used in your Web site’s navigation might have an obvious meaning to people within your organisation but be confusing to end users who are non-specialists. User testing with mock-ups at the start of the project would identify these issues quickly. Retrospectively having to change the text across the site is very time-consuming.

Web 2.0 and the “perpetual beta” is cool

A worrying new trend is that of businesses launching beta versions of their service and justifying this by saying the service is in “perpetual beta” and therefore being “Web 2.0”.

Rather than following the true ethos behind the perpetual beta (that of a product being developed in the open) the moniker appears to be used to cover up a lack of testing or proper functional specification in the the first place.

When it comes to your average Web site (rather than services like Flickr etc.) then surely the site should be developed continuously anyway? So the perpetual beta tag on a site promoting a company is pointless anyway… Once again we return to the problem of customers being exposed to problems, it may be cool to use the phrase perpetual beta but it’s not cool to make customers suffer (for them or your business).

Conclusion

At the end of the day you want to give users the best possible experience you can. The better the experience the more likely they are to return and buy something, use your service or get another ad impression. Can you really afford to cut the testing phase from your project?