Judith Garman gives a presentation at Manchester Digital’s usability event, titled Usability: What’s The Use?
Judith: It’s good to see a full house here. I am going to talk to you today about knowing your users.
Ok, so why are we interested in users, they buy our products, they use our services and ultimately they make us a success or a failure. What I want to do is look at a few different products. BBC iPlayer, Facebook, Heathrow terminal 5 baggage handling. So lots of different systems there with things which may or may not work. I am going to talk about a couple of these one is TheHub. This was a social networking site that was set up by WalMart and it was directed towards the teen market. One of the criteria was when you signed up they sent an email to your mum asking if she agreed to you using the site, it lasted about 8 weeks.
Then we’ve got iYomu which was a New Zealand social networking site for older people. That actually lasted about a year and then they figured out that older people didn’t actually want to be in their own ghetto and they wanted to talk to other people and that closed down as well. So what you can see there is these are prime examples of how not understanding your users and not understanding your audience can lead to product failures.
Now there are lots of other reasons why products and software may fail. It could be poor communication it might be unclear or incomplete requirements. Or changing requirements where you are working towards something that you agreed 2 years ago and suddenly everything is different. You might be using the wrong technology or a poor technology or the old favourites bad coding or poor quality control.
Now what I am going to talk to you about today is communication because that essentially is the core part of our job as usability professionals. And good communication is down to the difference between knowing what is a good product and a bad product. And the secret is that only your users know the difference. There are lots of different techniques we can use to find out what people think and what their issues are. A very simple one just user interviews, otherwise known as stakeholder interviews, just talk to people find out what they are interested in. Another common technique is watching them, ethnography, I quite like having a job where you get paid to watch other people work. And then you can test out your products, user testing, so get people into an environment where you can see them using your product. These techniques and how frequently you use them very much depend on your product. So what you are doing, what the scale is whether it’s something very innovative or whether you are following standards that have been previously set.
Now you might be thinking oh well if we can talk to people well why do we actually need to bother user testing. We’ll just go and see what people think, we’ll have a focus group. But if you really want to find out how people engage with your product you need to see them using it. And one of the reasons for this is what people say and what people do are actually very different. There are a number of reasons for this. For example if you are in a focus group environment somebody might be trying to impress you or they might have turned up with their boss and think oh I should say this or I need to pretend that I like that or that I know this. And also what we find with a lot of common behaviour people actually aren’t aware of what they are doing. Their behaviour becomes so automatic.
Frequently in user testing somebody will say to me oh I never do that, then 10 minutes later we’ve got them on video doing the things that they never do. They don’t actually realise, so if you ask them whether they do something or whether they like something they’d say no I don’t – but they do.
And also products can look fantastic, they can really wow people and look brilliant and they might actually be impossible to use, until you start trying them out you are not going to know. And very importantly user testing gives you measurable results. If you ask somebody what they think it can be quite subjective. What is good to one person or OK to another person might be dreadful to somebody else so you need a way of putting that within a framework and you can look at the product and say well actually this is good and good means that 9 out of 10 people can buy something on this online checkout in under 5 minutes. You are only going to get that if you user test, you are only going to get that kind of information.
Now as humans we always make a lot of assumptions. This helps us a lot it gets us through our daily lives. It also helps us innovate I don’t think we’d have any of those wonderful new products if people hadn’t been innovating. The iPod, we wouldn’t have all these very very interesting Web 2.0 applications, it just wouldn’t be happening. But what we need to make sure is that we validate these assumptions because a really good idea and a really bad idea there is not really very much difference between them and we don’t always know. I can’t really improve on Mark Twain here, but what he said is ‘What gets us into trouble is not what we don’t know. It’s what we know for sure that just ain’t so’.
So I am going to look at some common assumptions, you might be familiar with these. One is people will contact you when they don’t understand something on your website. A lot of people still cite this to me. If they have had poor results with user testing and things are a bit difficult they will say ‘oh well you know the contact information is clear people will phone us up’, well they don’t.
And also you have a very common issue a lot of time with product registration. Where you get rafts and rafts of pages of forms. This is normally a marketing people thing. You have to collect all this information and you’ll be asked shoe size and personal information and it will just go on for pages and pages. This actually really can put people off. So there are things that you can’t get away with. Now these assumptions are actually creeping into best practices. Some of you may be sitting there thinking I know that , I don’t do that and I know that that is wrong. So that’s the start of it, what we should be seeing when we are user testing is there are some assumptions at some point that we won’t need to be testing out because everybody will know about them because they have crept into best practice.
So let’s look at something different. Ok, I think this is a reasonable assumption somebody who is visually impaired doesn’t want a camera phone or won’t use the camera feature. Well I think that makes sense to me. Why on earth would you want to use a camera phone if you couldn’t see? Well we recently did some research with various categories of disabled users with mobile devices and here’s what a couple of them said. Ok so one lady was saying I am going on a cruise and I want to be able to take pictures to share with my family and she wanted a camera phone. It’s a common assumption that a lot of people who are visually impaired can’t see at all, because a lot of them will be able to see a fair amount but probably not enough to engage visually with everything. And here is another one, I think this is great, this guy taking photos of a can of food and sending it to a sighted friend to find out what was in it so he didn’t end up with tinned peaches instead of tinned tomatoes in his spaghetti bolognaise. Great innovation I think it’s really useful. So what we have managed to do is we’ve managed to prove that assumption is wrong. And what you need to do in the sites and software systems that you are developing is look at those assumptions you are making and see whether they do need to be challenged.
Now we are talking about user testing and you might wonder what it actually is. Well essentially it’s measuring how easy it is to use a product. It’s also helping to understand people’s perceptions and attitudes because it’s not quite as simple as how easy it is. It’s also interacting with any system it’s also emotional and it’s about how people feel. It could be really really easy but people don’t like to use it and you can establish that from user testing as well.
Now there are a couple of golden rules when we are looking at usability testing. One is you need to test with the people who represent your user base. So if you are developing a web site for non-technical people and you go down the corridor and you test it out on developers in your team then don’t be surprised if it’s a flop when you go live. Because they weren’t your user base. They didn’t think the same way, they didn’t understand the same product. Likewise if you test out on your marketing team and your commercial team those people know about the company and the organisational jargon. You need real people who represent real users. And also doing realistic tasks you need to find out what people really do and that’s going to be different for every site. It might be buying goods it might be finding information. But that’s something whenever you work with usability professionals in user testing they will work with you to identify what it is people actually do.
Now we can test in the lab environment, you might be familiar with the rooms that have double sided mirrors so you can observe but the person you are testing can’t see you and that always works quite well. And with the kind of equipment that we have today it’s very mobile so we can test in people’s homes and work places. And remotely moderated testing is becoming a lot more popular now. You might have a test participant sitting at home in Aberdeen, a facilitator in Manchester and your clients watching also via the web by WebEx. So you’ve got that capability now and that’s useful if you’ve got very niche groups you are trying to recruit within your user base and you can’t always necessarily get them into the lab that’s something worth considering.
And what we do is we use these results to improve the product in line with client goals. Now something that quite often I get asked is if it’s all about usability what about us, the client if it’s about meeting the user needs. But it’s always a balance , it’s a balance between the user needs and the client goals that we will be looking to match up so we can’t have a perfect user experience if it doesn’t meet your company goals.
Now when can you do your user testing? You can do it from very very early on I mean you can actually do user testing with screen designs scribbled on bits of paper, paper prototyping or very basic electronic prototypes. Sometimes this is very very useful because you might want to test out an early design or concept and if you don’t want to spend 10’s or 100’s of thousands of pounds on development then you can find out about the ideas and you use it to help you evolve your design. You can also test on completed systems and live systems. A lot of people will be recommending that you do usability tests on a regular basis perhaps every 18 months as part of your maintenance just to make sure that things haven’t crept in. Particularly with a lot of the user generated content you do need to check up that things are still working and information architecture hasn’t lost its way.
Now to sum up I am going to talk about what you can expect from user testing. What you should get if you go through the user testing process is an identification of where users had problems and where they had successes and also an understanding of what motivates and engages users. Your report should include recommendations on how to improve your products. So we don’t just say well this didn’t work and this didn’t work and this didn’t work, we’ll give you different designs and again that will be based on your goals. If we know that you are going live in a couple of weeks and you need a quick solution we will recommend a quick solution and a more long term solution. And the last one is I think what is used the most is video clips and user quotes. A lot of the time you’ll have somebody in an organisation who is championing usability and they know they need to do something but they have got somebody else they have to convince. They have probably talked till they are blue in the face and not got anywhere. But then if you see a video clip of a user struggling to find something on your site and you can see what they are looking at and they are not getting anywhere and they tell you that they are not getting anywhere that is very compelling. So a lot of people recycle these video clips and use them to help promote further changes in their organisation.
Now because I am from AbilityNet I am also going to talk to you about what you can expect from disabled user testing. Well you might be surprised to know exactly the same because disabled user testing is just user testing with a disabled demographic it’s not anything any different. But what it does provide you with is a more rigorous approach and the reason for that is as soon as you are testing with disabled participants you are also testing for accessibility and you’re testing access technologies things like screen readers. So it’s very economical in the credit crunch because if you had a separate accessibility and usability budget you could combine them. And just in case you were wondering why just can’t you cover all your accessibility with an accessibility audit then common best practice is now that that will actually only cover about 40% of the accessibility issues that you will find on any site. That is very well documented and if there is anybody here who likes reading standards the WCAG standards, the ISO standards or PAS 78 which is being rewritten you will see they all recommend testing with disabled users rather than just doing an audit.
And finally a little bit about us. We are the only charity dedicated to ICT access for people with disabilities. And our consultancy team provides usability and accessibility services and we do have a range of clients both in the private and public sector.
Ok and that’s it thank you very much.
Sean: Thank you very much Judith, we’ve got time for just a couple of questions before we move on so you don’t have to ask a question but if you’ve got a question.
Woman from Manchester University: You mentioned on one of your slides something called Wizard of Oz testing. Do you mind just telling me briefly what that is?
Judith : Wizard of Oz testing, I just like that time it’s basically we manoeuvre things behind the scenes. So if you’ve got for example a paper prototype, say if you are testing a phone you might just have little cards that you move in and out. Or you might have a clickable prototype where not everything is programmed in but the particular user journey that you are testing does have certain things programmed in. You’ve got this image if you remember that bit from the Wizard of Oz where the Wizard was behind the scenes moving things around so its creating the image of things working when they are not actually working. But it actually doesn’t really make any difference to the users.