Responses from the online survey administered after the study. Be sure to click on the links for individual responses. Much to learn there.
The actual results of our study were much better than anticipated. The procedure worked very well, although we lost audio on one of the sessions. All five of our testers completed the study, seemed to have a good time, and said that they had learned more about the Library than they had known before.
Testers took between 21 and 55 minutes to complete all fifteen scenarios. Interestingly (thankfully?), the completion rate for the various tasks was very high: two testers did not complete 1 task; two testers did not complete 2 tasks, and one tester did not complete 4 tasks. Several of the tasks, though eventually successfully completed, puzzled the testers for several minutes – lots of hunting and pecking and heavy sighs. Many times, the testers took alternate routes to complete their tasks and we counted that as a successful complete.
In spite of the high success rate, certain scenarios were consistently more difficult than others. Even with only five testers, the problem areas of the website were very clear. For example, four of the testers had a very difficult time finding a “librarian-evaluated website”. These were buried under Subject Guides where no one ever went! That was very bad news; hence a new Resource Icon for Subject Guides. Finding an online tutorial on citation style was another difficult one, as was requesting the login/password for remote access. Sometimes the problem seemed that the testers simply weren’t reading carefully; other times, the links were buried too deep or were located in a place the testers didn’t think to look in. In the revamp of the site, we addressed as many of those issues as possible – duplicating links in alternate areas where our testers had looked, and bringing links (like the login/password request) closer to the surface.
It was very interesting analyzing the results. Watching the screen capture and listening to the running commentary was a fascinating look into how other people think. I would highly recommend using screen capture and audio if you are considering doing a study. Even with these tools, the analysis was very time consuming. As well as humbling and frustrating and even funny at times.
|Results of the Library Website Usability Post-Study Questionnaire|
|1. On the Library's home page, how easy is it for you to find the appropriate link for information you need?|
|2. What is the liklihood of you using this site in future for academic research?|
|3. How would you rate the site’s visual appeal?|
|4. How would you rate the amount of information and text on any given page?|
|Too much information and text||1||20%|
|Just about right||4||80%|
|Not enough information and text||0||0%|
|5. Do you like how the pages are set up with three columns?*|
|1. In the three columns you get more info and don't have to scroll down as far.|
|2. I certainly prefer the columns to a long list I have to scroll through.|
|3. sometimes it get too busy|
|4. I actually did not notice that|
|6. Do you like how the information is organized in boxes?*|
|I like everything seperated but there is an awful lot of text everywhere which makes it so nothing really jumps out at me.|
|7. How likely is it that you would leave comments, feedback, and ratings on the pages?*|
|The possibility exists||2||50%|
|8. What did you like the most about the website?|
|1. How easy it is to find articles and find books from home. I like that it is easy to get help in evatuating websites.|
|2. Once you get to know how to access certain information it is extremely valuable and easy|
|3. It seemed like the FAQ was more helpful than most websites. I love the "ask the librarians" feature! They actually answer you quickly! Keep up the good work. Time is of the essence when doing a project and most people wait until the last minute to start :)|
|4. being able to move around from task to task|
|5. that it anticipated most of my questions and had the answers|
|9. What did you like the least about the website?|
|It was hard to find the login passwords.|
|Getting off the beaten path feels uncomfortable. Everytime I open something new it looks completely different with different text styles and formatting. So I have to learn to navigate the website page all over again.|
|It's bland. Dull boring colors. Headings for important things do not really jump out to make it easy to find them. Everything seems to be of equal importance which can make it more difficult to find something specific. Way too much text and information everywhere to wade through. I like to read but come on...|
|I couldn't find minerva to sign in for my card to check on book. That was frustrating.|
|hm, I would like to be able to get answers on the weekends as i do much of my work then. i suppose that i need to get into the habit of getting my sources mapped out before the weekend arrives.|
|10. What suggestions do you have to improve the website?|
|A link to the passwords on the front page.|
|There is a ton of really cool stuff on the website that I don't think people know bout. It is uncomfortable to explore so people are unlikely to attempt it and discover the "Audiobooks" Feature for instance. I, however, am no website designer and couldn't possibly make recommendations.|
|Add some color. Maybe the tabs at the top could be color coded according to their purpose (ie blue for research, green for books, etc.)Use more attention grabbing headers and try to say more with less like instead of having an entire box full of words instead of just say "ask a librarian" or "FAQ". i like to find what I want quickly so I can save my time and efforts for the paper writing.|
|Other than try not to make it so busy not much.|
|I did miss a few things initially even though i thought that i was reading everything. Even though I did find themm later as i was looking for something else, i might have listed the buttons alphabetically. Not sure, I suppose I would try that and see how it looked All in all I think that you did a fabulous job! I know that i will be using it often. Thanks!|