Library Website Usability Study: Results

Responses from the online survey administered after the study.  Be sure to click on the links for individual responses.  Much to learn there.

The actual results of our study were much better than anticipated.  The procedure worked very well, although we lost audio on one of the sessions.  All five of our testers completed the study, seemed to have a good time, and said that they had learned more about the Library than they had known before.

Testers took between 21 and 55 minutes to complete all fifteen scenarios.   Interestingly (thankfully?), the completion rate for the various tasks was very high:  two testers did not complete 1 task; two testers did not complete 2 tasks, and one tester did not complete 4 tasks.  Several of the tasks, though eventually successfully completed, puzzled the testers for several minutes – lots of hunting and pecking and heavy sighs.   Many times, the testers took alternate routes to complete their tasks and we counted that as a successful complete. 

In spite of the high success rate, certain scenarios were consistently more difficult than others.  Even with only five testers, the problem areas of the website were very clear.  For example, four of the testers had a very difficult time finding a “librarian-evaluated website”.  These were buried under Subject Guides where no one ever went! That was very bad news; hence a new Resource Icon for Subject Guides.  Finding an online tutorial on citation style was another difficult one, as was requesting the login/password for remote access.  Sometimes the problem seemed that the testers simply weren’t reading carefully; other times, the links were buried too deep or were located in a place the testers didn’t think to look in.  In the revamp of the site, we addressed as many of those issues as possible – duplicating links in alternate areas where our testers had looked, and bringing links (like the login/password request) closer to the surface.

It was very interesting analyzing the results.  Watching the screen capture and listening to the running commentary was a fascinating look into how other people think.  I would highly recommend using screen capture and audio if you are considering doing a study.  Even with these tools, the analysis was very time consuming.  As well as humbling and frustrating and even funny at times.