RAA 5: User-Centered Design and Usability Testing of a Web Site

Corry, M., Frick, T., & Hansen, L. (1997). User-centered design and usability testing of a Web site: An illustrative case study. Educational Technology Research and Development, 45(4), 65-76. doi:10.1007/BF02299683

The authors of this article were given several tasks from administrators at Indiana University. They were to determine how useful the current university website was through needs analysis and usability tests, and then develop a new site that would better meet the information needs of users.

A needs analysis was first conducted. The authors interviewed 35 campus departments to determine most frequently asked questions. These questions were put onto index cards and were used in card sorting by frequency, in which over 30 categories were revealed. These findings were used to create a first paper prototype.

Usability testing was then conducted with 21 people, through usage of paper versions of both the original website and the new prototype. Participants could only view one page at a time and were asked a think aloud while they answered 15-20 questions for each website.

A second phase of usability testing was then conducted with 16 participants, focusing only on the newer website. Changes that were made before testing included renaming links, reducing multipage nodes to a single page, and organizing university departments into a long list of alphabetized links.

Once usability testing using paper prototypes were completed, the authors conducted another usability test with an online version of the newer website, using 11 participants. You can tell that this article is dated because the website was tested on Lynx, Mosaic, and Netscape browsers by all participants.

Lastly, a second testing with the computer prototype was conducted to look at the changes that were made to fix the problems identified in the previous phase.

Main Findings
The first paper prototyping and usability testing revealed that the proposed website was more usable than the existing, when finding most-frequently asked information. In general, participants were often faster and more successful when completing tasks with the new prototype.

Results of the second usability testing helped identify more links that were confusing and/or misleading.

As for the usability testing on the computer prototype, there were several problems identified including too many key presses and scrolling to navigate. These problems often had to do with the browsers they were using.

In the second phase of testing the computer prototype, there were higher success rates than the phase before it due to clearer navigation and terminology, fewer keystrokes required, and more of a breadth-based navigation structure.

I thought this article had a lot of commonalities with what our Computer Interaction Design class was doing right now. The authors basically used an iterative process to clarify and reorganize the information architecture of the university’s website. Similarly, our class is taking the information from nanoHUB.org and using card sorting and usability testing to validate our own information architecture. That being said, this was a helpful reading to further understand the process we will be going through in class.

I would also like to mention that this article did well in putting the information we learned about IA into context. For example, using breadth rather than depth for navigation structures, and limiting information to one page because users will often ‘suffice’ and not even bother looking at the next page. Overall this reading was a very good supplement to our current course content, despite being dated. But then again, I guess that shows how some design guidelines tend to be timeless.

RAA 4: The Use of Guidelines in Menu Interface Design

Souza, F., & Bevan, N. (1990). The use of guidelines in menu interface design: Evaluation of a draft standard. Proceedings of the IFIP TC13 Third International Conference on Human-Computer Interaction, 435-440. Retrieved from http://dl.acm.org/citation.cfm?id=725751

This article reported that only few designers religiously follow design guidelines. For this reason, the authors evaluated the extent that designers are able to use such guidelines to offer new improvements in clarity and efficiency.

By developing the guidelines further, it would help improve accuracy and present information in a way that makes them more usable. However, refinement doesn’t mean that designers would necessarily use them in their interface design processes.

Three interface designers were given a set of 87 guidelines in which they marked any difficulties or terms they found unclear. Later, they were observed during a study in which they were to identify and redesign problems using a whiteboard, but were encouraged to use the guidelines. At first they were not obligated to follow them, but were asked to think aloud their reasoning. Afterwards, they were told to change the new interface by applying all the guidelines one by one.

Main Findings
91% of guidelines resulted in errors with at least one designer, however only 11% of the guidelines were actually violated by their new design. The authors found that the designers tended to misinterpret the guidelines and mainly focus on prior designer experience. Examples the paper provided show lack of clarity for conditions and nature of guidelines and difficulties with certain terms.

As a designer the results were not surprising. Personally, I often rely on past experiences rather than the clarity of guidelines. I also think that guidelines are just that– guidelines: encouraged to be followed but you should know when they can and should be broken.

The difficulties the designers had were also relatable. For example, reading about the design process for class and applying them later in class is a completely different matter. I often find myself not knowing how to effectively apply a process until experiencing it firsthand.

I would say that clarifying guidelines is a good proactive and should be done, but this study revealed that following them isn’t completely necessary to make an exceptional design.

RAA 3: The effects of cognitive ageing on use of complex interfaces.

CognitionReddy, G., Blackler, A., Mahar, D., & Popovic, V. (2010). The effects of cognitive ageing on use of complex interfaces. Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction, 180-183. doi:10.1145/1952222.1952259

Many older adults find it difficult to use modern products because of their function and interface design. Past research shows that the decline in cognitive functioning as you get older affects your speed and accuracy when using complex technological products, but another study shows that effectively using a product is not generation-specific or different depending on your age.

Therefore, the authors of this paper wanted to look deeper at how cognitive aging and prior technology experience affect using such complex interfaces.

37 subjects between ages 18 and 83 participated in the experiment. First, they were given an information package, consent form, eye acuity test, and a questionnaire on their technology experience. Then they were given trials that involved tasks with a body fat analyzer. Task time, errors, body language, and facial expressions were recorded at this time. After, the subjects were given a post test on their technological experience, and used an application to measure “different aspects of cognitive function” (p. 181).

Main Findings
The researchers found a significant negative correlation between experience with technology and task time. Younger people were more likely to be tech-savvy and take less time.

There was also a strong negative correlation between sustained attention, task time, and errors. Older adults were less able to sustain attention.

Lastly, older adults tended to make more errors than the younger participants and also took longer to recover from errors. The authors state that this opposes the Processing-speed Theory proposed by Salthouse (1996), which suggests that there would be minimal differences between older and younger performance if given unlimited amount of time. The experiment had no time restrictions, and therefore the findings contradict this.

Again, due to the subject being elderly and technology, this does relate to my thesis. I thought that the contradiction to the Processing-speed theory was an interesting find and that the experiment resulted good ground work on how complex systems are more difficult for older adults due to cognitive aging. I also thought it was surprising that they had older subjects (aged 60-83) that had decent experience with technology. Although the graph they supplied made it obvious that the time it took to complete tasks were higher than that of younger participants. Maybe the older participants just felt the need to take their time, and the younger ones just wanted to get out of there? A possible threat to internal validity, I suppose.

RAA 2: The use of communication technology by older adults

I just noticed that my previously scheduled dates for finishing my RAAs are incorrect, since the actual due date for all of them are November 1st, not the end of the semester like I had thought. Therefore, I will have to complete one RAA every week until the 1st starting now.

Elderly On Computer

Melenhorst, A., Rogers, W., & Caylor, E. (2001). The use of communication technologies by older adults: Exploring the benefits from the user’s perspective. Human Factors and Ergonomics Society Annual Meeting Proceedings, 45(3), 221-225. doi:10.1177/154193120104500305

The authors’ goal was to investigate the benefits that older adults saw from using communication technologies such as email or telephone.

The article focused on the fact that older adults are selective with their actions – weighing the cost with the benefits before ever committing to something. This means that there is some benefit they see from using the communication methods they use, but what?

13 focus groups were formed with a total of 48 older adults in independent living. They ranged from 65 to 80 years of age.

These participants were given a questionnaire at home for their demographics and background information. Once in a focus group, they were given booklets that contained scenarios of communication that an older adult could experience in everyday life. Participants then discussed the scenarios in 2.5 hour sessions, giving real examples from their lives and pointing out their motivation for using such communication.

Sessions were recorded on audio tape, then transcribed before being analyzed to discover three things: the medium or method, the communication scenario, and the motivation or consideration for using the method. Motivations were then categorized as a cost or benefit.

Main Findings
The benefit of a communication method is very context-related, but all users recognized the fact that email increased communication frequency. Alternatively, participants felt that email was not intimate or as interactive as a phone. Plus, there was the cost of the effort it would take to learn how to use email.

All participants also highly valued personal visits, where the costs were minimal but were from too much (undesirable) intimacy and the time and effort a visit takes. Regardless, they were found to be irreplaceable for most.

This article very much relates to my thesis on elderly and the iPhone and has touched on a reoccurring theory in my research: the Socioemotional Selectivity Theory. While the research focused more on email, I will definitely remember their thinking on context-related perceived benefits. I would also like to remember that, in general, older adults view communication as a positive concept, and that knowledge of the benefits of a specific media likely determines if this audience will use them.

RAA 1: The Inference of Perceived Usability From Beauty

Hassenzahl, M., & Monk, A. (2010). The inference of perceived usability from beauty. Human-Computer Interaction. doi:10.1080/073700242010500139

The purpose of the study was to re-examine the relationship between beauty and usability, since a review of papers showed high variation in results. The authors assumed it was due to inconsistent methodologies, so they wanted to take another look at the implication that “what is beautiful is usable”.

The authors created a questionnaire that had participants rate websites in terms of beauty and goodness. Four studies were constructed. (1) 60 participants received a random list of 10 E-commerce websites and were told to browse each home page briefly before rating them on their “first impressions”. (2) 10 female students rated all 60 websites that were chosen for the study, which took around two hours. (3) 57 students were given only 30 seconds to get an overall feel for 30 different websites before rating them. (4) 430 participants took a questionnaire about 21 different websites.

Main Findings
Six parallel analyses on the data showed a similar conclusion: that the pragmatic and hedonistic qualities of websites are related. The article concludes that their explicit model presents a correlation between the two, allowing for a better understanding of the relationship between beauty and usability.

I’ve previously looked into articles about accessibility and aesthetics, however they were not empirical. This article gives some real data about the relationship between usability and aesthetics that relates to my interest in interaction design. Possible concerns would be that websites in study 4 had to be translated from German to English. While content may not noticeably alter the look and feel of a site, I would indeed say that is would affect the usability of it.

RA4: Looking at, looking up or keeping in touch with people?: motivations

Joinson, A. (2008). Looking at, looking up or keeping in touch with people?: motivations. Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems . Retrieved from http://portal.acm.org/citation.cfm?id=1357213

The purpose of this paper was to investigate the gratifications that users derive from using the social networking site, Facebook.

For their first study, 137 users were asked to describe how they used Facebook, and what they enjoyed about it. The phrases they came up with were then grouped and categorized into certain types of uses and gratification. Those items were then used in the second study, where 241 Facebook users were asked to rate how important these uses were to them. Afterwards, an exploratory factor analysis was conducted to determine which of these gratifications were most valued.

From the first study, generated themes were categorized and ranked as follows: keeping in touch, social surveillance, re-acquiring lost contacts, communication, photographs, design related, perpetual contact, making new contacts. In keeping with previous research, using Facebook to “keep in touch” was mentioned the most with 52 mentions.

The second study lead the identification of seven specific uses and gratifications of Facebook: social connection, shared identities, photographs, content, social investigation, social network surfing and status updating. The following images show the amount of use for each factor: (The higher number, the more use.)

These factors were found to have correlations with one another in some cases. For example, photographs were strongly correlated to social connection.

There was also report of females acting differently than males in terms of gratification and uses. For example, females were more likely to make their profile more private.

This article investigates social networking sites using a “uses and gratification” framework. I thought the approach that was taken was well thought-out and that the researchers were able to obtain valuable information. This data is very helpful to my own research in TECH621, related to the value of specific Facebook features.

In general, I found the design implications to be very helpful and extensive. Much of the quantitative data and statistics were difficult to understand, but the charts were helpful. Overall, it revealed and analyzed the presented uses well and I would say it is an essential read for timely data on gratifications for social networking features.

RA3: Changes in Use and Perception of Facebook

I hate when I forget to publish things… Does it count that I wrote this in EverNote on March 20th?

Lampe, C., Ellison, N., & Steinfield, C. (2008). Changes in use and perception of Facebook. Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work. doi:10.1145/1460563.1460675

The purpose of this study was to examine if users perceived and used Facebook differently over a 3 year period (from 2006-2008). Many studies previously done on the use of Facebook has taken place at a static point in time, as opposed to a period of time, which means that changes made since then may or may not have altered the perception of the site. Research questions to be answered are as followed:

How has the reported use of Facebook to interact with other members changed over time?
Do people still use FB mostly to stay in touch with existing offline networks?

How has the perception of audience on Facebook changed over time?
Do people still assume that their FB profile is only viewed by peers?

How have the attitudes of users towards Facebook changed over time?
Do people still believe that their profiles portray them both accurately and positively?

The researchers conducted surveys about use and perceptions of Facebook, using randomized samples of MSU undergraduate students. To track possible changes over them, these surveys were done in 2006, 2007, and 2008.

Questions that were asked were as follows:
“I use Facebook to…”
“Since you have created your profile, who do you think has looked at it?”
“Please rate the following attitudes you have towards Facebook:”

Interviews were then conducted within the group of respondents, totaling to 18 interviews, where students were asked more in depth about their Facebook activity and choices.

The researchers found that Facebook use remains consistent over time – Users continue to utilize Facebook to keep in touch of offline friends. This data is often reflected in other research about Facebook usage. However, some interview respondents also admitted to using Facebook to find out more about people they intend on socializing with or dating in the future.

In terms of using Facebook to learn more about people in class, numbers increased slightly. This might be due to the change in Facebook interface in 2007 when the course list was removed, making it harder to find classmates.

The number of friends and time spent on Facebook increased at first and then leveled off, which from interviews, suggest that new users spend time adding people as friends and getting used to the site. After a while, this behavior lessens as time is more spent seeing what is happening to friends instead of expanding their friend base. Also, new users are more likely to use Facebook to “Find people to date” or “Meet new people” than long term users.

As for perception of audiences, users believed that strangers were less likely to view their profile in later years. This might be due to changes in the Facebook interface as well, since the creation of the News Feed, the removal of the “Browse Network” option, and increase privacy settings makes finding viewing profiles a different process. They also believed that family members were more likely to view their profile in later years, likely due to the change in who could obtain a Facebook profile. Changes and new features such as these were found to make the site more compelling for some and deterred others.

Finally, attitudes of the site has changed over time, likely because of the change in features offered or because the change in population. For example, Facebook was found to be more integrated in daily routines and everyday activities than in the past. Fewer people consider it “just a fad”, as more and more are becoming attached to the service and the accessibility of information it provides.

This article was easy to follow and the experiment conducted presents data well, however, I thought that the actual results of the experiment were somewhat predictable. For example, much previous research has proven that users utilize social network sites to keep in touch with offline contacts, and while this article examines that through a period of time as oppose to one instance, the results were the same. In this case, I would not say much was contributed to the greater body of knowledge from that finding. Other findings proved to be more beneficial, focusing on how users perceived Facebook and their audience. These perceptions changed over time, which were likely due to interface design changes, giving future designers information about how changes effect user perception.

Overall, I found the results to be pretty interesting and highly relevant to my interesting in interactive design. Change does in fact alter how people perceive and use things, and this was a great example of that. Facebook users may bicker a lot when changes are brought about, but this research shows that it does not cause them to feel significantly less about the network site; after all, attachment to the site increased! Perhaps users feel they have invested too much into it to leave when a disliked change occurs, or maybe there is no other alternative network site they feel is superior enough to change over to. Maybe these changes were made for the better and they grew to like them. I’m guessing it’s a little bit of all the above, but it would be interesting to discover what is the driving force that causes users to stay with a service when unfavorable changes to it occur.

RA1: Neomillennial User Experience Design Strategies

Baird, D.E., & Mercedes, F. (2005). Neomillennial user experience design strategies: Utilizing social networking media to support “always on” learning styles. Journal of educational technology systems, 34(1), 5, Retrieved from http://baywood.metapress.com/link.asp?id=6wmw47l0m81q12g1

Because today’s generation is very much “net-centric”, this paper focuses on investigating a teaching curriculum designed around the digital world and the integration of social networking, user experience design strategies and other technologies. It also explores how new social media can support these learning styles and aid in the creation of future learning communities. The authors’ overachieving goal is to help instructors and course directors find ways to use the digital world to their advantage to enhance the learning experience for students.

The researchers looked at current learning theories as well as theories that attempt to integrate technology within them. First, they examined current course designs and certain cognitive events that occur during learning. Then they refined their understanding of instructional styles before discussing other technology alternatives to delivering information to achieve the same results. How users navigate and learn through the web is observed, specifically touching on certain technologies such as RSS feeds and weblogs.

Main Findings
Because this article focused on examining previous knowledge on learning styles, many of the main findings were examples of current theories and the types of possible technologies to use. They found that certain events that occur during learning can be applied to the web. For example, to gain attention of the student, you present that data; so on the web, you would present the information on the screen. They discover how users read the web; for example, hypertext links prove to serve as one form of highlighting. Students’ perceptions of blogging and the psychology behind language and design use on the web were also found due to their research. From these findings, they suggest possible technologies.

This subject was well researched, but in that sense, it was very much research-focused. A lot of background information was given on current learning theories, and while it transitioned well into how these theories can be integrated with social media, this claim was backed by second hand knowledge. When presenting possible technologies, much of the section was spent explaining and defining the technology, with few lines claiming how it could be used within an academic setting.

Regardless, I think that the compilation of information was very well done and reviewed. While I might not be interested in learning how technologies could be applied in this sense, the theories discussed presented great knowledge to my interest in design. Specifically, the psychology of navigation and how users read the web. These standards were what I was hoping to find in an article and is what I took away most from it.

In conclusion, great review of previous research and a decent starting point for instructors and course directors to follow in terms of integrating social media within a curriculum. I would, however, be interested in research done to see how each of these suggested options actually effect learning.