InformalPrototype:Innoventions
From IEOR 170 Spring 2007
Contents |
[edit] Introduction and Mission Statement
The system that we are evaluating is our Wiki Wakey alarm clock. We invented the Wiki Wakey alarm clock to help people wake up on time and feel refreshed at the same time. People have different sensitivity to external stimuli such as light or sound. Our Wiki Wakey alarm clock will have several methods to wake people up. People can be woken up by using light or sound by our Wiki Wakey alarm clock. In this experiment, we will be testing different external stimuli to several experiment subjects and gather their opinion on which method works best in waking them up.
Our mission statement:
“To help people wake up on time and feel energized at the same time so that they are able to fulfill important tasks in their life.”
List of our group member and individual contribution to this assignment:
Nicolas Suryono: Completed the Mission Statement and took photographs during experiment, completed description for Prototype section, contributed to the appendix section. Compiled other member’s part into one final report (hardcopy) and uploaded group assignment to group wiki page.
Urvashi Gupta: Completed the Discussion section and became the “computer” during the experiment.
Monica Tanza: Did the demo for the experiment subject, wrote the script for the experiment, contributed to the Appendix section, completed the Prototype section and completed the Method section.
Hong How Quek: Completed the Results section, Contributed to the Appendix section, took notes and pictures during experiment.
Raymond Kim: Made the PowerPoint slides for presentation, did the in-class presentation, took notes during experiment.
Picture of Wiki Wakey Alarm Clock Interface
Descriptions: The top buttons are for day setting. User can set the time that they want to wake up by pressing the date first and then adjust the time that they want to wake up. Instead of using up down button to adjust the clock setting, we use a knob so the user can adjust the time by rotating the knob. We also included the snooze button, time button, off button (to turn the alarm off) and the AM/PM button.
Picture of light source that we use:
We use the above item as a prototype for our light source for our Wiki Wakey alarm clock. The left light source has a white light color and the right light source has a blue light color. The light frequency can be adjusted on both devices so that we can measure the user’s reaction toward both light color and light frequency.
[edit] Method
Since we have a fairly broad user group, we were not concerned about finding participants, since almost any college student would fit the requirement of using an alarm clock. To select a random group of students, we went to Bechtel Engineering Library and approached people that were awake and did not look too busy (we did consider testing the flashing lights with a sleeping person but could not since we did not have their signed consent). We asked if the person had time and was willing and made sure that they used an alarm. We then got them to sign the consent form.
We tested the subjects in 101 Bechtel. We choose this location for a couple of reasons. First of all, it is secluded, so others could not watch the procedure. This was important because not only did we want to ensure that the participant was comfortable, we had to make sure no one that would late be tested would see the test run through before hand. The other reason why 101 Bechtel was a good location was that it’s a small room with light control and closed blinds so we could turn off the lights when we got to the section where we needed to test out the different lights on the participant. We had the participant sit down at a corner with the facilitator at the opposite corner. The observers were across the table from the participant and the computer stood next to the facilitator.
Before testing on a subject, we tested the prototype in our team. Monica’s task was to be the facilitator. She made sure to get the consent form signed by participants and then read through the script (see appendix). Urvashi was the computer. For our prototype the computer had to do a few things. First of all, when the time or any of the day buttons were “pressed”, the computer had to pull off the top layer to reveal the adjustment mode. Then, when the dials were “turned”, the computer pulled the number strips through until the appropriate number was showing. Finally, the computer had to mark the lights with a piece of post-it note once that day had an alarm set. When we ran through our practice test, Ray was our user. By having him run through the tasks, we discovered that our original alarm schedule (which had five days with alarms, two without) was tedious to fill out, so we cut it down to three set days. For the actual test, Ray, Andrew and Howie were observers. Ray and Howie had their laptops out and wrote down both the verbal and physical reactions that the participant had. They also took note of how the participant was using the prototype and if there were any differences between our idea of the system and the user’s idea of the system. They also noted the answers to the questions at the end of the test. Andrew also took notes as well as photos of the testing procedure.
We followed the procedure outlined in the script. After we got the consent form signed, the facilitator showed how one would set the time on the system. They then asked the participant to complete their first task which was setting the alarm as they would need it for tomorrow. After that task was completed, the user was given their second task, which was to set the alarms for a given alarm schedule. After they completed both of those tasks, we tested the flashing lights on them. We told them that we were testing different lights as a means of waking someone up. We turned off the light and asked them to close their eyes. We then tested out three different lights. There was the blue light at a high frequency, the blue light at a low frequency and the white light at a high frequency. We then asked them some follow up questions on both the lights and the paper prototype. Lastly, we made sure all their questions were answered and then thanked them for their time.
We were looking out for a few things during the procedure. First of all, we were watching to see how the user interacted with the interface and took note of all their remarks of confusion. We were trying to make sure all the different buttons made sense. We also wanted to make sure that the method of using a dial to set time was useful. For the flashing light, we asked them to rate the light on a scale of 1 to 10, with 1 being very comfortable and 10 being very uncomfortable. We asked them to rate the light both when their eyes were closed and when they were open. We also asked it they thought the specific light could wake them up. This was a difficult thing to test, since they were conscious. Because of this, we tried testing the lights on our sleeping roommates. Monica was able to wake her roommate up with the blue light, but was later told that her roommate’s alarm had gone off once and then been turned off, so her roommate might have been in a lighter slumber than usual. Howie tried to test the white light on his roommate, but his roommate woke up when Howie opened the door. We are hoping to have more opportunities to try to wake up our roommates so that we can test the efficiency of the lights as a way of waking someone up.
[edit] Results
I. Alarm Clock Interface
We tested the paper prototype of the alarm clock interface on 5 test users, who encountered a range of problems when they interacted with the interface. The major concerns raised by our users are summarized as follows:
1. Absence of an ALARM button
When directed to set the alarm according to the schedule we provided, the first reaction of 3 out of 5 users was to search for an explicitly labeled ALARM button they can use to toggle the interface to an alarm time setting mode. These users intuitively thought that the TIME button was used to set the current time, and not the alarm time. This was a major source of confusion, and 2 out of 3 users ended up not pressing the TIME button at all, which is a major error as the user cannot go into the alarm time setting mode if that button is not pressed.
2. Absence of Action Feedback
2 test users commented that they could not ascertain if they had set the alarm time successfully because of the absence of an action feedback response. They mentioned that any form of feedback would have be very helpful, for example a short “beep” sound if the alarm is set successfully and a longer “beep” sound to alert users of errors, if any.
3. Tendency to mistake the SNOOZE button for the OFF button
2 test users mistakenly hit the SNOOZE button when what they wanted was to permanently turn off the alarm (even though the button was explicitly labeled). They did that because the OFF button was very much smaller in size compared to the SNOOZE button and thus more difficult to locate. Also, previous experience with alarm clocks made them intuitively regard the biggest button on the interface to be the OFF button.
4. Pressing Buttons in the Wrong Order
2 test users pressed the buttons in the wrong order—instead of pressing the TIME button to go into the alarm time mode before clicking on a DAY button to set the alarm for that day, they pressed the buttons the other way round (a big mistake, because that causes the alarm time to be reset!). Users complained that it wasn’t apparent what the correct order was, so getting the right order becomes a matter of trial and error when it shouldn’t be.
Despite the problems users encountered, they also had many positive comments about the interface layout. For example, they liked how turn dials were used instead of buttons to adjusting the hour and minutes, as dials are a lot more responsive than buttons and also more comfortable to use. Also, they thought that the interface was easy to read, as buttons were adequately spaced out and did not overly clutter the interface.
II. Flashing Lights
Users were asked to rate their experiences when lights of varying colors, intensity and flashing frequencies were shone into their eyes (when opened and closed). They were asked to rate their comfort level on a scale of 1 (most comfortable) to 10 (most discomforting). The average ratings we recorded (correct to 1 decimal place) are shown in the table below:
Column 1: BLUE light 1 (Intensity: Medium,Frequency: Low)
Column 2: BLUE light 2 (Intensity: Medium,Frequency: High)
Column 3: WHITE light (Intensity: High,Frequency: Low)
Eyes Closed 5.3 5.5 6.5 Eyes Opened 6.5 7.8 8.5
Users varied tremendously in their responses for this section of the experiment. A user mentioned that blue lights put people to sleep instead of waking them up, while another user felt that colored lights were more distracting than white light. While two users felt that the lights were far too intense and might have given them a headache if they stared at the lights for too long, another user felt that the lights may not be bright enough to wake her up from sleep. However, none of the users reported any acute pain during the experiment beyond the levels of discomfort that were expected.
Despite the wide ranging responses, we can confidently make these conclusions when we look at the average ratings:
1.The more intense the light source, the more discomfort the user feels 2.The higher the flashing frequency, the more discomfort the user feels 3.An increase in light intensity increases discomfort level more than an increase in flashing frequency
[edit] Discussion
There were a lot of observations we made during the experimental runs and our paper prototype helped us identify a lot of the possible problems that people would encounter with our product. These problems can be grouped into different categories and severity ratings, depending on how crucial they are to the accomplishment of the tasks the user sets out to complete.
The main problem was the users’ confusion as to how to get into the mode of setting an alarm on the alarm unit. We feel this is definitely a rank 5 problem, which is usability catastrophe. This is because the unit will be useless to people if they can’t figure how to even begin setting the alarm clock, let alone facing difficulties in setting the alarm. This has to be fixed so that users know for sure when they are in alarm mode and they can start setting the times. We can do that by having our button labeled as “alarm” instead of “time” so users don’t get thrown off and have the assurance that hitting alarm enables them to start setting an alarm.
The next major issue was not being able to set the time on the alarm clock correctly by pressing buttons in the wrong order. This is also a rank 5 problem, as it follows closely with the above problem of not being able to correctly get into alarm mode and setting the right time. This problem will inconvenience the user greatly by waking him up at a time he doesn’t want to be woken up or not even waking him up at all. This is because pressing the buttons in the wrong order resets the time instead of setting the alarm to the time specified by the user. Hence, we have to figure out a way to make it clear to the users that they press the “time” button to enter alarm mode before hitting the “day” button to set the alarm.
Due to affordances, users also made the mistake of confusing between the “off” and “snooze” buttons. This is mostly a minor usability problem, as simply labeling those buttons or making the “off” button more prominent in the interface will take care of the problem. Also users will realize what button is for snooze and what is for off after using the alarm unit a few times, and will get used to it. Another suggestion we got was to use military timings so there’s no need for the AM and PM toggle and people don’t have any confusion in setting the times either.
Users suggested having a sound feedback system in addition to the light feedback system we currently have in place to let people know that they have correctly set their alarm. We feel this is a minor usability problem. Which feedback system to use just depends on what most users are already used to given the current alarm clocks in the market and we feel it would be better to use either lights or sounds by testing this specifically on several users and then drawing a conclusion from those results.
The flashing lights proved to be very tricky, as its important to have just the right level of discomfort that is required to wake the user up, yet have it comforting enough to not put the user in a bad mood or give him “bad” dreams as one of our volunteers put it. This isn’t really a usability problem but the problem is more of figuring out the right intensity that would be suited to the nature of the task at hand.
At the end of all the experiments and analysis of the data, we realize that obviously some tasks, like being able to set the time on the alarm clock correctly are a lot more important than other problems that may not have such a big impact. Our aim is to tackle these problems starting with the most important ones so that there is a reasonable trade off reached in the final design, and our product is as simple and intuitive to use as possible.
[edit] Appendix
Attached in the appendix:
1.Experiment script 2.Wiki Wakey Alarm clock paper prototype 3.Statement of informed consent from 5 experiment subjects 4.Observation notes 5.Pictures
