System Prototype and Evaluation Plan
Project Part 4
Redeye Rovers
November 9, 2005
INFO 608
Jennifer Falasco
Michael Gallagher
Craig Nelson
Jimmie Schmidt
Introduction
The Redeye Rovers have developed a website designed to serve the needs of the members of the Quinnipiac Valley Audubon Society. For the purposes of this project, the developed prototype has been implemented and can be viewed at:
http://www.pages.drexel.edu/~jjf46/index.html/index.html
The following report outlines the evaluation plan used to address the website as well as a table of usability specifications complete with a narrative. Furthermore, a description of the prototype itself has been included with screenshots to illustrate the nature of the design. A brief discussion follows relating the implementation issues the Redeye Rovers experienced along with justifications for our choice of design. The structure of this report is as follows:
I. Proposed evaluation plan
II. Usability specifications table with narrative
III. Prototype description
I. Proposed Evaluation Plan
Evaluation of the Quinnipiac Valley Audubon Society will be conducted with improvement of the website as the primary purpose. The usability specification table represents the majority of tasks QVAS website users will need to perform. To determine usability observation will need to be the primary means of evaluation. The optimal number of respondents would be a large number of participants. More participants in the usability evaluation will increase the number of improvements to the QVAS website. Constraints must be employed so a group of four will be used as participants in the usability specification portion of evaluation. Feedback from the usability specification table will be applied to the QVAS website for the final presentation of the website.
The final website presentation will be evaluated using an online questionnaire format. Due to distance constraints members of the design team will not be able to implement questionnaires in person. The use of an online questionnaire can be justified because of two criterions. Users of the QVAS website are presumed to have a working knowledge of the Internet and websites. If users do not have online experience the website will not be a particularly functional tool. With this predefined knowledge the evaluation can be extended to an online questionnaire. Additionally an online questionnaire will enable the Redeye Rovers to reach a larger number of participants in the evaluation process.
The questionnaire will implement both a rating system, or interval scale, format and open ended questions for participants. The questionnaire will be designed to obtain ease of use and overall satisfaction with website design. In earlier surveys the user task list was revised according to results. This questionnaire will try to determine if the user task list is still acceptable.
II. Usability specifications table with narrative
The following tables provide a detail matrix on the performance of some of the major performance categories, such as time and difficulty of use, for major interaction scenarios with the website. The table specifies the usability attribute being measure in this case, which is based on the initial performance of the system for a first time user. Each scenario is identified in the “Measuring Instrument” column. The performance category being measured is contained in the “Value to be Measured” column. The “Current Level” column depicts the measured result of a single user and the “Target Level” column is the measure that is expected of the system. The Time performance categories are measured in seconds it takes to complete the interface scenario and the Difficulty category is measured in terms of low, medium and high. The first table displays results from an individual user navigating the eight major information display links of the site dealing with time it takes to click a link and display the results. The second table measures the difficulty level of performing the navigation on each of the eight informational links. The third table displays results from interaction performance measures and expected performance measures with the two major form types, the material request form and the membership form. It isolates each process in the form and provides target performance criteria for each process within the form. The forth table provides the detailed performance criteria for the administrative maintenance for the site this includes adding an item to the inventory, editing an exiting item in the inventory and deleting an item from the inventory. It also includes performance criteria for site membership maintenance that includes adding a member from this particular portal by an administrative member of QVAS, editing the information of an existing member and deleting a person as a authorized member of the site. USABILITY SPECIFICATION TABLE
Navigation Specifications for Time
Usability Attribute
Measuring Instrument
Value to be Measured
Current Level
Target Level
Initial Performance
Navigate to and view Mission Statement
Length of time to navigate
5 seconds
5 seconds
Navigate to and view Hours of Operation
Length of time to navigate
5 seconds
5 seconds
Navigate to and view Directions
Length of time to navigate
5 seconds
5 seconds
Navigate to and view Contact Information
Length of time to navigate in Seconds
5 seconds
5 seconds
Navigate to and View Frequently Asked Questions
Length of time to navigate in Seconds
5 seconds
5 seconds
Navigate to and view Membership Form
Length of Times to Navigate in Seconds
5 seconds
5 seconds
Navigate to and view Volunteer Opportunities
Length of time to navigate
5 seconds
5 seconds
Navigate to and view Material Request Form
Length of time to navigate
5 seconds
5 seconds
USABILITY SPECIFICATION TABLE
Navigation Specifications for Difficulty
Usability Attribute
Measuring Instrument
Value to be Measured
Current Level
Target Level
Initial Performance
Navigate to and view Mission Statement
Difficulty level to view Mission Statement
Low
Low
Navigate to and view Hours of Operation
Difficulty level to view Hours of Operation
Low
Low
Navigate to and view Directions
Difficulty level to view Directions
Low
Low
Navigate to and view Contact Information
Difficulty level to view Contact Information
Low
Low
Navigate to and view Frequently Asked Questions
Difficulty level to view FAQ
Low
Low
Navigate to and view Membership Form
Difficulty level to view Membership Form
Low
Low
Navigate to and view Volunteer Opportunities
Difficulty level to view Volunteer Opportunities
Low
Low
Navigate to and view Material Request Form
Difficulty level to view Material Request Form
Low
Low
USABILITY SPECIFICATION TABLE
Form Specifications
Usability Attribute
Measuring Instrument
Value to be Measured
Current Level
Target Level
Initial Performance
Search for an item
Length of time to
search
15 seconds
15 seconds
Add an item to list of requests
Length of time to add an item
10 seconds
10 seconds
Initiate request
Length of time to send request and receive conformation
20 seconds
20 seconds
Initial Performance
Search for an item
Difficulty level to
search
Medium
Medium
Add an item to list of requests
Difficulty level to add an item
Low
Low
Initiate the request
Difficulty level to send request and receive conformation
Medium
Medium
Add a member
Length of time to add a member
90 seconds
90 seconds
Add a member
Difficulty level to add a member
Low
Low
Process a request
Difficulty level to process recv request
Low
Low
USABILITY SPECIFICATION TABLE
Administrative Maintenance Specifications
Usability Attribute
Measuring Instrument
Value to be Measured
Current Level
Target Level
Initial Performance
Add an item to inventory
Length of time to
add item
40 seconds
40 seconds
Edit an item in inventory
Length of time to edit an item
20 seconds
20 seconds
Delete an item from inventory
Length of time to delete an item
10 seconds
10 seconds
Initial Performance
Add a member
Length of time to
add a member
40 seconds
40 seconds
Edit a member
Length of time to edit a member
20 seconds
20 seconds
Delete a member
Length of time to delete a member
10 seconds
10 seconds
Benchmark 1) The viewing links in the About category that includes mission statement, hours of operation, directions and contact information should only require at most two clicks of the mouse to get to the content of the selections.
Benchmark 2) The viewing links in Get Involved that include volunteer opportunities and membership application should take no more than two clicks of the mouse to get to the actual content of the selection.
Benchmark 3) The viewing links in the About category that includes mission statement, hours of operation, directions and contact information should require no more than 5 seconds to get to the content of the selections.
Benchmark 4) The viewing links in Get Involved that include volunteer opportunities and membership application should take no more than five seconds to get to the content of the selection.
Benchmark 5) The process of adding a member through the online membership form should no more than 90 seconds to complete and submit.
Benchmark 6) The process of editing a member’s information through the online membership maintenance portal should take no more than 100 seconds to complete and submit.
Benchmark 7) The process of deleting a member through the online membership maintenance portal should take no more than 15 seconds to complete from locating records to submitting and confirming.
Benchmark 8) The process of searching and locating an item in the material request database and then adding it to the browser’s request list should take no more than 40 seconds.
Benchmark 9) The process of submitting a request list and receiving confirmation email should take no more than 20 seconds.
Benchmark 10) The process of adding an item to the database through the online administrative maintenance portal should take no more than 30 seconds to complete and submit.
Benchmark11) The process of editing an item in the database through the online administrative maintenance portal should take no more than 30 seconds for a novice skilled browser user to complete and submit.
Benchmark12) The process of deleting a item in the database through the online portal should take no more than 15 seconds to complete and submit.
III. Prototype Description
The prototype for the proposed website for the Quinnipiac Valley Audubon Society has been designed according to the needs and specifications of the user groups as defined by the previous components of this project. The prototype is designed to be simple in nature, requiring as few steps as possible to navigate through the various functions. Furthermore, the prototype attempts to match the visual medium to the topic being presented, namely the national Audubon Society website. Therefore, the color coordination and general aesthetic of the prototype were designed to meet that end.
Although full systems implementation was beyond the scope of this prototype, the Redeye Rovers have enabled a few of the various functions that a user might encounter in a realistic situation. The homepage of the prototype is functional, offering information pertaining to the purpose of the Quinnipiac Valley Audubon Society as well as the nature and purpose of Riverbound Farm. Furthermore, the links are arranged on the left in order of importance as dictated by frequency of use. The links are further grouped according to common relevance, with similar items grouped together. This was intentionally organized to offer the user the most efficient access to the information they so require.
The links proceed with information concerning the mission of the Quinnipiac Valley Audubon Society (QVAS), as well as information involving the hours of operation, directions, and contact information. This cluster of links represents the general information that would best serve to indoctrinate new users into the purpose and scope of QVAS. This group of information was considered to be of great importance by the survey results of our subject group.
The next link, clustered alone, is the Frequently Asked Questions (FAQ) link. This link was generally regarded to be of such importance that it was isolated to promote use. The Redeye Rovers assumed that users would be inclined to look for information if general questions were located within one area. This isolated link would enable the user to look for the answers to common questions in an efficient manner. This assumption was supported by the survey results, which polled the Frequently Asked Questions (FAQ) link as being of significant importance. Furthermore, the FAQ link supports the needs of one of the more significant user groups as determined by previous analysis, that of the students. Students would find it useful to have all answers to general questions located in one area. If the information required is not located in the FAQ area, information located within the link would direct the student to pursue other options, namely the Online Library function.
The Online Library function is located directly below the FAQ link, providing a seamless logical transition between the two related pieces of information. The Online Library is a function that requires users to be a member of the Quinnipiac Valley Audubon Society. In order to reduce any potential frustration users might have in navigating this area of the site, the Redeye Rovers felt it best to include a caveat within the link itself to offer warning to non-members. This represents a trust building exercise on the part of the design and their burgeoning relationship with the users. The Online Library is a online database of materials in the possession of QVAS. These materials can be requested and delivered if the user is a registered member. This is an attempt to represent the needs of the users. Rather than put this information online and expect the user to read it in a foreign medium (in respect to our user group), the Quinnipiac Valley Audubon Society has decided that print versions of this material would better serve the needs of our user groups. This function further extends the purpose and mission of QVAS into the local community.
Directly below the Online Library function is a Calendar of Events, which outlines various community events being sponsored or supported by the Quinnipiac Valley Audubon Society. Information is presented on the homepage in an attempt to inform the user about the purpose of Riverbound Farm. Furthermore, the link in the navigational toolbar is located within close proximity to the actual description, thus presenting a positive visual learning field.
The final grouping of links on the navigational toolbar is the Get Involved section. This grouping of links includes links for Volunteer Opportunities, as well as a link for Become a Member of QVAS, which outlines an online application procedure for becoming a member of QVAS, and a link for donating money. The Redeye Rovers felt that since this information was specific to user groups who expressed significant amounts of interest in the purpose and programs of the Quinnipiac Valley Audubon Society, it was best, design-wise, to include that information last. Therefore, the casual user would have the opportunity to inform themselves about the specifics of the organization before being presented with an opportunity to join.