Formative Evaluation

Evaluation Plan

The Package Acceptance Training at UPS CD-Rom will be reviewed by newly hired UPS employees.

The learners will:

  1. Take a pre-test about the content knowledge contained in the module
  2. Review the module
  3. Take the post-test that is included in the module
  4. Fill out a survey about the module
The formative evaluation will be conducted November 26, 2003 at the corporate office of UPS. Three newly hired Customer Center Associates will review the module.

Evaluation Report

Formative evaluation was conducted on November 26, 2003 at the corporate office of UPS. Three newly hired Customer Center Associates reviewd the module.

Each learner worked individualy through the pre-test, module, post-test and survey, in that order. Evaluation was conducted by comparing the pre-test and post-test scores, totaling the survey scores and the survey comments.

Learner Profile

Learner A
Learner B
Learner C

Female

Male

Female

25 years old

18 years old

42 years old
High school education
High school eductaion
Some Vocational education
Average computer user
High computer user
Lite computer user

Results
Learner A
Learner B
Learner C
Pre-Test
11/20 = 55%
3/20 = 15%
15/20 = 75%
Post-Test
22/24 = 92%
20/24 = 83%
23/24 = 96%
Survey
75/80 = 94%
82/85 = 96%
76/85 = 89%

Success! There was an increase in the scores from the pre-test to the post-test. On average, there was a 42% increase in scores. While this may not be the case if the sample size was large, this is still a godd indication that the module is indeed meeting its goals.

Some of the Survey Comments

Positive Comments

  • Navigation worked flawlessly
  • Interactions were wonderful and tested the content accurately
  • Graphics really add to the learning
  • The certificate is so rewarding

Negative Comments

  • Self-assessment words are too small
  • Some of the graphics take a long time to load
  • Module contains too much information
  • Module is too long

Recommendations for Revisions

Overall, the evaluation was a success. The learners said they enjoyed being a part of the pilot test and hoped that their comments were helpful.

Recommended revisions of the module will be presented to the client and the client will have the final say. There is a high chance that the module will keep its length. The client has already been notified of this issue, but wishes to keep the content as is. However, all of the technical issues will be addressed and fixed before the next prototype is released.

Abstract | 15/5's | Boards | Goals & Objectives | Content Analysis | Lesson Summary

Flowchart | Prototypes | Curriculum Integration | Rationale | Cost Estimate | Evaluation

Personal Home Page | 6200 Management Site | Project Documentation | Course Requirements