4. Methods

4.2 Summary of Methods

The summary of evaluation methods listed in Table 1 details each of the measures used in the study. These measures have been used as consistently as possible to avoid over-complexity and to ensure that a coherent and well designed set of tools was developed. TABLE 1: SUMMARY OF EVALUATION METHODS

EVALUATION MEASURES CHARACTERISTICS EVALUATION GOALS Pre-test content questions Questions based on the learning objectives for the CAL package. Devised by teaching staff at participating sites and administered immediately beforehand. To determine the pre-knowledge of candidates in subject area Post test content questions Set of comparable questions to pre-test, again linked to learning objectives and set by teachers. Administered either immediately after the teaching episode or after some delay. To determine the learning gains made in subject area whilst using the package. Pre-test questionnaire Covers previous experience and confidence. Completed at the same time as pre-test content questions. To assess the readiness of students in approaching the learning task.   Post test questionnaire Covers parts of package completed, help needed, areas found to be easy or difficult and subject confidence after studying the package. To assess student views on the package and its objectives and their own achievements. Observation of student learning Analysis of actions and interactions of students in the classroom setting, using a critical incidents pro-forma. Unusual or unexpected outcomes also documented. To observe the context in which the learning is taking place and document any significant or unexpected problems, issues or program features. Discussion groups Semi-structured, taped interviews with groups of eight to ten students, lasting approximately twenty minutes. At least one group discussion at each participating site. To explore some issues in more depth, to verify impressions gained through observations and to allow the students the opportunity to speak for themselves.