Program evaluations should be planned and conducted by-

• individuals with expertise and training in mathematics education;
• individuals with expertise and training in program evaluation;
• individuals who make decisions about the mathematics program;
• users of the information from the evaluation.

Focus

An evaluation of a mathematics program should be planned and directed by a team whose members are competent, enthusiastic, and committed. This team should be composed of those who have the greatest interest in the results of the evaluation--the stakeholders. These include the program decision makers and the users of the information that will be produced. In a mathematics program, these participants are teachers, mathematics supervisors, school and district administrators, school board members, and members of the community. Here teachers are considered both decision makers and users. Teachers can and do make critical curriculum decisions and should participate in any program evaluation. In addition, mathematics educators should be involved in interpreting the spirit and the letter of the Standards. The team should also include a program evaluator with some knowledge of evaluation procedures that coincide with Program Evaluation Standards 11-13.

Such an evaluation team includes people who have the greatest need for information about the program and who care most about what happens to it. An evaluation conducted by such a team has at least one key advantage over any conducted by a single evaluator or small group: A panel of participants with vested interest in the program is more apt to generate the important questions to be addressed in the evaluation, thereby increasing the likelihood that the effort will produce useful and meaningful information.

Discussion

An evaluation often is threatening, especially when it is conducted by "outside" experts. The results of such an evaluation might be regarded with suspicion and therefore might be of limited use. When individuals involved in a program also participate in its evaluation, they have an enhanced sense of "ownership." They tend to view the evaluation conclusions as more valid and more useful in decision making. Some observers might question the credibility of evaluation findings from efforts directed largely by insiders. School districts must be sensitive to this issue and consider it in light of local conditions and circumstances. An outsider's view might be warranted in some situations, or someone with more objectivity or different ideas might be needed. However, the team approach to evaluation places a higher value on producing a useful and valid evaluation than on meeting all the requirements of a less subjective form.

The program evaluator and the mathematics educator are responsible for bringing their expertise to the evaluation. A program evaluator should know how to phrase questions and develop procedures for collecting information. A mathematics educator will bring a knowledge of mathematics and instruction to the process. Such knowledge is helpful in identifying how the full meaning of the Standards relates to the existing program. Some districts might need to bring in outside experts if they cannot be identified from within.

Because of the primary role of teachers in achieving the vision of the Standards, their participation is crucial to an effective and useful evaluation. Since the implementation of the program can vary by grade, it is important that the evaluation team include teachers from different grade levels. At a minimum, representatives from each of the grade-level ranges established in the Standards should be included; this requirement applies to teams evaluating programs in elementary, middle, or secondary schools (i.e., for districts that are K-4, 5-8, or 9-12), as well as for K-12 programs. For example, the evaluation team for the mathematics program of an elementary school district should include a teacher from the neighboring middle school district to ensure articulation across programs.

School and district administrators who help create and implement the curriculum are particularly important members of any evaluation team. In addition to their knowledge of the curriculum, they, along with teachers, can provide insights into methods of implementation and related problems, as well as up-to-date information about available support.

In addition, the team may include other individuals who are interested or involved in making decisions regarding the outcomes of the mathematics program, such as parents, school board members, and representatives of the business and work communities. Members from institutions or organizations other than schools are useful because they have different viewpoints and can act as interpreters between the school and the community.

The membership and function of an evaluation team can vary according to existing conditions, availability of local expertise, and resources. In all situations, however, it is crucial that a team of interested people participate in the evaluation process rather than leave the task to one or two individuals. Every team member need not be involved in every part of the evaluation; a team may at times function as a body of subgroups, each assigned a particular function that best taps its members' expertise.

As described in Evaluation Standard 13, classroom observations are essential to determine how actual instruction compares with the Standards' vision. An adequate evaluation of instruction requires the participation of someone knowledgeable about the mathematics being taught and the goals and approaches envisioned by the Standards. If students are using manipulatives, the observer should know what mathematics is being modeled, whether the manipulatives are appropriate, and whether they are being used correctly.

A thorough evaluation of a mathematics program is a major task. It requires considerable time for both planning and implementation. A district must provide members of the evaluation team with the resources and time necessary to assess with care all aspects of the program: the curriculum, the implemented curriculum, and student outcomes.