Indicators of a mathematics program's consistency with the Standards should include--

• student outcomes;
• program expectations and support;
• equity for all students;
• curriculum review and change.

In addition, indicators of the program's match to the Standards should be collected in the areas of curriculum, instructional resources, and forms of instruction. These are discussed in Evaluation Standards 12 and 13.

Focus

The central theme of the Standards is that knowing mathematics is doing mathematics. Through solving problems, reasoning, communicating, investigating, and exploring, students will know mathematics. Inherent in this theme is that any evaluation of a program's match to the Standards must consider what mathematics students know; how they learn mathematics; and the curriculum, means of instruction, and expectations of those who influence the program. In addition, an evaluation should consider any barriers that prevent students from attaining the full benefits of the program and what can be done to eliminate such barriers. Finally, evaluation should consider the dynamic nature of the program, including its process for self-monitoring and for making necessary adjustments and changes.

Discussion

A primary purpose of any evaluation, using the Standards as a criterion, is to obtain information and suggestions about how a given program can more fully incorporate the spirit--as well as the letter--of the Standards. An evaluation should provide evidence that a program does or does not match the Standards. In addition, an evaluation should gather information on how a program can be changed to better meet the Standards. Thus, an evaluation should collect information on a range of indicators that signify a match between the mathematics program and the Standards and that help explain what needs to be done differently.

Student Outcomes. Students in a mathematics program that is consistent with the Standards will become knowledgeable about a variety of topics and the relationships among them and will develop a positive disposition toward mathematics. The previous Evaluation Standards have described the means and possible indicators for assessing students' knowledge and disposition. This discussion focuses on the assessment of mathematics programs as a whole.

A valid evaluation of a program's alignment or match with the Standards depends on a very comprehensive assessment of students' mathematical knowledge. A great many topics and processes should be evaluated using a variety of methods, such as observations of students doing mathematics, performance and oral tasks, and written tests. To achieve adequate coverage, for example, an assessment of student outcomes for one grade, such as the eighth grade, might require collecting information on at least seven indicators, one for each of the student-assessment standards. Such an assessment would include a number of tasks, reaching a possible total of 300 tasks or test questions. Obviously, even a comprehensive assessment cannot require each student to answer all questions, but this is not necessary for a valid or purposeful program evaluation. A procedure such as multiple-matrix sampling should be used to reduce substantially both the number of students and the amount of time any single student would need to be involved. The use of such a procedure addresses the issue of too much testing being done in schools.

Indicators of student outcomes should be aligned with the Curriculum Standards. Norm-referenced tests, for example, reflect too limited a range of content to measure adequately the Standards' expectations of what students should know. As such, norm-referenced test scores, as the sole indicator of student outcomes, are inappropriate for comparing a mathematics program with the Standards.

The evaluation of student outcomes of a program should give some attention to long-term effects. These can be considered in longitudinal studies of the effects of the program on the lives of its graduates one, two, or five years after they have graduated. Such studies should try to establish the effects of the program in helping the graduates reach their goals and meet the challenges they face after finishing the program.

Program Expectations and Support. An adequate support system is a prerequisite to bringing a mathematics program into alignment with the Standards. Such a system is based on the view that a variety of instructional activities are needed and that all students must know the mathematics described in the Standards. If such a view is prevalent throughout the school district and shared by most decision makers--including mathematics supervisors, district administrators, and school board members--the likelihood of a program coinciding with the Standards is increased. Indications of a strong support system are the existence of staff-development programs and the provisions made to allow students sufficient time to take an inquiry approach in learning mathematics. Evaluation should offer some evidence on whether the necessary expectations and support exist.

Program evaluation also should consider the degree to which parents are interested, knowledgeable, and involved in the mathematics education of their children. Do parents take an interest in their children's progress in learning mathematics? Do parents monitor the time students spend on doing mathematics homework compared with other activities, such as watching television? What ways can parental involvement be increased in light of the Standards' view of mathematics?

Equal Access. A critical component of any mathematics program, particularly one that strives to fulfill the spirit of the Standards, is the equal access that all students have to take advantage of the full benefits of the program. The official position of the NCTM is that "all students, regardless of their language or cultural background, must have access to the full range of mathematics courses offered. Their patterns of enrollment and achievement should not differ substantially from those of the total student population" (NCTM 1987). Program evaluations should include indicators that the mathematics program is meeting these essential criteria. Enrollment figures by gender, race, language, and cultural background should be maintained for all mathematics courses. As unacceptable patterns emerge, an evaluation should identify the barriers creating the situation and recommend action. All students should have equal access to the full range of mathematics courses, and this should be continually monitored and included as a part of the ongoing program review.

Curriculum Review and Change. Just as the world is changing rapidly and new technologies are developing continually, so must mathematics programs evolve and grow. The vision articulated in the Standards is that all children will develop the mathematics they need to function and succeed in a world in which the ability to think about and solve problems has become increasingly important. It is obvious, however, that the Standards cannot foresee all the changes in calculators, computers, and software or other advances that might affect what mathematics students will need to know in the future. One function of evaluation is to determine whether a given program has established a self-monitoring process by which it keeps current with the dynamic nature of mathematics.