Teacher effects and teacher effectiveness:
A validity investigation of the
Tennessee Value Added Assessment System

H. Kupermintz (Fall, 2003). Educational Evaluation and Policy Analysis, No. 3, pp. 287-298.

Summary by Denise McKeon, NEA Senior Policy Analyst

Student test score gains have recently been proposed as a measure of the educational "value-added" contribution by teachers and schools to student learning. Recent educational reform efforts (such as "No Child Left Behind") seek to employ standardized test score gains as a key policy instrument for holding educators and school systems accountable.

Currently the most influential value-added model is the Tennessee Value Added Assessment System (TVAAS) -- twenty-one states including Colorado, Ohio and Pennsylvania are experimenting with or using the TVAAS, which analyzes student test score data and estimates the effects of individual teachers on score gains. These effects are used to construct teacher value-added measures of teaching effectiveness.

What The Study Examined

This article examines the TVAAS definition of teacher effectiveness, the mechanism employed in calculating numerical estimates of teacher effectiveness, and the relationships between these estimates and student ability and socioeconomic background characteristics. Kupermintz shows that there are several logical and empirical weaknesses of the system and he calls for a strong research program to establish its validity.

What the Study Found

First, Kupermintz questions the notion that there are solely two distinct variables – teacher effectiveness and differences in student learning – and that the former causes the latter. He suggests that between-teacher variability on the average test score gains of their students may arise for different reasons – some of which are associated with teacher effectiveness, but others of which are not.

The other reasons may reflect the context in which learning occurs or the qualities of the specific group of students being taught. Kupermintz reanalyzes data that show a strong association between teacher effects and prior student achievement, suggesting that difficulties arise when trying to disentangle responsibility for observed gains (were those gains due to students’ prior experience or to teacher effectiveness?).

Additional Reasons for Concern

Kupermintz points to the relatively weak program of research on the TVAAS, citing the fact that TVAAS findings on teacher effects have been discussed in only three peer-reviewed journals, two book chapters and three unpublished research reports (all of which are authored by TVAAS staff). He goes on to state that only one journal article and two unpublished research reports actually present findings from original empirical studies – none of which used the full TVAAS model in its analyses. Kupermintz adds that although repeated requests for access to the TVAAS data have been made by several researchers, they have been met with refusals or have been stalled, so the normal scientific process of verifying scientific data and others’ findings has been thwarted.

Related research:

See Teacher Effects As a Measure of Teacher Effectiveness: Construct Validity Considerations in the TVAAS (Tennessee Value-Added Assessment System) by Haggai Kupermintz, CSE Technical Report 563 (2002)