Is It Just a Bad Class? Assessing the Stability of Measured Teacher Performance

November 2008
Dan Goldhaber, Michael Hansen

Download Full Report (PDF: 2642 K)

CRPE Working Paper #2008-5

This paper reports descriptive findings on the stability of measured teacher effectiveness. Using a unique longitudinal dataset from North Carolina that includes the records of virtually all teachers and students in North Carolina, we estimate various measures of teacher effectiveness using different specifications of a value-added education production function model. We then compare these different estimates of teacher effectiveness to one another and use them to assess the extent to which measures of teacher value-added vary over time, and across subjects and teaching contexts. We find average correlations of 0.3 in reading and 0.5 in math in year-to-year estimates of teacher effectiveness, and cross subject correlation that averages near 0.5. Chi-squared tests support the notion of some stability with these effectiveness measures; however, the year-to-year variation is greater than what is predicted were random error the only unstable component, implying that teacher job performance does vary over time.

Not surprisingly, having more information about teachers in the form of larger classes or additional years of matched teacher-student data increases the precision of estimated teacher effects, but various investigations show the change in rankings that result from introducing more student observations within a class are generally small—fewer than 10 percent of teachers move their relative rankings by more than one quintile equivalent in math. Introducing more information from a successive year of teaching, however, has a considerable impact on relative rankings—in this case, close to 50 percent of teachers change relative rankings by more than one quintile equivalent in math.

We conduct additional investigations into the stability of estimated effectiveness focused around other variations within the context of teaching: before and after a obtaining tenure, before and after transferring schools, across various student demographics, and at different points along teachers’ career paths. Generally, we find estimated effectiveness in math is considerably more stable over time than in reading; however, estimates did not support the notion of “stable” performance over time in either subject.