Update on India Merit Pay Study

I recently wrote about a merit pay study in India. One of the authors (Karthik Muralidharan) wrote me an email, wanting to clarify a point:

…as the text of the paper (and table 9) makes clear, there was *NO* difference in test scores between the control and ‘pure control’ schools even though the teachers in the control schools (who were observed more) appeared to be working harder. Also, note that the incentive and control schools had exactly the same level of monitoring and measurement to ensure that these effects (Hawthorne effects) would be netted out while estimating the effectiveness of performance pay.

So the main puzzle is as follows:

Test Score: Incentives > Control = Pure Control
Observed Processes: Incentives = Control > Pure Control

The most logical interpretation of this is that behavior under regular observation converges to some norms of ‘good teaching’ but that this only lasts as long as the observation (which happened 6 times during the year), and so this ‘observed’ improvement in processes has no predictive power on test scores.

Also, good news: Karthik says the study will be updated soon with second year data.