Evaluating state programmes: "Natural experiments" and Propensity Scores

Evaluations of programmes - for example, labour market interventions such a s employment schemes and training courses - usually involve comparison of t he performance of a treatment group (recipients of the programme) with a co ntrol group (non-recipients) as regards some response (gaining employment, for example). But the ideal of randomisation of individuals to groups is ra rely possible in the social sciences and there may be substantial differenc es between groups in the distributions of individual characteristics that c an affect response. Past practice in economics has been to try to use multi ple regression models to adjust away the differences in observed characteri stics, while also testing for sample selection bias. The Propensity Score a pproach, which is widely applied in epidemiology and related fields, focuse s on the idea that "matching" individuals in the groups should be compared. The appropriate matching measure is usually taken to be the prior probabil ity of programme participation. This paper describes the key ideas of the P ropensity Score method and illustrates its application by reanalysis of som e Irish data on training courses.