Comparing the agreement among alternative models in evaluating HMO efficiency

Objective. To describe the efficiency of HMOs and to test the robustness of these findings across alternative models of efficiency. This study examine s whether these models, when constructed in parallel to use the same inform ation, provide researchers with the same insights and identify the same tre nds. Data Sources. A data set containing 585 HMOs operating from 1985 through 19 94. Variables include enrollment, utilization, and financial information co mpiled primarily from Health Care Investment Analysts, InterStudy HMO Censu s, and Group Health Association of America. Study Design. We compute three estimates of efficiency for each HMO and com pare the results in terms of individual performance and industry-wide trend s. The estimates are then regressed against measures of case mix, quality, and other factors that may be related to the model estimates. Principal Findings. The three models identify similar trends for the HMO in dustry as a whole; however, they assess the relative technical efficiency o f individual firms differently. Thus, these techniques are limited for eith er benchmarking or setting rates because the firms identified as efficient may be a consequence of model selection rather than actual performance. Conclusions. The estimation technique to evaluate efficient firms can affec t the findings themselves. The implications are relevant not only for HMOs, but for efficiency analyses in general. Concurrence among techniques is no guarantee of accuracy, but it is reassuring; conversely, radically distinc t inferences across models can be a warning to temper research conclusions.