Problems with the interpretation of pharmacoeconomic analyses - A review of submissions to the Australian pharmaceutical benefits scheme

Context Pharmacoeconomic analyses are being used increasingly as the basis for reimbursement of the costs of new drugs. Reports of these analyses are often published in peer-reviewed journals, However, the analyses are comple x and difficult to evaluate. Objective To describe the nature of problems encountered in the evaluation anti interpretation of pharmacoeconomic analyses used as a basis for reimbu rsement decisions, Data Sources All major submissions to the Department of Health and Aged Car e (DHAC) by the pharmaceutical industry for funding made under the Australi an Pharmaceutical Benefits Scheme. Specifically, the DHAC's database of sub missions that were received between January 1994 and December 1997 were rev iewed. Study Selection Of a total of 326 submissions, 218 had serious problems of interpretation and were included in the analysis, The nature of the serious problems reviewed were classified as estimates of comparative clinical eff icacy, comparator issues, modeling issues, and calculation errors. Data Extraction Ail submissions in the DHAC's database were reviewed and da ta were extracted if both the DHAC evaluators and technical subcommittee co nsidered problems to have a significant bearing on the decisions of the par ent committee, Data Synthesis Of a total of 326 submissions, 218 (67%) had significant pro blems and 31 had more than 1 problem. Of the 249 problems identified, 154 ( 62%) related to uncertainty in the estimates of comparative clinical effica cy, and 71 (28.5%) related to modeling issues, which included clinical assu mptions or cost estimates, used in the construction of the economic models. There were 15 instances of disagreement over the choice of comparator, and serious calculation errors were found on 9 occasions. Overall, 159 problem s (64%) were considered to be avoidable. Conclusions Significant problems were identified in these pharmacoeconomic analyses. The intensive evaluation process used in the Australian Pharmaceu tical Benefits Scheme allowed for identification and correction of pharmaco economic analysis problems, but the resources that are required may be beyo nd the capacity of many organizations, including peer-reviewed journals.