UNITED-STATES-DEPARTMENT-OF-VETERANS-AFFAIRS CHIROPRACTIC SERVICES PILOT PROGRAM-EVALUATION STUDY SDR =86-09 - A CRITIQUE

The following critique of the Chiropractic Services Pilot Program (SDR #86-09) focuses on two major issues: the terms of reference establish ed for the study and the research constraints that arose from either t he terms of reference or their interpretation; the technical design an d execution of the research. The review suggests that the constraints invalidated the study and ensured that no comparisons are possible bet ween chiropractic and medical care for VA patients based on these resu lts. The constraints resulted in the use of a nonexperimental design, distinct samples being chosen and nonequivalent care settings being co mpared. The critique also reveals that in each of the design steps eli gibility criteria, sampling, protocols, data collection, analysis, int erpretations) there were serious methodological flaws. These ensured t hat the two populations being compared (chiropractic patients versus m edical patients) were in fact noncomparable. In terms of the economic cost comparisons, the design guaranteed the comparison was unfair, pit ting a private, fee-for-service chiropractic practice against a not-fo r-profit, managed-care, federally regulated and budgeted institution. Furthermore, the allocation of costs to the two groups was done inaccu rately. The critique concludes that the results are not valid, they ca nnot be used for generalizing, they cannot be used for statistical ana lysis and they should not be used to establish policy. The research de sign and the methodological flaws meant that the objectives of the stu dy could not be met.