Id. Coulter, UNITED-STATES-DEPARTMENT-OF-VETERANS-AFFAIRS CHIROPRACTIC SERVICES PILOT PROGRAM-EVALUATION STUDY SDR =86-09 - A CRITIQUE, Journal of manipulative and physiological therapeutics, 16(6), 1993, pp. 375-383
The following critique of the Chiropractic Services Pilot Program (SDR
#86-09) focuses on two major issues: the terms of reference establish
ed for the study and the research constraints that arose from either t
he terms of reference or their interpretation; the technical design an
d execution of the research. The review suggests that the constraints
invalidated the study and ensured that no comparisons are possible bet
ween chiropractic and medical care for VA patients based on these resu
lts. The constraints resulted in the use of a nonexperimental design,
distinct samples being chosen and nonequivalent care settings being co
mpared. The critique also reveals that in each of the design steps eli
gibility criteria, sampling, protocols, data collection, analysis, int
erpretations) there were serious methodological flaws. These ensured t
hat the two populations being compared (chiropractic patients versus m
edical patients) were in fact noncomparable. In terms of the economic
cost comparisons, the design guaranteed the comparison was unfair, pit
ting a private, fee-for-service chiropractic practice against a not-fo
r-profit, managed-care, federally regulated and budgeted institution.
Furthermore, the allocation of costs to the two groups was done inaccu
rately. The critique concludes that the results are not valid, they ca
nnot be used for generalizing, they cannot be used for statistical ana
lysis and they should not be used to establish policy. The research de
sign and the methodological flaws meant that the objectives of the stu
dy could not be met.