An assessment of observer bias in the shunt design trial

Introduction: An international multicenter randomized trial comparing stand ard pressure differential valves, Orbis Sigma valves and PS Medical Delta v alves, for children with newly diagnosed hydrocephalus failed to show a dif ference in the time to first shunt failure (power 80%). In order to avoid o bserver bias, the primary end-point, shunt failure, was defined by detailed clinical and radiological criteria. Surgeons notified the data center when their patient reached endpoint. Their decision was blindly reviewed by a c entralized adjudication process. The current analysis asks: (1)Did the surg eons and the blinded adjudication process agree on the presence of shunt fa ilure? (2) Do the shunt survival curves change when the surgeons' assessmen t of outcome is used? Methods: The two methods of outcome assessment (surgeons' decision and adju dication process) were compared with a kappa statistic for the presence of shunt failure and a log rank statistic for time to shunt failure. Results: The surgeon and the adjudication process agreed on the presence of the primary outcome in 313/344 patients in the trial (kappa = 0.81). The 3 1 cases of disagreement were evenly distributed among the three shunts. The survival curves generated from the surgeons' decision were not significant ly different from those generated by the adjudication process (log rank = 0 .70, p = 0.37) and did not alter the study conclusions. Discussion: Observer bias in the assessment of outcome is always a concern in an unblinded clinical trial such as this one. We did not detect an obser ver bias of sufficient magnitude to alter the study conclusions.