Anticonvulsant teratogenesis 4: Inter-rater agreement in assessing minor physical features related to anticonvulsant therapy

Background: We report on inter-rater agreement in the assessment of newborn infants with respect to a range of minor physical features in a cohort stu dy of the fetal effects of maternal anticonvulsant use during pregnancy. Methods: Infants from three groups (exposed to anticonvulsants, seizure his tory but no medication exposure, and unexposed controls) were examined by b oth a pediatrician/teratologist, who was blinded with respect to the mother 's exposure status, and an unblinded research assistant. Agreement on asses sments for selected anomalies associated with anticonvulsant therapy was me asured by kappa-statistics, as well as by more sensitive log-linear modelin g techniques, which allow examination of possible covariate effects on the strength of agreement. Although the physician and research assistant agreed on a high proportion of cases (80-90%), kappa values were modest (0.2-0.5) , partly because of the low prevalence of the anomalies considered. To expl ore how agreement varies within subgroups, we used recently developed metho ds for studying agreement based on log-linear models. Results: Log-linear modeling indicated that there was substantial variation in pattern of agreement between different individual research assistants b ut that other factors (e.g., exposure category, sex, and birthweight) did n ot appear to be related to agreement. Our results suggest that research ass istants with more experience showed the highest degree of agreement with th e physicians. Conclusions: Our results have implications for both clinical practice and e pidemiologic research and underline the importance of thorough training of staff in the definitions to be used and also the need for multiple independ ent assessments of these subtle anomalies. (C) 2000 Wiley-Liss, Inc.