Reliability assessment of a rod photoreceptor outer segment grading system

The purpose of this study was to assess the reliability of a rod photorecep tor outer segment (PR-OS) grading system based on the analysis of 1 mum thi ck retinal sections obtained from Xenopus laevis whole-eye organ cultures. Digitally captured images, representative of the entire spectrum of rod PR- OS organization levels, were selected and coded numerically. A total of 102 individual rod PR-OS profiles were graded according to a six-step classifi cation scheme based on the percentage of rod PR-OS membrane organization. U nweighted (exact agreement) and weighted kappa (kappa) coefficients (for us e with ordered categorical rating scales) were calculated. Differences betw een kappa coefficients were tested for by chi-square analysis. To investiga te the intra- and inter-rater variability and the possible presence of an i nteraction of the measurements with time, a repeated-measures analysis of v ariance was performed. The overall unweighted. and weighted intra-rater kap pa coefficients were 0.78 and 0.92, respectively. The overall unweighted an d weighted inter-rater kappa coefficients were 0.73 and 0.90. respectively. There was no significant difference between raters or between first and se cond reading, nor was interaction between raters and time of rating documen ted. Individual kappa coefficients were equivalent both between raters and between sessions. Intra- and inter-rater agreement was within one step in 1 00 % of cases. The estimated values of the kappa coefficients are consisten t with a good to excellent degree of reliability and reproducibility of thi s rod PR-OS grading system. This system will be useful in the assessment of rod PR-OS morphology in studies of photoreceptor physiology and pathology. (C) 2001 Academic Press.