Reliability of Telephone Interview for Assessment of Long-Term Stroke Outcomes: Evidence from Interrater Analysis

BACKGROUND: A comprehensive evaluation of interrater reliability is crucial when it comes to multiple coders assessing the stroke outcomes using telephone interview. The reliability between telephone raters is important, as it could affect the accuracy of the findings published.

OBJECTIVE: This study aimed to establish the interrater reliability between multiple telephone interviewers when assessing long-term stroke outcomes.

METHODS: Patients alive at discharge selected in a retrospective cohort stroke project were recruited in this study. Their contact numbers were obtained from the medical record unit. The patients and/or proxies were interviewed based on a standardized script in Malay or English. Stroke outcomes assessed were modified Rankin Scale (mRS) and Barthel Index (BI) at 1-year post discharge. Fully crossed design was applied and 3 assessors collected the data simultaneously. Data was analysed using the software R version 3.4.4.

RESULTS: Out of 207 subjects recruited, 132 stroke survivors at the time of interview were analysed. We found a significant excellent interrater reliability between telephone interviewers assessing BI, with intraclass correlation coefficient at 0.996 (95% CI 0.995-0.997). Whereas substantial agreement between the telephone interviewers was revealed in assessing mRS, with Fleiss', Conger's and Light's Kappa statistics reporting 0.719 and the Nelson's model-based κm kappa statistic reporting 0.689 (95% CI 0.667-0.711).

CONCLUSION: It is reliable to get multiple raters in assessing mRS and BI using the telephone system. It is worthwhile to make use of a telephone interview to update clinicians on their acute clinical management towards long-term stroke prognosis.