Evaluating results of the implementation research MOOC using Kirkpatrick's four-level model: a cross-sectional mixed-methods study

INTRODUCTION: An implementation research (IR) massive open online course (MOOC) was developed by the Special Programme for Research and Training in Tropical Diseases, to address the scarcity of training in low-income and middle-income countries in the field of IR. The Kirkpatrick model was used to evaluate the IR MOOC as it is widely applied for evaluation of training and educational programmes. The Kirkpatrick model evaluates training programmes on four levels: reaction, learning, behaviour and results. This paper addresses the impact of the IR MOOC on participants' professional practice.

METHODS: Findings are based on analysis of survey and interview data collected 1.5-2 years after the conclusion of the two 2018 IR MOOC offerings. Of the 3858 MOC participants, 748 responded to the anonymous online survey and seven of these respondents were interviewed. All data are self-reported.

RESULTS: The IR MOOC was successful in enhancing the professional practice of participants and for their organisations. Over 40% reported modifying or implementing changes in their professional work. Respondents reported that participation in the MOOC had improved their ability to conduct IR, enhanced their professional profiles and increased their opportunities for collaboration, research and job promotion. Respondents stated that the MOOC had improved their work quality and productivity, and allowed them to contribute to research, initiate and develop professional collaborations and train others in IR. Respondents reported an increase in applying for grants and scholarships and presenting and publishing work on IR after participating in the MOOC. Barriers applying the knowledge gained from the IR MOOC were experienced, for example, due to a lack of funding and lack of support from colleagues, managers and organisations.

CONCLUSION: Participants perceived that the IR MOOC was successful in its aims of delivering medium-term and long-term results in relation to their own and their organisations' professional outcomes.