On Friday, 26th February 2016, Qpercom participated successfully in the National University of Ireland, Galway (NUIG) School of Medicine’s multi-site Year 4 Objective Structured Clinical Examination (OSCE). New from the previous year’s OSCE was the opening of the Clinical Skills Suites at Sligo General Hospital Medical Academy.
Together with the St Angelas NUIG Academy, two beautiful sites were participating in Sligo along with the old reliable four Galway OSCE sites. These included two HSE Ireland sites within the Obstetrics and Pediatrics departments of University College Hospital Galway (UCHG), and two NUIG sites at the Clinical Science Institute and the Comerford Education Centre. All in all, a total of 60 stations were rolled out across the six sites in Galway and Sligo, in this large scale OSCE assessing 189 medical student’s clinical competencies.
To avoid students briefing the content of individual stations to their fellow examinees, some of the most critical stations used the ‘multi-scenario’ feature in Qpercom’s Observe OSCE management information system, with adjusted scenarios and likewise, adjusted forms. Furthermore, due to the functionality of using the ‘multi-examiners’ feature in Observe, examiners could be easily swapped during the course of the examination. Even inexperienced examiners or Simulated Patients (SP) could join the examination process while being supervised by more experienced and higher rated examiners. In total, 80 examiners participated during this multi-site OSCE.
In data-control centres in both Galway and Sligo, team members of Qpercom kept an eye on the incoming streams of live data with CEO of Qpercom, Dr Thomas Kropmans. This data could be retrieved, stored and analysed in real time offering a basic quality review of raw data entry, examiners analysis and summary psychometric reports comparing online results from each of the six participating sites. Assessment and quality assurance data became available for module coordinators and program directors while the exam was still ongoing. Compared to the usual paper-based approach, this is a win-win scenario ensuring all exam data is successfully captured during the exam.
OSCEs are very labour intensive during preparation, examination and results analysis phases. However, due to the implementation of Qpercom Observe at the School of Medicine in NUIG in 2008, 70% of the administrative costs were saved due to the OSCE Planning and Management software. Where lower technology approaches with paper forms contain up to 30% errors, the Observe system is error free and data cannot be lost or misplaced. Furthermore, since the implementation of Observe, Generalizability Coefficients (reliability and validity measures) have improved over time. Average results and their variations have decreased to more normal dispersion due to improved standard setting and quality assurance procedures within Observe.
Students also benefit from the implementation of all functionalities within Observe, such as the instant Student Feedback System. This system offers the opportunity to inform students about their performance in an individualized PDF report highlighting those aspects that went well, and those competencies that need further improvement. Feedback is the most powerful learning tool, and suggestions for improvement are displayed via personalised radar plots explaining student’s results.