The relationship of individual examinee characteristics and acceptability of smart device-based testing to test score in the practice test of the Korea Emergency Medicine Technician Licensing Examination.

PURPOSE: Smart device-based testing (SBT) began to be introduced in the Republic of Korea's high-stakes examination system, starting with the Korea Emergency Medicine Technician Licensing Examination (KEMTLE) in December 2017. In order to minimize how variation in examinees' environment may affect the test score, it aimed to identify any variables related to individual characteristics and acceptability that are related to the examinees' test scores in the SBT practice test.

METHODS: Out of 569 candidate students who were administered the KEMTLE on September 12, 2015, 560 students responded to the survey questionnaire on the acceptability of SBT after the examination. The questionnaire addressed 8 individual characteristics: 2 satisfaction, 9 convenience, and 9 preference items. Comparison analysis between individual variables was performed. Furthermore, generalized linear model (GLM) analysis to find the effect of individual characteristics and acceptability of SBT on test score were conducted.

RESULTS: Among those who preferred SBT compared to paper-and-pencil testing, test scores were higher for male participants (M=4.36, SD=0.72) than for female participants (M=4.21, SD=0.73). According to GLM, no variables evaluated, including gender, experiences of CBT(Computer based test), SBT, and using a tablet PC, were statistically significantly different by total score, scores on multimedia items, or scores on text items.

CONCLUSION: The variables of individual characteristics and acceptability of SBT did not affect the SBT practice test scores of emergency medicine technician students in Korea. Adoption of SBT for the KEMTLE should be possible to execute without interference from the variables examined in this study.