Assessment analytics: The missing step

Abstract

Assessment is a central element of the learning process; it defines what students consider important and measures their personal achievements across all phases of professional development. It gives institutions a valuable feedback about the success of realizing program goals and institution objectives.1 E-assessments are special types of assessments that use technology for management or delivery. Although the field is relatively new, the number of e-assessments delivered online or supported by technology continues rapidly to grow in size, number, and diversity in both formative and summative domains.2,3 E-assessment can be particularly helpful in improving feedback given to students, enhancing self-regulation skills, promote higher achievement through formative assessment and improve retention rates, it is also noted for increasing student engagement, and motivation.1,2 In contrast to traditional methods of assessment that leave little trail behind; e-assessments have the potential to generate large amounts of data about the students and assessments alike, coupled with increased computation power, a rapidly improving data science and big data research methods, it serves as an ideal platform for assessment and learning analytics (LA).4,5 LA is a rapidly developing research field in the field of technology enhanced learning that aims at “measurement, collection, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimizing, learning and the environment in which it occurs.â€5 Using analytics in medical education can improve decision-making based on actual data that are derived from studying students’ online behavior, it can foster institutional growth, increase productivity, create innovative models and enable institutions to understand their strengths and challenges.4,6 The main applications for analytics so far have been the creation of early alert systems of underachieving, identification of at-risk students, student profiling, estimation of the impact of interventions and adaptive learning recommendations.7 Although the potential of assessment data is huge, it is still underexplored and largely under-developed, due to the fact that researchers treat assessment as an outcome they work to optimize, rather than an integral part of the analytics data cycle. Harnessing the power of e-assessment analytics can effectively close the LA data loop and add to the yield and usefulness of LA.8 With the advance of research in both disciplines; boundaries between learning and assessment would fade, since LA are used to assess learning and assessment analytics are used to understand learning, enabling better adaptive assessments and personalized learning strategies.3

Assessment analytics benefits would extend to all stakeholders involved in the educational process. At the institutional level, it will help assess programs by comparing cohorts of students, offer inter- and intra-school benchmarking, nationwide inter-institutional benchmarking or even at the international levels. It will also shed lights on efficiency of teaching methods, evaluate curricular developments and quality assurance. At the teacher level, it can inform curriculum design, teaching and learning practices, assess how far intended learning outcomes of a course were achieved, identify problematic teachers, deficient item authors and spot areas in need of improvement. At the student level, it will provide detailed reports about their learning progress, enable them to measure attainment of knowledge and skills over time (ipsative progress), compare it to where they started, benchmark performance against other learners or cohorts. Moreover, improvements in programs, curricula, teaching methods or assessment would always be positively reflected on students.3,9

Medical institutions should work to harness the power of data that e-assessments generate, to better understand how they perform and what it takes to be better.