Estimating Measurement Error in University Students’ Work Experience Programme (SWEP) Assessment using Generalizability Theory: Implications for Transdisciplinarity in Engineering Training

Abstract

Test experts and other stakeholders in education are interested in the quality of students’ assessments, while traditional reliability indices are giving way to Generalisability statistics encouraging transdisciplinarity in problem-solving. This study aimed at estimating measurement error in University Students’ Work Experience Programme (SWEP) assessment using generalizability theory. The design adopted for the study was a one-facet nested fixed design with assessors nested within persons. The study’s target population was all the 200 level undergraduate Engineering students registered for 2015/2016 academic sessions in the Faculty of Engineering in a Nigeria University and all the technicians who assessed them, with 591 students purposively selected. Their assessment scores were collated using a Proforma. Data obtained were analyzed using ANOVA statistics. Findings revealed that the assessor effect confounded the person by assessor interaction. The residual contributed more to measurement error in university engineering SWEP scores; the relative and absolute error variances were equal, with a value of 8.02. Therefore, relative and absolute interpretations cannot be distinguished. Based on these findings, educational researchers engaging in Genralizabilty theory analysis should employ the crossed facets G-study designs in assessment designs to fully harness G-theory’s strength to distinguish between the absolute and relative decisions. This conclusion is so as decision-makers may be interested in one or both decisions while interpreting measurement results in defining error and Generalisability coefficients to employ synergetic efforts to ensure quality engineering education through a transdisciplinary approach to their training.