Rapid dynamic assessment of expertise: A comparison of performance and mental efficiency measures in accordance with cognitive load theory.

Abstract

The purpose of this study was to determine whether the use of performance measures for instructional adaptation were more effective and efficient than the use of mental efficiency measures. Fifty-three undergraduate accounting students were randomly assigned to a performance group, a mental efficiency group, and a non-adapted control group. Participants were administered an initial diagnostic test, were placed in a training session about accounting cost-volume-profit analysis, and were administered a final diagnostic test, similar to the initial diagnostic test, and a mental effort rating of the training session.

Performance group participants were placed in the training session and allowed to skip certain training session stages based on the results of rapid verification tests administered during the initial diagnostic test. Mental efficiency group participants were placed in the training session and allowed to skip certain training session stages based on the results of rapid verification tests and mental effort ratings administered during the initial diagnostic test. The non-adapted control group participants were placed in the training session at the beginning and did not skip any stages.

The training session consisted of four difficulty levels, each with five stages. At each stage a faded worked example and a faded completion problem were provided and a rapid verification test and a mental effort rating were administered. Performance group participants advanced to the next stage or repeated the current stage based on the results of the rapid verification test. Mental efficiency group participants advanced to the next stage or repeated the stage based on the results of the rapid verification test and mental effort ratings. The non-adapted control group did not repeat any training session stages.

The study produced no significant differences between any treatment groups for instructional time, final diagnostic test score, mental effort rating of the training session, or instructional efficiency (final diagnostic test score divided by mental effort rating of the training session). The author speculated that the non-significant results of the study were attributable to either an insufficient training session length or to the use of faded completion problems rather than conventional problems.