Developing a Concept Inventory and Active Learning for Common Computer Security Misconceptions

Abstract

Cybersecurity incidents are on the rise. Tracing these security breaches back, we linked them to people making an error due to a commonsense misconception. There is no one standard tool that gauges a student's understanding of security topics. In this research, we surveyed 75 security experts about security novices' misconceptions, coded the results, and identified 17 top misconceptions. We created open-ended questions and labs/active learning to identify and remediate those misconceptions. After revising the open-ended questions, we gave them to undergraduate students and successfully extracted real-world instances of the misconceptions in practice. We created a ten-question multiple-choice exam by converting the open-ended questions into multiple-choice with many distractors drawn from students' misconceptions. We then conducted "think-aloud interviews" with students to make sure that the questions were clear. After integrating their feedback, we administered multiple-choice exams to two groups of students; 114 CS 1 students with no formal security education and 28 students from a security course. Almost 30% of CS 1 students failed to answer more than one question correctly, and only 3.5% of CS 1 students passed with a score of 60 (a D-). However, only 21.4% of the security students passed, and no individual student got more than seven out of ten correct. Our results show that both groups of students have these common security misconceptions. While security students earned markedly higher scores, our test unequivocally shows that students are leaving the security course retaining significant misconceptions, pointing the way for improvements in teaching.