Measuring the Effectiveness of Cybersecurity Awareness Campaigns at a University

Problem Description

Organizations regularly invest in cybersecurity awareness campaigns yet rarely know whether they work. This problem is particularly acute in university settings, where a heterogeneous population of students and employees exhibits highly varied security behaviors and where awareness initiatives are difficult to evaluate rigorously. At the Carl von Ossietzky University of Oldenburg, a structured experimental approach is being developed to address this gap: rather than relying solely on self-reported attitudes, the study aims to use observable behavioral outcomes as measures of campaign effectiveness across treatment and control groups. While the IS literature has established strong theoretical foundations for understanding security behavior (Siponen & Vance, 2010; Johnston et al., 2015; D'Arcy et al., 2009), it has rarely connected these insights to rigorously evaluated field experiments in institutional settings. This thesis contributes to closing that gap.

 

Goal of Thesis

The primary goal of this thesis is to support the design, implementation, and evaluation of a cybersecurity awareness field experiment at the University of Oldenburg, targeting both students and employees. You will play an active role in shaping the study design. Concretely, the thesis will:

  1. Review theoretical foundations from the IS security literature on security policy compliance, protection motivation, and the effectiveness of awareness interventions, drawing on relevant IS research (https://aisnet.org/page/SeniorScholarBasket).
  2. Support the selection of awareness interventions from a set of candidates (including leadership endorsement, best practice communication, podcasts and videos, webinars, gamification, and posters) by assessing which are most likely to produce measurable behavioral change based on theory and prior evidence.
  3. Support the selection and technical setup of behavioral outcome measures from a set of candidates (including phishing experiment results, password change rates, VPN usage patterns, firewall blocks, SentinelOne software data, and eduroam device logs) in coordination with the university's IT department.
  4. Analyze outcome data across treatment and control groups and critically reflect on the methodological challenges of field experimentation in cybersecurity awareness research.

Requirements

  • Solid understanding of or at least interest in quantitative research methods and experimental design.
  • Willingness to familiarize with IS security behavior literature.
  • Strong organizational and communication skills for coordinating across university IT, administration, and research teams.
  • Proficiency in English; German language skills advantageous given the institutional context.

References

D'Arcy, J., Hovav, A., & Galletta, D. (2009). User awareness of security countermeasures and its impact on information systems misuse: A deterrence approach. Information Systems Research, 20(1), 79–98.

Johnston, A. C., Warkentin, M., & Siponen, M. (2015). An enhanced fear appeal rhetorical framework: Leveraging threats to the human asset through sanctioning rhetoric. MIS Quarterly, 39(1), 113–134.

Siponen, M., & Vance, A. (2010). Neutralization: New insights into the problem of employee information systems security policy violations. MIS Quarterly, 34(3), 487–502.

Siponen, M. T. (2000). A conceptual foundation for organizational information security awareness. Information Management & Computer Security, 8(1), 31–41.

Vance, A., Siponen, M., & Pahnila, S. (2012). Motivating IS security compliance: Insights from habit and protection motivation theory. Information & Management, 49(3–4), 190–198.