via ArXiv AI
Measuring Curriculum Alignment across Topical Coverage, Competency, and Cognitive Depth: A Longitudinal Framework Applied to CS2013 and CS2023
# Abstract
Undergraduate computer science programs are guided by international curricular guidelines that are revised approximately once per decade. Despite this periodic update, programs lack a reliable, reproducible method for measuring how comprehensively they cover the current guidelines—and how that coverage changes when the guidelines are restructured. This paper addresses that gap by introducing a human-in-the-loop pipeline that measures a program's coverage of an external body of knowledge. We apply this pipeline longitudinally to one accredited BSc in Computer Science program, evaluating its alignment with both the Computer Science Curricula 2013 (CS2013) and 2023 (CS2023) standards.
The pipeline represents the program and each set of guidelines as structured corpora. Candidate course-to-knowledge-unit matches are generated via semantic retrieval and confirmed through human judgment under an explicit coverage definition. Among seven benchmarked retrievers, a reciprocal-rank-fusion ensemble performed best, while a reputed long-context model underperformed a smaller sentence model—underscoring the importance of careful retriever selection. Both alignment maps were validated by an independent second rater, achieving Cohen's kappa of 0.64 for CS2023 and 0.69 for CS2013.
Results show that the program covers 49.7% of CS2023 knowledge units and 50.9% of CS2013 knowledge units, indicating nearly constant coverage across a decade of guideline evolution. Extending the same retrieve-then-confirm methodology to competency articulation and cognitive depth reveals that the program articulates competency for approximately 88% of covered units under each guideline, yet delivers competency at the recommended depth for only 76% of covered units under CS2023, compared to 95% under CS2013. This gap reflects the higher expectations embedded in the newer guideline rather than a decline in program quality.
The longitudinal comparison distinguishes persistent structural gaps—notably in parallel and distributed computing, foundations of programming languages, and systems fundamentals—which remain uncovered under both CS2013 and CS2023 as well as ABET standards, from differences that purely reflect the evolution of the guidelines themselves. The reusable instrument is available from the authors upon request.