Enhancing Cybersecurity Curriculum Development: AI-Driven Mapping and Optimization Techniques

Enhancing Cybersecurity Curriculum Development: AI-Driven Mapping and Optimization Techniques Petr Dzurenda [email protected] Brno University of Technology Brno, Czech Republic Sara Ricci [email protected] Brno University of Technology Brno, Czech Republic Marek Sikora [email protected] Brno University of Technology Brno, Czech Republic Michal Stejskal [email protected] Brno University of Technology Brno, Czech Republic Imre Lendak [email protected] University of Novi Sad Novi Sad, Serbia Pedro Adão [email protected] Instituto Superior Técnico, ULisboa, and Instituto de Telecomunicações Lisboa, Portugal ABSTRACT Cybersecurity has become important, especially during the last decade. The significant growth of information technologies, internet of things, and digitalization in general, increased the interest in cybersecurity professionals significantly. While the demand for cybersecurity professionals is high, there is a significant shortage of these professionals due to the very diverse landscape of knowledge and the complex curriculum accreditation process. In this article, we introduce a novel AI-driven mapping and optimization solution enabling cybersecurity curriculum development. Our solution leverages machine learning and integer linear programming optimization, offering an automated, intuitive, and user-friendly approach. It is designed to align with the European Cybersecurity Skills Framework (ECSF) released by the European Union Agency for Cybersecurity (ENISA) in 2022. Notably, our innovative mapping methodology enables the seamless adaptation of ECSF to existing curricula and addresses evolving industry needs and trend. We conduct a case study using the university curriculum from Brno University of Technology in the Czech Republic to showcase the efficacy of our approach. The results demonstrate the extent of curriculum coverage according to ECSF profiles and the optimization progress achieved through our methodology. CCS CONCEPTS •Security and privacy → Human and societal aspects of security and privacy; •Human-centered computing → Human computer interaction (HCI);•Social and professional topics → Computing education;•Applied computing →Education. KEYWORDS Curricula Design, ECSF framework, Methodology, Cybersecurity Education This work is licensed under a Creative Commons Attribution International 4.0 License. ARES 2024, July 30–August 02, 2024, Vienna, Austria ©2024 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-1718-5/24/07 https://doi.org/10.1145/3664476.3670467 ACM Reference Format: Petr Dzurenda, Sara Ricci, Marek Sikora, Michal Stejskal, Imre Lendak, and Pedro Adão. 2024. Enhancing Cybersecurity Curriculum Development: AI-Driven Mapping and Optimization Techniques. In The 19th International Conference on Availability, Reliability and Security (ARES 2024), July 30– August 02, 2024, Vienna, Austria. ACM, New York, NY, USA, 10 pages. https: //doi.org/10.1145/3664476.3670467 1 INTRODUCTION Cybersecurity breaches pose a significant threat to governments, businesses, and individuals worldwide, particularly as digital technologies become increasingly integral to economies. The COVID-19 pandemic further accelerated society’s reliance on digital platforms, with remote work becoming a fundamental for operational continuity during lockdowns. However, this shift exposed individuals and organizations to unprecedented cybersecurity risks (World Economic Forum, 2022 [ 15 ]). Despite the cybersecurity workforce reaching a record high of 5.5 million professionals, a global shortage of 3.9 million workers persists, with Europe facing a deficit of over 347,000 cybersecurity professionals (International Information System Security Certification Consortium (ISC2), 2023 [ 9 ]). Moreover, certain sectors, like governments and central banks, struggle to attract highly skilled professionals compared to industries such as finance (European Union Agency for Cybersecurity (ENISA), 2021 [ 11 ]). ENISA warned of a broader shortage of cybersecurity skills in the labor market, emphasizing the increasing demand for professionals with updated knowledge across various cybersecurity domains, including legal and policy frameworks (ENISA, 2021 [ 11 ]; ENISA, 2023 [ 5 ]). Effective cybersecurity training on all levels is essential to address both professional shortages and the general population’s limited cybersecurity knowledge. 1.1 Contributions and Paper Organization This paper contributes to advancing cybersecurity education by offering a systematic and data-driven approach to curriculum development that addresses evolving industry needs and trends. The main objective of this article is to enhance the development and availability of cybersecurity courses by unifying and synchronizing curriculum and training programs. This aims to provide the workforce with reliable and standardized options for upskilling and ARES 2024, July 30–August 02, 2024, Vienna, Austria Petr Dzurenda, et al. reskilling. To achieve this goal, we address the following Research Questions (RQ): (RQ1) How can ECSF profiles be made comprehensible to end users, and what practical methods exist for mapping academic courses to these profiles? To address RQ1, we proposed a novel grouping methodology for ECSF key skills and knowledge. This grouping establishes a connection between NICE competencies (from which they are derived) and the ECSF framework, thereby assigning competencies to profiles and enhancing the readability and usability of the framework. (RQ2) How can a user-friendly AI-driven cybersecurity curriculum development tool be created, and which computational intelligence tools and method are suitable for this purpose? To reflect RQ2, we proposed a novel cybersecurity curriculum development methodology based on machine learning and integer linear programming optimization mechanisms. (RQ3) How can such a tool be used by end users and what are its limitations? To address RQ3, we provide implementation details for our CSProfiler online web tool, which incorporates our AI-driven cybersecurity curriculum development solution. Additionally, we conduct a case study using the existing university curriculum from Brno University of Technology in the Czech Republic to demonstrate the effectiveness of our approach. The rest of the article is organized as follows. Section 2 reviews the key components of our AI-driven cybersecurity curriculum development solution. Section 3 presents our curriculum development methodology, addressing (RQ1). Section 4 provides the details about our CyberSecurity Profiler tool which implements the AIand ILP-based solution and discusses (RQ2). Section 5 reports our experimental results. Section 6 sums up some open problems and potential extensions of our tool. Both sections reflect (RQ3). The final section contains the conclusions. 2 KEY COMPONENTS OF CYBERSECURITY SKILL DEVELOPMENT In this section, we present key components of our Artificial Intelligence (AI)-driven cybersecurity curriculum development solution. Namely, we present the National Institute of Standards and Technology (NIST) Workforce Framework for Cybersecurity (NICE framework) and the European Cybersecurity Skills Framework (ECSF) as the basic building blocks of our solution. We introduce the REWIRE skills grouping enabling practical mapping of ECSF to actual curricula, professional training courses, and job market requirements. Then, we describe AI and Integer Linear Programming (ILP) methodologies that we use to process and evaluate cybersecurity curricula in our solution. The last section describes the Curricula Designer web application as our solution builds on this application. 2.1 National Initiative for Cybersecurity Education (NICE) framework The National Initiative for Cybersecurity Education (NICE) in the United States, led by the National Institute of Standards and Technology (NIST), serves as a collaborative platform involving government, academia, and the private sector. NICE facilitates cybersecurity training and education by developing standards and best practices. The NICE Framework [ 14 ] delineates 1) work roles, 2) Knowledge, Skills, and Abilities (KSAs), and 3) tasks required for various cybersecurity roles. The NICE Framework provides a comprehensive structure for understanding cybersecurity work roles, knowledge, skills, and tasks. With 7 categories and 33 Specialty Areas, the framework delineates the diverse aspects of cybersecurity work. The defined 52 Work Roles are the most detailed groupings of cybersecurity and related work, which include KSAs and tasks performed in each role. In addition, a Competencies classification is derived from the NICE Cybersecurity Skills Framework [ 19 ]. These competencies are of particular interest for our analyses since they allow "education and training providers to be responsive to employer or sector needs" as mentioned in [19]. 2.2 European Cybersecurity Skills Framework The European Cybersecurity Skills Framework (ECSF) [ 4 ] is the result of a collaborative effort between ENISA and the ENISA ad-hoc working group on cybersecurity skills framework. The ECSF is designed to establish a standardized understanding of roles, competencies, skills, and knowledge in the field of cybersecurity, facilitating skills recognition and aiding in the development of cybersecurity training programs. This framework categorizes cybersecurity roles into 12 profiles, each delineating key tasks, skills, knowledge, and competencies. Notably, the ECSF encompasses a total of 84 key skills and 69 key knowledge areas, providing a comprehensive taxonomy for academia and guiding the content of cybersecurity-related education. The 12 ECSF profiles are: Chief Information Security Officer (CISO), Cyber Incident Responder, Cyber Legal, Policy & Compliance Officer, Cyber Threat Intelligence Specialist, Cybersecurity Architect, Cybersecurity Auditor, Cybersecurity Educator, Cybersecurity Implementer, Cybersecurity Researcher, Cybersecurity Risk Manager, Digital Forensics Investigator, and Penetration Tester. 2.3 REWIRE Mission and Skills Grouping The REWIRE project [ 16 ] is a European initiative that aims to address the shortage of cybersecurity experts by improving the availability, accessibility, and quality of cybersecurity courses and certifications. To achieve this goal, the project incorporates the ECSF framework as a key reference point in its actions. Upon analyzing the ENISA framework, it becomes apparent that the key skills and knowledge outlined to describe the profiles are expressed in unique ways, posing challenges in depicting interconnections among the profiles. To address this issue, a method was developed to group skills and knowledge representing similar concepts but expressed differently. This approach enables the clustering of skills and knowledge based on their descriptions, resulting in the identification of 31 distinct groups. This grouping not only simplifies the readability and usability of the profiles but also enhances their Cybersecurity Curriculum Development ARES 2024, July 30–August 02, 2024, Vienna, Austria comprehensibility, particularly for individuals lacking technical knowledge in the field. We refer to Section 3.2 for more details. 2.4 Artificial Intelligence Machine Learning (ML) is increasingly used for tasks like image analysis and time-series forecasting [ 1 , 2 ]. Recurrent Neural Networks (RNN) are popular for tasks such as time series analysis [ 18 ] and Natural Language Processing (NLP) [ 20 ]. RNNs compress input data into a context vector. They exhibit strong performance in sentiment analysis, accurately categorizing sentence sentiment as positive or negative. This methodology can also be effectively applied to classify required skills in course descriptions. Our model uses a pre-built word-piece tokeniser for Bidirectional Transformer (BERT) [ 1 ]. With the tokens from the word-piece tokeniser, a model is used to concatenate the mean of all intermediate context vectors and the last context vector. The concatenated vector is fed to an Artificial Neural Network (ANN) to classify security skills. The hidden unit of Long Short-Term Memory (LSTM) is 128, and the hidden layer of ANN is 256. The model training involved testing its ability to predict required security skills using a labeled dataset, with the Long Short-Term Memory (LSTM) model trained solely on the training dataset to prevent bias. The ML learning approach offers several advantages. It reduces the labor and time required for textual analysis, allows for nuanced analysis of sentiments, and increases flexibility by enabling easy addition of new data to the dataset. Compared to survey-based approaches, accessing publicly available course description allows for obtaining more data with less difficulty. 2.5 Integer Linear Programming (ILP) To determine the optimal combinations of curriculum courses to achieve the desired ECSF profile, it is essential to first identify a suitable search algorithm. This algorithm should be capable of identifying the best combination of the courses based on user-defined input conditions, such as the number of years of the study program, the maximum number of courses per semester, the total number of European Credit Transfer System (ECTS) credits, and the desired ECSF profiles to be covered by the curriculum. In addition, the algorithm should maintain a balance between the distribution of ECTS credits load between individual study years and semesters. These restrictions necessitate solving an optimization problem. One potential approach to addressing this challenge is to use Integer Linear Programming (ILP) [ 13 ], specifically 0-1 Integer Linear Programming, as the decision variables (i.e., whether to select a course or not) are binary in nature. Although this problem is known to be Nondeterministic Polynomial (NP)-hard, existing solvers demonstrated efficient handling of problems with thousands of decision variables, which is well beyond the scope of our application. 2.6 Curricula Designer The SPARTA project established a robust community dedicated to collaboratively define, develop, and disseminate solutions aimed at strengthening cybersecurity measures. As part of the SPARTA initiative, a novel free web application named Cybersecurity Curricula Designer [ 7 ] was developed. This tool enables the creation of cybersecurity curricula tailored to meet specific job requirements. 1. Data Collection 3. AI-labeling 4. (Optional) Adjustment 5. ILP Setting and Selection 6. Results and Enhancement 2. ECSF Preprocessing Figure 1: Curriculum development methodology. Leveraging the SPARTA framework, which is derived from the National Institute of Standards and Technology (NIST) Workforce Framework for Cybersecurity (NICE framework), the application employs an insertion mechanism to analyze and align generated curricula with the NIST NICE work roles. Furthermore, the application was expanded to incorporate the key skills and knowledge outlined in the ENISA framework [8]. 3 CURRICULUM DEVELOPMENT METHODOLOGY We propose a novel AI-driven mapping and optimization techniques for cybersecurity curriculum development, establishing a connection between academy and industry. Syllabi serve as detailed documents outlining course information, structure, objectives, learning outcomes, and content, providing a comprehensive framework for understanding the skills and knowledge students are expected to acquire. In our approach, we analyze courses using an RNN algorithm trained on job ads, which represent market demand and contain skills groups from ECSF profiles. This analysis correlates the skills and knowledge described in the ECSF profiles with course contents gathered from the syllabi. Subsequently, we apply an ILP algorithm to select courses that match the required skills. These techniques facilitate the alignment of academic offerings with industry needs, ensuring that cybersecurity curricula are tailored to address current demands and trends in the field. The proposed methodology of AI-driven curriculum development is depicted in Figure 1. This methodology involves six key steps: data collection from various sources such as university websites and course catalogs; definition of an efficient mapping methodology to link identified courses with ECSF profiles; AI-labelling of courses for alignment with ECSF profiles; human adjustment of AI output if needed; selection of the most suitable combination of processed courses based on user-defined conditions; and presentation of the results on curriculum development/enhancement. These steps allow for the construction of a newly developed cybersecurity ARES 2024, July 30–August 02, 2024, Vienna, Austria Petr Dzurenda, et al. curriculum by combining available academic courses or adapting existing curricula to align with desired ECSF profiles. 3.1 Data Collection The data used for curriculum development and analysis were extracted from existing course syllabi. A syllabus can be set out by an examination board or tailored by the instructor who teaches or controls the course. These syllabi serve as comprehensive guides, providing essential information for curriculum development. They include details on course information, structure, purpose, objectives, learning outcomes, and content, offering a comprehensive overview of the skills and knowledge expected to be acquired by students. Specifically, a course syllabus includes: • Course Information: provides basic details such as the course title, code, type, level, and the academic year/semester it is offered. • Instructor Information: where the instructor’s names are listed. • Course Structure: outlines the workload distribution, including ECTS credits, weekly lectures and hours, as well as laboratory sessions, if applicable. • Course Purpose and Objectives: are stated to clarify the aim and expected outcomes for students. • Learning Outcomes: describe the knowledge, skills, and competencies students are expected to gain upon completing the course successfully. • Course Prerequisites and Co-requisites: required for enrollment in the course. • Course Content: provides a detailed breakdown of the topics and subjects covered throughout the duration of the course. • Teaching Methodology: describes the approach used for delivering the course content, such as face-to-face instruction or online delivery. • Bibliography: lists required readings, textbooks, and additional resources. • Assessment: includes the weighting of examinations, assignments, and class participation. • Language: is specified to ensure students are aware of the medium of communication used in lectures and materials. Language, course information, and structure sections provide fundamental details used for filtering courses, such as ECTS credits and semester allocation. Conversely, the content of sections on course purpose and objectives, learning outcomes, and course content were collected and fed into the AI-labeling process. In fact, these sections detail the specific skills and knowledge that students are expected to acquire. 3.2 ECSF Preprocessing The ECSF describes 12 profiles with a total of 84 skills and 69 knowledge, often expressed in unique phrasing. This presents a challenge in illustrating the relationships among profiles through the connections of identical skills and knowledge. One approach to address this challenge is to group together skills and knowledge that convey the same concept but are phrased differently. Consequently, skills and knowledge can be clustered based on their descriptions. For instance, Table 1 shows one such cluster along with the corresponding ECSF key skills and knowledge. Additionally, it displays the profile associated with each skill or knowledge. Table 1: "Software Development" skill group with ECSF key skills and knowledge linked to the associated ECSF profile. ECSF Skills and Knowledge ECSF Profile Skill. Configure solutions according to the organisation’s security policy Cybersecurity Implementer Skill. Develop codes, scripts and programmes Penetration Tester Knowledge. Review codes assess their security Penetration Tester Knowledge. Secure development lifecycle Cybersecurity Architect Knowledge. Computer programming Cyber Threat Intelligence Specialist, Cybersecurity Implementer, Penetration Tester Knowledge. Secure coding recommendations and best practices Cybersecurity Implementer It is noteworthy that when we initiated our analysis of skills needs in cybersecurity, the ECSF had not yet been established as a European classification of skills. Consequently, we relied on the NICE NIST competencies framework as our starting point due to its comprehensive structure and alignment with other existing frameworks [6]. The following steps were undertaken: (1) Some of the NICE competencies required adjustments to better suit the European (EU) market, were irrelevant for our analysis, or could be merged. Hence, REWIRE cybersecurity experts proposed a total of 29 REWIRE skills groups derived from the NICE competencies, which were then validated through a survey involving cybersecurity stakeholders. We refer to REWIRE Deliverable R2.2.2 [10] for more details. (2) The 29 REWIRE skills were used as the foundation for grouping the ECSF key skills and knowledge, with strict adherence to the descriptions provided by the REWIRE skills. (3) Three REWIRE skills were renamed to more accurately reflect the groups and their newly defined scope based on ECSF skills and knowledge. Specifically, “Communication” was modified to “Collaborate and Communicate,” “Enterprise Architecture” became “Enterprise Architecture and Infrastructure Design,” and “Information Technology Assessment” was updated to “Information Security Controls Assessment.” (4) Two new skills were identified as missing: “Problem Solving & Critical Thinking” and “Technology Fluency.” The definition of these new groups was informed by the descriptions provided in the NIST NICE competencies. (5) Following the creation of the initial draft of groups, REWIRE experts collaborated to ensure consistency in the grouping process. Cybersecurity Curriculum Development ARES 2024, July 30–August 02, 2024, Vienna, Austria Figure 2: Mapping REWIRE Skills Groups to ECSF profiles. Figure 2 illustrates the correlation between REWIRE skills groups and ECSF profiles. It is noteworthy that the proposed grouping establishes a connection between NICE competencies and the ENISA framework, thereby assigning competencies to profiles and enhancing the readability and usability of the profiles. Furthermore, the grouping strategy for key skills and knowledge identified a discrepancy in the ECSF framework, which may warrant future improvements. Specifically, several REWIRE skill groups lack assigned skills, knowledge, or both. These groups were initially identified independently from the ECSF framework and subsequently expanded with associated key skills and knowledge from the framework. This approach strengthened the definition of cybersecurity skills and identified skills overlooked in both REWIRE groups and the ECSF framework. Potential improvements include adding missing descriptions to existing profiles and considering the inclusion of any missing profiles in the ECSF framework. For further details, please refer to REWIRE Deliverable R3.4.1 [3]. 3.3 AI-labeling Our application utilizes a RNN to analyze the text inserted into the system, specifically targeting the extraction of relevant skills. RNNs are particularly adept at processing sequential data, making them well-suited for tasks such as NLP. In our approach, sentences are interpreted as sequences of words, which are then converted into integer representations and fed into the RNN model. This methodology, commonly used for tasks like sentiment analysis, is adapted to classify the required skills in the given text. Before the RNN can effectively analyze text and extract skills, it needs to be trained on a dataset of labeled examples. The training consisted of testing the model’s ability to predict the required security skills using a test dataset. The ground truths were manually labeled. Furthermore, the LSTM model was only trained on the training dataset to prevent the model from learning the test dataset and thereby biasing the training results. The training dataset comprised 937 cybersecurity job advertisement descriptions. Note that employing the same algorithm in job ads and course description datasets ensure: • Consistency in Skill Identification: Employing the same algorithm ensures consistency in identifying cybersecurityrelated skills across both datasets, enabling a unified approach to skill extraction and analysis. • Cross-Domain Skill Mapping: By applying the algorithm to both job ads and course descriptions, it becomes possible to map the skills demanded in the job market to the skills taught in academic courses. This facilitates alignment between industry needs and educational offerings. • Feedback Loop for Curriculum Enhancement: Analysis of both job ads and course descriptions can establish ARES 2024, July 30–August 02, 2024, Vienna, Austria Petr Dzurenda, et al. a feedback loop for curriculum enhancement, allowing institutions to adapt their programs in response to evolving industry requirements and technological advancements. After training the RNN, each course description collected in Section 3.1 was input into the RNN algorithm, which then automatically generated the list of identified skill groups. 3.4 (Optional) Adjustment The next step involves an optional manual review of the assigned skill groups for each course. This process requires a cybersecurity expert to examine the course descriptions provided as input to the RNN algorithm and verify the accuracy of the assigned skill groups. The manual review of the assigned skill groups serves as valuable feedback to the RNN results, allowing cybersecurity experts to validate and refine the accuracy of the skill group assignments. 3.5 ILP Setting and Selection The objective of this section is to give a detailed description of the ILP algorithm providing course selection, and therefore, curriculum design itself. The algorithm was configured to match the courses to chosen ECSF profiles so that all skill groups included in the profile are represented by these courses. As we theorised that a single course would not be able to match all skills and knowledge, multiple courses were planned to be selected. The found combination of courses is optimal if it matches the profile and the selected parameters set up by the user. It is also essential to get an optimal solution in the shortest possible time. The filtering applied by the user includes important numerical values such as the number of study years, number of ECTS credits, and included skills groups of the courses. Furthermore, the algorithm maintains a balance between the distribution of ECTS credit load between individual study years and semesters. These values, together with the maximum number of courses per semester, are subjected to optimization to find the best possible result. The ILP optimization method helps us to solve this task. The process of optimization is following: (1) First, we remove REWIRE skills groups from the selected ECSF profile that we are unable to cover with our set of courses. The subset of the skills groups is denoted R. (2) Second, we run the ILP algorithm. The algorithm takes as input: 1) the subset of skills groups R , 2) number of ECTS credits 𝐶 and its tolerance, 3) a number of study years 𝑌 , 4) the set of available courses 𝑀 with the associated skills groups they cover, and 5) the number of maximum courses per semester N C. (3) The ILP algorithm settings is defined by Equation 1. The equation says that we are looking for the smallest subset of courses 𝑏𝑖 that meet the given conditions. 𝐹𝑚𝑖𝑛 = 𝑀 ∑︁ 𝑖=0 𝑏𝑖,(1) where 𝐹min is a minimization of the objective function where 𝑏𝑖 represents a binary variable deciding whether the course will be included or not. (4) The following equations describe the constraints defining the range of tolerance of the number of ECTS credits in the entire study program. The 𝐶min and 𝐶max values defined in equation (2) refer to the credits range in which the sum of the courses should move. 𝑀 ∑︁ 𝑖=0 (𝑐𝑖𝑏𝑖) ≥ 𝐶𝑚𝑖𝑛 𝑀 ∑︁ 𝑖=0 (𝑐𝑖𝑏𝑖) ≤ 𝐶𝑚𝑎𝑥,(2) where 𝑐𝑖 defines number of credits per course 𝑏𝑖 , 𝐶min is the lower bound, and 𝐶max is the upper bound. (5) The condition defined by Equation 3 ensures an even distribution of courses between semesters. Namely, 𝑆min and 𝑆max define the minimum and maximum difference between the distribution of ECTS credits between semesters. The 𝐶 bounds are set by the user input, while 𝑆 bounds are set by default to 𝑆min =− 1and 𝑆max = 1. 𝑀 ∑︁ 𝑖=0 (𝑠𝑖𝑏𝑖) ≥ 𝑆𝑚𝑖𝑛 𝑀 ∑︁ 𝑖=0 (𝑠𝑖𝑏𝑖) ≤ 𝑆𝑚𝑎𝑥,(3) where 𝑠𝑖 defines semester type ( 𝑠𝑖= 1for summer semester and 𝑠𝑖=−1for winter semester). (6) The condition defined by Equation 4 guarantees that on average there are NC per semester. 𝑀 ∑︁ 𝑖=0 𝑏𝑖≥ N C · 2𝑌, (4) where 2 𝑌 defines the total number of semesters in the whole study program. (7) The last constraint defined by Equation 5 searches for courses that include the skills groups from the subset R . In the equation, 𝑥𝑗 stands for skills groups in R and 𝑛 is a number of skills group in R . The skills group 𝑥𝑗 takes the value 1 if it is covered by the course 𝑏𝑖, and 0 otherwise. 𝑀 ∑︁ 𝑖=0 (𝑥𝑗𝑏𝑖) ≥ 1for 𝑗=1, . . . , 𝑛 (5) (8) The algorithm is run twice. In the first run, an optimal solution is found. This solution aims to fulfill all parameters like credits, semesters, and matching the profile. All courses in the solution are given priority two. The algorithm is then executed again just with the courses already included in the solution and the output is only necessary courses to fulfill the profile criteria and also the semester balance is maintained. These courses are then given priority one. (9) In the final phase, the selected courses in the winter and summer semesters are evenly distributed into semesters. This spreads the selected courses over the entire length of the study program and maintain an even degree of difficulty between semesters. Note that we do not consider how deep each course covers each skill group. 3.6 Results and Enhancement The final step of the curriculum development presents the evaluation of the designed curriculum. The user gets visual information on key building courses of the curriculum which are essential to cover selected ECSF profiles, the number and distribution of ECTS credits across semesters, the level of coverage of all ECSF profiles, and information on missing skills groups to cover the required ECSF profile. The user is also given the option to replace non-key courses of the curriculum with other available courses, and thereby achieve an even higher adaptation of the curriculum to his/her own needs Cybersecurity Curriculum Development ARES 2024, July 30–August 02, 2024, Vienna, Austria Figure 3: Graphical user interface of the CSProfiler tool. e.g., by adding a higher importance to preferable skills groups or expanding the scope of the curriculum to cover more ECSF profiles. 4 CYBERSECURITY PROFILER: PURPOSE, TARGETED USERS, AND USABILITY The REWIRE CyberSecurity Profiler (CSProfiler) is an open-source, freely available, dynamic web application that serves as a comprehensive tool for designing and analyzing curricula, professional training, and certification schemes in the cybersecurity domain. Building upon the foundation of the SPARTA Curricula Designer, it extends its capabilities by allowing dynamic analysis of courses and their optimized selection for the creation of specific cybersecurity curricula. This comprehensive tool enables users to design their curricula, certifications, or training programs while accessing real-time statistics in compliance with ENISA ECSF and NICE frameworks. Specifically, the CSProfiler maps existing curricula, trainings, and certifications to specific cybersecurity roles defined by ENISA profiles, identifies recommended courses for these roles, and allows users to design study programs accordingly. Through its integration with the 31 REWIRE group, users can easily navigate and maintain a database of existing cybersecurity educational resources. 4.1 Using the Tool The CSProfiler tool fully integrates the AI-driven cybersecurity curriculum development concept presented in the previous sections. The tool is divided into three main sections, see Figure 3: 1) the left section (Available Courses) allows users to define new courses or modify the existing ones, 2) the middle section (Suggested Curricula) allows the composition of a study program from defined courses with help of AI, and 3) the right section (Statistics) provides the statistical data and compliance of the designed program with the requirements. When users add courses, they are required to input various data, including the course name, type, semester, and number of ECTS credits, along with a text describing the course aims, objectives, and outcomes. This is where the AI algorithm plays a crucial role. By analyzing the provided course description, the algorithm determines the skill groups covered by the course as shown in Figure 4. Figure 4: Course edit menu in the CSProfiler tool. ARES 2024, July 30–August 02, 2024, Vienna, Austria Petr Dzurenda, et al. If users do not agree with the AI recommendation, they can easily remove or add skills groups or even more, they can use more detailed classification of their courses by using the specific ECSF skills and knowledge as depicted in Figure 5. Figure 5: Course classification with skills groups. When a user creates all needed courses, then he/she can run the automatized curriculum development driven by the CSProfiler. To do so, the user has to fill in his/her requirements on the curriculum and run the analysis process in the corresponding CSProfiler section as depicted in Figure 6. Figure 6: Curricula setting Menu. The result of the analysis is the selection of the most optimal combination of courses from the set of available courses covering the selected ECSF profiles and their division into individual semesters (Figure 3, middle). The tool highlights (red text font of the courses titles) those courses that are key to the selected ECSF profiles. Notably, key courses are a minimal combination of courses with needed skills for the selected profile. The other courses have a supplementary character and deepen knowledge in the already covered ECSF profiles with regard to the maximum number of credits for the entire study. In the right column of Figure 3 (Statistics), the tool displays the statistics of the created curriculum, i.e., the total number of ECTS credits, covered ECSF profiles, and missing and covered skills’ groups. If necessary, users can move courses between the left column (Available Courses) and the middle column (Suggested Curricula) and thus optimize the proposed curriculum according to their needs. 4.2 Technical Implementation As shown in Figure 7, the implementation of the web application is divided into two parts: 1) client and 2) server with database. The client is a Javascript application that is downloaded and launched in the user’s web browser after the first loading of the website. This application contains a Graphical User Interface (GUI) and mediates all interaction with the user in real time. In the case of working with database data or processing user data, the application connects to the server, which mediates communication with the database and provides computational analysis tools. Web Browser with JS App Server API MariaDB Python Analysis Scripts Client-side Server-side HTTPS PHP PDO Terminal Figure 7: Application structure. 4.2.1 Client-side. This part represents the front-end of the entire system. It is a dynamic Javascript web application based on the React framework. The code is executed directly in the user’s web browser, allowing for immediate feedback to the user. The following programs and libraries were also used: •Antd – Graphical design and components, •React Bootstrap – Graphical components, and structure, •Apexcharts – Graphs and plots, •Crypto – Cryptography features, •React Beautiful DnD – Drag’n’Drop mechanism, •Node-sass – Style preprocessor, •Axios – Connection to the data server, •NPM - Node Packet Manager. The CSProfiler application is part of a larger web application called the CyberAbility Platform 1 which is open-source and available on the public GitLab repository 2 . The CSProfiler is implemented as a React component that inherits several variables and callback functions from the parent application within component 1https://cyberability-platform.informacni-bezpecnost.cz/ 2https://gitlab.com/brno-axe/rewire/cyberability_platform Cybersecurity Curriculum Development ARES 2024, July 30–August 02, 2024, Vienna, Austria attributes called props. These mainly concern the management of user accounts and data, connection to the server, various dialog windows and functions for displaying content. CSProfiler itself also contains its own set of variables called state, in which it stores data related to its own content. 4.2.2 Server-side. On the server side, there is a server Application Programming Interface (API) created in Hypertext PreProcessor (PHP). This API accepts requests from the client application and returns confirmation of the performed action or requested data. Individual actions are distinguished based on the requested URL and depending on the received data from the client application. The data is stored in the MariaDB SQL server on the same server machine. Access to the database is done through the PHP API. In order to keep the data safe from unauthorized manipulation through the web application, an authentication and authorization mechanism using user tokens is also implemented in the API. The server also offers analysis tools that allow users to analyze their data. These tools are implemented as Python scripts and executed on the same machine where the API server and the database are running. These analysis tools are more detailed in the next subsection. 4.2.3 Server Analysis Tools. The server includes two Python scripts to perform 2 different analysis. These Python scripts are run using the input provided by the client, and upon successful termination, the result of the analysis is processed by the PHP API and sent to the client. The first analysis tool is for skill prediction using a machine learning algorithm. This tool consists of a tokenizer and an Long short-term memory (LSTM) model. The entire tool is implemented in Python 3 and also allows retraining of the model. This analysis tool was introduced and described in [ 17 ]. The second analysis tool is used to design an optimal curriculum based on the required professional profile from available study courses. This analysis is based on the integer linear programming algorithm described in Section 3. 5 CASE STUDY The proposed curriculum development methodology was applied to the "Information Security" master’s study program [ 12 ] offered at Brno University of Technology in the Czech Republic. This program comprises a total of 60 courses, consisting of 15 compulsory, 17 compulsory-elective, and 28 elective courses. Compulsory courses provide fundamental knowledge, compulsory-elective courses guide students towards specialization in desired work roles, and elective courses offer opportunities to broaden students’ general knowledge. The master’s program spans two years, during which students are required to earn a minimum of 120 ECTS credits from a total of 271 credits offered. The "Information Security" master’s program offers an interdisciplinary education, encompassing technical fields such as applied mathematics, cryptography, informatics, and humanities, with a special emphasis on law. This interdisciplinary approach provides students with a comprehensive understanding of cybersecurity from various perspectives, making it an ideal candidate for applying our curriculum development methodology. With the program’s emphasis on cybersecurity, we aim to explore how students can Table 2: Adjustment results of the AI-labeling. ♯Courses Added Skills ♯Courses Deleted Skills 23 0 37 <5 23 1 2 5 8 2 21 >5 6 3 or 4 tailor their studies to align with specific ECSF profiles, thereby enhancing their readiness for roles in the cybersecurity sector. The data collection phase involved downloading the syllabi of each master’s course and extracting the content from sections such as course description,aims, and syllabus of lectures and laboratories. Subsequently, each course underwent AI-labeling followed by human adjustment. The AI-labeling process was executed by simply clicking a button in the CSProfiler, while manual adjustments were carried out by REWIRE cybersecurity experts. To track the changes made during the adjustment process, an Excel file was created to monitor the addition and removal of skills identified through AI-labeling. Table 2 provides an overview of the number of courses that had specific amounts of skills added or removed. Notably, the RNN algorithm demonstrated good performance, with only a small number of the expected skill groups missing from the courses. Specifically, only 4 courses required the addition of 3 skills each, and only 2 courses required the addition of 4 skills each. However, several courses necessitated the deletion of incorrectly assigned skill groups. It is noteworthy that courses in law and languages exhibited a higher frequency of skill group deletions. This can be attributed to text descriptions that, while mentioning technical topics, do not deepen into details. After inputting the first-year compulsory courses into the CSSProfiler, it became apparent that the curriculum already covers the requirements for the Cyber Legal, Policy & Compliance Officer profile. This profile necessitates proficiency in four skill groups: "Collaborate and Communicate," "Data Analysis," "Data Privacy," and "Law, Policy, and Ethics," all of which are adequately addressed by existing language, data processing, and law courses. Additionally, the master’s program offers a foundational coverage of essential skill groups for various other profiles, including Cyber Incident Responder, Cybersecurity Architect, Cybersecurity Auditor, Cybersecurity Educator, Cybersecurity Implementer, Cybersecurity Researcher, Cybersecurity Risk Manager, and Digital Forensics Investigator. Furthermore, the ILP algorithm can assist in structuring a curriculum tailored to specific profiles, such as the Cyber Incident Responder. For this purpose, we configured the ILP with the following parameters: a two-year academic timeline, a maximum of six courses per semester, an ECTS range of 120 ± 5, and a focus on the Cyber Incident Responder profile. A comparison between the results obtained from the ILP selection and those following the compulsory/elective credit requirements reveals an overlap of 8 out of 19 courses. This discrepancy arises from the ILP’s ability to select courses based on the presence of one or more skill groups required for the profile, without consideration of compulsory and elective course constraints. The ILP analysis results can be used to