Building a large, realistic and labeled HTTP URI dataset for anomaly-based intrusion detection systems: Biblio-US17

Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 https://doi.org/10.1186/s42400‑024‑00336‑3 RESEARCH Open Access © The Author(s) 2025. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. Cybersecurity Building alarge, realistic andlabeled HTTP URI dataset foranomaly‑based intrusion detection systems: Biblio‑US17 Jesús Díaz‑Verdejo1 , Rafael Estepa2 , Antonio Estepa2* , Javier Muñoz‑Calle2 and Germán Madinabeitia2 Abstract This paper introduces Biblio‑US17, a labeled dataset collected over 6 months from the log files of a popular public website at the University of Seville. It contains 47 million records, each including the method, uniform resource identi‑ fier (URI) and associated response code and size of every request received by the web server. Records have been classified as either normal or attack using a comprehensive semi‑automated process, which involved signature‑based detection, assisted inspection of URIs vocabulary, and substantial expert manual supervision. Unlike comparable datasets, this one offers a genuine real‑world perspective on the normal operation of an active website, along with an unbiased proportion of actual attacks (i.e., non‑synthetic). This makes it ideal for evaluating and comparing anomaly‑ based approaches in a realistic environment. Its extensive size and duration also make it valuable for addressing challenges like data shift and insufficient training. This paper describes the collection and labeling processes, dataset structure, and most relevant properties. We also include an example of an application for assessing the performance of a simple anomaly detector. Biblio‑US17, now available to the scientific community, can also be used to model the URIs used by current web servers. Keywords Anomaly detection, Intrusion detection systems, Data acquisition, Training datasets, Web application filters, Biblio‑US17 dataset *Correspondence: Antonio Estepa [email protected] Full list of author information is available at the end of the article Introduction More than 200 million web servers are operational on the Internet today, facilitating the delivery of Web Services and websites. These servers play a critical role in enabling users to share information and accomplish much of their daily tasks (Forbes. 2023; Ageed et al. 2021). Consequently, the Hypertext Transfer Protocol (HTTP) now accounts for a significant portion of Internet traffic (Ibrahim etal. 2021). However, this widespread use also makes web servers frequent targets for cyberattacks and malware distribution (Husák etal. 2021). According to Apps (2021), some 30,000 websites were hacked daily in 2021. Most of these attacks target vulnerabilities in website software, especially in content management systems, where plugins and outdated software play a significant role in increasing the risk of breaches (BenMartin etal. 2022). Web attack detection has traditionally relied on intrusion detection systems (IDS) (Agarwal and Hussain 2018) and Web Application Firewalls (WAFs) (Pałka and Zachara 2011). Many IDS and WAFs identify attacks by looking for recognizable patterns in data or its features. Feature-based intrusion detection techniques primarily rely on predefined intrusion characteristics, known as Indicators of Compromise (IoCs), to Page 2 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 identify known attack patterns. As such, they are wellsuited for detecting known threats. However, despite the availability of open-source repositories (Díaz-Verdejo etal. 2022), IoC datasets, with their high accuracy, are often considered valuable intellectual property by threat intelligence vendors and are typically not made public. On the other hand, anomaly-based intrusion detection systems (AIDS) focus on identifying deviations from established normal behavior, making them particularly effective at detecting unknown or novel attacks (which made up over 42% of attacks in 2021 according to some estimates zer 2023). AIDS are not without challenges, such as false positives (Diaz-Verdejo etal. 2024a) or their reliance on quality datasets to learn normal behavior (Betarte etal. 2018; Hajj etal. 2021; Khraisat etal. 2019). Using raw data, such as traffic and logs, is more appropriate for anomaly-based detection because it captures the full spectrum of user and system activities (Garcia-Teodoro etal. 2009). The dataset introduced in this paper is designed specifically for anomaly-based IDS evaluation, which is a reasonable approach given the nature of the data. The challenges associated with constructing real-world log datasets are significant. Firstly, privacy concerns arise because these datasets often contain sensitive information, such as IP addresses, which organizations are reluctant to make publicly accessible to avoid privacy violations. As a consequence, reproducibility issues emerge since evaluations are frequently conducted on private datasets, preventing the reproducibility of the results and comparability of the proposed approaches. Secondly, ground truth determination becomes challenging amidst unknown user activities, as the labeling process ideally requires expert examination, which is prohibitive in large datasets (Díaz-Verdejo etal. 2020). On the other hand, synthetic log datasets generated through simulations offer some advantages. They provide a more flexible technical environment, enable the launch of arbitrary attacks, and ensure a reliable ground truth due to the absence of unknown activities (Gao etal. 2017). However, there are limitations to using synthetic data. Firstly, real-world complexity may not be fully captured, as simulations might miss erratic user behavior or neglected configurations, leading to a lower rate of false alarms (Landauer et al. 2022). Secondly, long-term changes in system infrastructure or content are often overlooked in shortterm simulations, resulting in a lack of representation of evolving environments. Lastly, the number and content of synthetic attacks may not reflect the proportion, diversity and sophistication of real-world attacks. Given these challenges, the dataset introduced in this paper aims to bridge the gap by providing a realistic yet manageable log dataset specifically designed for anomaly-based IDS evaluation. We present a new dataset intended for scientific use in the context of URI-based anomaly detection: Biblio-US17 (briefly outlined in Diaz-Verdejo et al. (2024b)). Our dataset is derived from access log files of the public website of the University of Seville Library in Spain and has three major distinctive characteristics: • It was collected in a real operating environment. • It contains a large volume of records (over 47 million) spanning a broad period (six months), allowing for the study of long-term data changes to some extent. • It is labeled: we conducted a thorough semi-automated process to determine ground truth. This required a formidable effort, but as a result, a realistic model of normal website behavior can be inferred. Additionally, the performance of anomaly-based IDS can be more accurately assessed using real attacks targeted at the active website. Biblio-US17 also has some limitations (see section"Limitations and challenges"), the most notable being that data is restricted to the method, URI, response code, and response size for each request received by the server. While records are in chronological order, they lack full timestamps, which limits the usefulness of this dataset for time-based anomaly detection techniques. Despite its limitations, we are confident that Biblio-US17 will be a valuable addition to public datasets, improving the evaluation of future URI-focused anomaly-based IDS and enabling comparison of different approaches with realistic data. We hope this effort will lead to broader industry recognition and acceptance of academic research outcomes. The remainder of this paper is organized as follows. Section "Background and related work" discusses the background of creating datasets for web-attack detection and reviews related work. Section"Acquisition and labeling" describes the processes of data acquisition, preparation, and labeling. Section"Dataset description" provides a detailed description of the dataset, including the structure of files, directories, and format. Section"Suggested use and illustrative case study" addresses partitioning for research purposes and includes an illustrative case study. Section"Limitations and challenges" highlights the main limitations of the dataset, and section"Conclusions" concludes the paper. Page 3 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 Background andrelated work The creation of suitable datasets with real-world properties is a challenging endeavor (Sharafaldin etal. 2018). For a dataset to be adequate for AIDS development it should at least be (Viegas etal. 2017): 1. Representative. This means that the data used should have similar properties to the data found in the system being protected, such as being valid, current, and specific to the deployment environment. It is worth noting that using data from operational environments is the most effective, whereas synthetic data may not accurately represent an realistic operational context (despite recent attempts to reasonably generate realistic log data (Landauer etal. 2020; Uetz etal. 2021)). 2. Sufficiently large. The dataset should include enough volume of data to be partitioned and still be useful for training, evaluation or validation purposes. The training dataset size should suffice to train the normality model with statistical significance (Estepa etal. 2020). Ideally, the collection period should also be long enough to address the issue of data variability over time. Synthetic data generated with scripts usually lacks the variability of data from operational infrastructures (Landauer etal. 2022). 3. Labeled. Each data record should be classified as normal or an attack. However, labeling can be challenging when dealing with datasets collected from public operational deployments, as they may contain errors or attacks (Mahoney and Chan 2003; Brugger and Chow 2007). For this reason, data sanitization is advised (Cretu etal. 2008) to identify attack instances in the collected data and reduce the number of unlabeled attacks or data errors as much as possible. This process should ideally include manual expert supervision (Paxson 2004), which is not feasible for very large datasets. A common method to reduce the sanitization cost is to utilize a signature-based IDS to identify the majority of known attacks (Díaz-Verdejo etal. 2022). Still, expert supervision is unavoidable if one aims to obtain quality datasets free of attacks or undesired artifacts (Cretu etal. 2008). In contrast, labeling is straightforward in a synthetic datasets since all activity is known. Besides the above requirements, a dataset collected from a public environment should comply with applicable privacy regulations, such as the European Union’s General Data Protection Regulation (Hjerppe et al. 2019). Furthermore, security should also be considered as some data could potentially disclose information about the web server infrastructure, facilitating future attacks. To address these issues, privacy-preserving techniques such as generalization, suppression or perturbation should be applied (Fung etal. 2010). The latter consist of replacing the original data values with synthetic ones that preserve owners’ privacy while still retain useful statistical information (Reiter and Rubin 1998; Salazar-Hernández and Díaz-Verdejo 2010). Datasets suited forweb attack detection The URI field in HTTP requests plays a central role in identifying web attacks (Saxena et al. 2022). Indeed, approximately 85% of attack signatures used by common SIDS/WAF include the URI field (Díaz-Verdejo etal. 2022). This section discusses datasets currently available to researchers for developing and evaluating anomalybased IDS that specifically target the content of HTTP messages, particularly the URI. It is important to note that we have excluded datasets that do not cover HTTP messages, even if they have other valuable properties for anomaly detection, such as those based solely on traffic flows. It is also important to note that there are unlabeled datasets of HTTP messages collected from reallife deployments, such as Schulz etal. (2021), Chodak etal. (2020), that are available. However, they have been excluded from this section because they cannot be used for AIDS evaluation, due to the absence of labeling. To the best of our knowledge, Table1 lists all publicly available labeled datasets containing the URI field of HTTP requests. The table provides information on the source file format, labeling categories, whether the dataset was artificially generated or collected from an operational environment, whether labels were manually supervised, the collection date (year) and period, and number of usable URIs in each dataset. A noticeable common characteristic among these datasets is that they have been artificially generated (i.e., synthetic) and automatically labeled after generation. The only exception is NSL-KDD, where labels underwent a subsequent supervision. Only UNSW-NB15 contains a mix of synthetic packets (attacks) and live-captured ones. However, the labeling has been done without supervision. From the perspective of dataset acquisition and construction, our approach significantly differs from those utilized in the datasets included in Table1. As previously mentioned, to prioritize error-free and ease labeling, all datasets in Table1 generate traffic synthetically within controlled environments. A basic approach is exemplified by DARPA99, where traffic is captured during the simulated normal operation of users in a controlled network. In this case, all user-generated traffic is assumed to be normal, while various types of attacks are injected from a specific station in a controlled manner, facilitating easy Page 4 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 labeling. Conversely, CIC-IDS2017 represents a more sophisticated approach, where user profiles are established based on real traffic monitoring before synthetic traffic generation. These profiles aim to maintain proportionality among different user types during dataset generation, thereby minimizing potential biases in the generated traffic. However, even in this approach, attacks are synthetically and controllably introduced, which does not reflect their actual frequency in real-world scenarios. In contrast, our approach starts from the opposite situation: we capture real-wold traffic from an active website, including artifacts, errors, and potential attacks, and then proceed to identify normal and attack traffic. Admittedly, this approach creates a “gray area” where some requests cannot be labeled due to a lack of information or ambiguities. However, similar situations are also observed in some datasets from Table1 (e.g., NSL-KDD). The main advantage of our approach is the realism and representativeness of the data, free from biases associated with the generation of both clean traffic and attacks. Regarding data volume, we observe a significant difference in the number of records of the datasets in Table1 compared to ours. A low size may negatively impact the statistical significance of the normality models inferred. Furthermore, the use of simulations results in reduced diversity in the URIs, typically responding to relatively simple servers. For example, DARPA98 used two servers with limited vocabularies and a low number of variables/ values per URI. In other cases (e.g., CIC-IDS2017), a relatively small number of URIs appear frequently, leading to reduced variability. In no case is a content management system (CMS) was used at the server (such as in our dataset), and the complexity of the service and URI variability is generally reduced compared to ours. Moreover, the frequency and type of attacks are synthetically generated in all datasets, introducing a significant bias in detection performance assessments (TP/FP rates lacking real significance). Due to the invariant traffic generation mechanism and the cost of simulation execution, the datasets in Table1 lack sufficient temporal duration and variability to analyze issues related to temporal drift. In contrast, capturing traffic in a real scenario is relatively straightforward, allowing for extended collection periods, such as in our case. In our approach, the cost of building the dataset is associated with the labeling process rather than data generation. Finally, it is worth noting that some of the datasets from Table1 are very old (e.g., from 2010 or older). As a result, there is a risk that the URIs (whether malicious or benign) may not accurately reflect current request patterns, potentially limiting their effectiveness in evaluating AIDS proposals intended for operational environments. Biblio-US17, collected from a real-world active website, is labeled with normal and attack classes. It features a large volume of records, a collection period suitable for studying time shift issues to some extent, and is recent Table 1 Public datasets used in the literature for web intrusion detection systems assessment (Díaz‑Verdejo et al. 2022) * Classes: N (normal), A (attacks), U (unlabeled), An (anomaly) ** Labels: G (As generated), S (Supervised) Dataset Format (URIs) Classes Real/Synth Labels Only HTTP Year Duration # URIS Comments DARPA’99 PCAP N/A/U S G N 1999 5w 100k Widely used for IDS research, flawed, obso‑ lete, not http specific traffic/attacks, only 2 web servers KDD’99 CSV N/A/U S G N 1999 5w – Parameterized flows, directly derived from DARPA’99, inherit DARPA’99 flaws NSL‑KDD PCAP N/A/U S GS N 2009 5w 10k Sanitized version of KDD’99, not for URI based analysis CSIC2010 LIST N/An S G Y 2010 – 96k Only URI, mixes attacks with anomalies, single server UNSW‑NB15 PCAP N/A RS G N 2015 31h 27k Lack of details about web servers and traffic ISCX‑URL2016 CSV N/A S G Y 2016 – – Outgoing traffic, 114k URIs, not suitable to train a site, highly unbalanced (only 35k normal URIs) CIC‑IDS2017 PCAP N/A S G N 2017 5d 272k 392k URIs, mix of outgoing and incoming traffic, most incoming URIS are attacks, low variability in URIs (only 6k queries and many repeated URIs) Biblio‑US17 CSV N/A R S Y 2017 6m 47M HTTP log from public Web Server with nor‑ mal and attacks URIs, high variability in URIs, wide timespan Page 5 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 enough to reflect current web dynamics. The dataset also benefits from being old enough for some 0-day attacks to now have public signatures, with privacy concerns mitigated as the server is not longer active. Acquisition andlabeling The dataset originates from the traces generated by the web service of the Library of the University of Seville (http:// bib. us. es) between January 1st and July 17th, 2017. The web application, developed with Drupal v7.96 content management system and the Apache web server (v2.2), provides information about bibliographic holdings. As such, most requests are bibliographic information searches (e.g., book titles, papers, thesis, authors) and, to a lesser extent, content updates from library staff. The website includes also news and service information. The site, with over 50,000 students, experiences high demand and a wide range of requests, making it highly dynamic. Our data source comes from the Apache access log files that span 198 days and contain a total of 47,902,323 request lines. During this period, the portal transitioned between HTTP and HTTPS, so both service ports generated log files. For security reasons, the university did not handed us the original log data files but a processed copy of them (one daily file named after the date of collection) with the following data fields for each request: method, URI, protocol version, response code and response size as shown in the following box. As illustrated in Fig.1, we carried out the tasks of data preparation and labeling with the files that were given to us: (i) Data preparation, in which we produced a unique ID for each record, suppressed unsuitable records (e.g., malformed messages according to standard syntax or requests without the URI field), and further anonymized sensitive information. (ii) Labeling, in which we conducted a semi-supervised sanitization process that allowed us to detect attacks with different reliability levels (labeled as LVL1 to LVL4) and further filter unsuitable records that did not comply with standard syntax for various reasons (labeled as OOS1 to OOS4). We obtained a final count of 47,402,996 suitable registers from which the clean/attack datasets were generated. The remainder of this section elaborates on these processes. Data preparation process This process aims to obtain an initial, homogeneous, well-formatted set of records that complies with privacy and security regulations and includes, at least the method and URI of each record. Preprocessing andconditioning To create a properly-organized and well-formed dataset, we carried out the following steps: 1. We eliminated request lines that missed the URI field, which led to the removal of 499,327 lines. Most of these lines were HEAD requests without a corresponding response code. 2. Each request was assigned a unique Register ID (RID) label that shows the collection date and sequence order in the original log file. For example, the RID for the 123rd record in the HTTPS log file dated March 1st would be [03-01-S000123]. The RID can also be seen as a basic temporal reference, indicating the date and order of the request rather than a full timestamp. 3. We utilized a parser to check the conformity of requests with standard syntax. If possible, the issues identified (e.g., character ’ \ ’ ending the request, truncation errors, etc.) were fixed. Otherwise, malformed requests were eliminated, resulting in the removal of 89 registers. Fig. 1 Sanitizacion steps as derived from Díaz‑Verdejo et al. (2020) Page 6 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 By the end of this task, the dataset contained a total of 47,402,907 request lines, each with a unique RID and the fields shown in the example above. These lines underwent the anonymization process described next. Anonymization This process aims to detect and obfuscate sensitive information that could compromise the website security by disclosing service infrastructure, or violate data protection regulations. This step was compulsory and required the cooperation and approval of independent and external personnel (legal and security teams). To achieve this, we carried out two sequential tasks: 1. Identification of Sensitive Information: we searched for information related to personal data or server security (considering percent encoding) in the URIs. To identify Personal Data, we carefully searched for specific keywords and patterns, such as login credentials, passwords, names, emails, or user IDs. Although the search was automated, every occurrence was manually inspected. As a result, this process was time-consuming and took about two months to complete. We found that most of the sensitive information belonged to the authors of the papers or books users were searching for, not the service users. However, we also identified some sensitive personal data, including 62 different University of Seville user IDs (uvus), two IP addresses, and one email address. To identify Infrastructure Sensitive Data, we searched for institutional server names (i.e., domain name us. es) and filenames with potential sensitive information. A total of 4371 different filenames (with extensions such as.doc,.js,.pdf,.pps,.ppt,.swf,.txt and.xls) were found in more than 16 million URIs. Regarding institutional servers, we found 27,187,785 URIs that contained up to 44 different server names. All the information identified in this task (e.g., filenames, server names, user’s sensitive information) accounted for 4480 text strings, which should be obfuscated in the corresponding URIs to prevent future attacks on the infrastructure and to comply with privacy regulations. 2. Pseudo-anonymization of sensitive information: we applied a pseudo-anonymization procedure to the strings identified in the previous task. To preserve to a large extent URI characteristics, we used a substitution technique with the following characteristics (Salazar-Hernández and Díaz-Verdejo 2010): • Dictionary-based: every occurrence of the same original string will undergo the same substitution. • The substituting string keeps the following statistical properties of the original string: length, number of alphanumeric characters, and number of upper and lower characters. Special characters (not alphanumeric) are preserved and kept in their original position since they are commonly part of malicious URIs (Saxena etal. 2022). • Filename extensions are preserved. We started the substitution process by the longest string to avoid iterative substitutions. Table2 shows some illustrative examples of substitutions (in bold). While obfuscation affects the data used for learning or evaluating AIDS, its impact varies depending on the extent and method of string substitution. We aimed to minimize this impact on the dataset. Ultimately, we substituted 5.4% of the URIs’ strings, while preserving most of the URI characteristics relevant to classification (Saxena etal. 2022; Abad etal. 2023). We conducted a preliminary assessment of the impact of the anonymization process based on known attack detection. The nonanonymized attack dataset would have only 3 additional attacks (out of +30k), while the clean dataset remains unchanged. The dataset obtained after the anonymization process is published under the name full dataset (see Fig.1) and contains all the registers considered in the labeling phase. Table 2 Examples of anonymization Original URI Pseudonymized URI Sensitive information /arquitectura/users/adolfo /arquitectura/users/kxvacj uvus (user id) /search/google/adolfo /search/google/kxvacj /search/node/benito2 /search/node/hy3urta /sites/server1.co.es/files/Jhon_1234_webSocket.pdf /sites/xzy2kpq.kp.le/files/nHom_3782_pNrdpvbqr.pdf Domain and Filename /turismo/sites/server1.co.es/files/Viajes20-22.doc /turismo/sites/xzy2kpq.kp.le/files/htY1gbn7-98.doc /sites/server2.co.es/files/ /sites/kjpqb7g.bn.yt/files/ /sites/server.us.es/files/js/js_vp4-pa9.js /sites/pomnht.mn.df/files/js/js_p8m-n1t.js Page 7 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 Labeling process We conducted a thorough study of the full dataset to establish ground truth and label each record as clean, attack, or not in compliance with standard syntax. The process is outlined in Fig.1, and, in general terms, it follows the sanitization methodology proposed in DíazVerdejo etal. (2020). We have followed three sequential steps: (S1) use of signature-based IDSs to detect known attacks, (S2) manual supervision of the alerts produced during S1 to detect false positives and confirm true positives, and (S3) inspection of the URI vocabulary: a semiautomated task to find new attacks or non-compliant registers. The following subsections elaborate on these steps. Known attacks detection (S1) We used the signature-based IDS tool InspectorLog (Díaz-Verdejo etal. 2024c) (a tool for offline processing), to detect attacks in log files by applying the detection rulesets from the official repositories of the open-source IDSs Snort (Tal 2023; ETo 2023), Nemesida (free version) (Nem 2023), and the WAF ModSecurity (OWA 2023). Only rules including the URI field were selected except for those labeled as DELETED in the Talos and ETOpen repositories. ModSecurity was run using paranoia levels 1, 2 (level 3 was discarded due to an overwhelming rate of false positives). Table3 shows, for each ruleset, the number of rules selected, the number of alerts produced, and the number of different URIs referenced in these alerts. The results in columns two to four show that Snort’s rules produced most of the alarms (83% of them). A total of 283,165 URIs (barely 0.6% of the URIs in the full dataset) were classified as attack by these signature-based IDSs. Inspection (S2) Each URI classified as an attack by InspectorLog was manually inspected by an expert to determine whether it was a true positive (TP) or false positive (FP).1 URIs found to be TPs were further classified and labeled according to their soundness and scope as either indubitable (labeled as LVL1) or context-dependent (labeled as LVL2). The latter indicates that, although the URI is considered an attack in our particular context, it could potentially be non-malicious in a different operating environment. The rightmost part of Table3 (columns five to seven) shows the results obtained for each ruleset. The last column shows the attack level category (i.e., LVL1 or LVL2) of the URIs manually verified as TP. In the bottom row, Table3 shows the overall number of attacks: 273,943 ( ≥ 99% indubitable), which accounts for 0.58% of the registers. It is worth noting that, during manual examination, we found that most of the attacks detected by Talos and ETopen rules were related to a percent encoding attack using the sequence %252525... with increasing size (there were 314,076 URIs using %252525). More specifically, in 257,258 cases, the URI size was greater than 1450 characters (which is a threshold defined in the rule SID=17,410). Moreover, only 15,076 of the known attacks detected in this phase were not related to this string, which would lead to an attack rate of 0.03 % of the URIs in the full dataset. Vocabulary inspection (S3): segmentation‑based analysis This step aims to further discover unrecognized attacks or URIs out of specification in order to build a quality dataset. For this, we followed the semi-supervised methodology described in Díaz-Verdejo etal. (2020) in which the revision focuses on the different text strings that compose the URI (some 80k) rather than the URIs (some 47M). The method can be summarized as follows: • Dictionaries formation. URIs deemed clean (which also include FP in S2) are parsed and segmented to Table 3 Attack detection results per ruleset Ruleset #Rules # URIs classified as Activated #Alerts #URIs #FP #TP (LVL1/LVL2) Talos (22/02) + ETopen (22/03)(Snort) 175 287,253 278,404 8571 269,833 (26,8148/1685) Nemesida (21/11) 125 34,520 19,252 627 18,625 (17,774/851 ) CRS3.3.2‑PL1 (ModSecurity) (22/04) 43 18,345 9206 108 9098 (9098/0) CRS3.3.2‑PL2 (ModSecurity) 75 22,629 10,085 243 9842 (9482/0) Total 418 344,942 283,165 9222 273,943 (272,103/1840 ) 1 A preliminary rule classification helped reduce the amount of manual work by deactivating those rules that generated exclusively false positives (e.g., SIDs 41742 and 1852 from the TALOS repository, which refer to robots and generated more than 50,000 FPs). Page 8 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 generate three dictionaries from the text strings (i.e., words) found between standard delimiters that constitute the path, key y value parts of the URI. To build the dictionaries, we used the Uriparser2 library, which is RFC 3968 compliant. During the segmentation process, the parser identified 169 non-compliant URIs due to the presence of characters not permitted or an incorrect sequence of fields. The registers containing these URIs were labeled as Out-of-Specification (OOS) and ignored in posterior analysis. Table4 shows the number of different words learned in each dictionary (size) and number of instances of such words in the URIs that were subject to vocabulary inspection (after S2). • Vocabulary inspection: The vocabularies in these dictionaries were manually supervised seeking traces of attacks. This required a formidable amount of work. URIs containing words associated with attacks were labeled accordingly with a certain level of confidence. A preliminary review of the three dictionaries allowed us to identify some issues in the segmentation carried out in the path and key dictionaries, which led us to further classify some URIs as OOS. The last two columns of Table4 show the number of different words learned in each dictionary (size) and number of instances of such words in the URIs after excluding the URIs labeled as attack or OOS in S3. The main issues found in each dictionary are summarized bellow (the interested reader is referred to A.1 for further details). – Key Dictionary. We identified common patterns followed by 1579 words (86%) of this dictionary, which were found to be normal after inspection (seeA.2). The remaining words were inspected in search of potentially troublesome characters according to RFC 3986 such as gen-delims: (: / ? # [ ] @) or sub-delims (! $ & ’ ( ) * +,; =). As a result of this revision, some URIs were labeled as OOS (further details are inA.2). – Path Dictionary. As with the previous dictionary, the search for delimiters and a preliminary inspection of the path-related vocabulary revealed some encoding issues (labeled as OOS) and semantic errors (see A.3 for further details). We discovered evidence of some attacks based on the use of incorrect or malicious encoding. The registers containing these malicious words were labeled as attack LVL3. During the examination of this lengthy dictionary, we identified bursts of requests with an obvious intent of scanning. We labeled these requests as attack LVL2. Similarly, we also identified one burst with identical requests (i.e., referring to the same resource), whose URLs were labeled as attack with a new confidence level LVL4. The identification of these malicious requests was challenging because malevolent entries were interleaved with normal registers, and because each individual request seemed apparently non-malicious. As such, LVL4 attacks are the least sound category. We also identified that bursts associated with scans produced predominantly 300-type responses. Due to the impossibility of detecting all bursts in the entries and the fact that the majority of these scans produced a 300-type response, we finally opted for including only registers with response code <300 in the clean dataset. – Value Dictionary. A pattern analysis allowed us to identify three legitimate prevalent patterns (sometimes with hexadecimal characters). A delimiterbased search allowed us to identify approximately 1000 (mostly legitimate) words. We finally classified as LVL2 attack those URIs that contained words with the character ’ at the end or in combination with newline. Finally, given the modest number of anomalies discovered in the words of this dictionary, we analyzed the combinations key=value extracted from the query parts of the URI (a total of 20,039 unique combinations). The results of this inspection allowed us to identify three combinations as OSS levels 2 and 4, and three key-value pairs as OOS2 (one with 1037 instances). The main findings of the vocabulary inspection (S3) can be summarized as: • several URIs did not comply with standard syntax or made an inadequate use of percent encoding. Affected registers were classified as OOS1 and OOS2 respectively. • several path segments included attacks based on the multiple encoding of the character ’%’ (%252525...) or Table 4 Dictionaries formed from the dataset Segment Initial (After S2) Final (After S3) Size # Words Size # Words path 51,130 275,392,782 26,488 259,515,518 key 1971 6,823,291 1578 6,689,066 value 28,513 4,732,008 27,559 4,637,691 2 https:// github. com/ uripa rser/ uripa rser. Page 9 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 malicious encoding. Affected entries were classified as attack (LVL2 or LVL3). • detection of bursts presumably associated with DoS and scanning attacks (LVL2 or LVL4). • Detection of a significant number (more than 55k) of new indubitable attacks unnoticed by the signaturebased detection from S1 (LVL1). Table5 summarizes the result of this phase. We finally established four categories of attacks (LVL1 to LVL4), and four categories of OOS (OOS1 to OOS4) to get a finer-grain labeling of the URIs that exhibit issues in their syntax or representation. Moreover, our vocabulary analysis has advised to suppress some entries based on their associated response code in order to attain a more quality clean dataset, since a significant part of the registers associated to ResponseCode (RC) 300 were deemed as either attack or OOS. It is worth noting that stage S3 added 74,422 new attacks to those identified after S23 (an additional 27%) which highlights the importance of this phase in the sanitization process. Dataset description Following the sanitization and labeling process described above, this section details the files and structure of Biblio-US17, which can be downloaded from Díaz Verdejo etal. (2023). A brief summary of this section is also available at Diaz-Verdejo etal. (2024b). Files andformatting The dataset comprises several text files organized in directories. Each line in these files refers to a record through a record identifier RID which relates it to its source log file and can be taken as a temporal reference for the record, indicating the date and order of collection. The RID syntax is as follows: ‘[’MM-DD-Fnnnnnn’]’ where MM-DD stands for the month and day of the original log file, F stands for the web service type (A for HTTP or S for HTTPS), and nnnnnn is an integer that indicates the order of the request in the original log file. An example RID would be [02-18-A001234], which would correspond to the request line 1234 from the original file of the HTTP service dated on February 18. Files are arranged in directories and most filenames have a common format: biblio-2017-mmdd. <ext> , where mm and dd refer to the month and day of the original log file, and the extension <ext> determines the directory and content format as shown in Table6. Table 5 Number of URIs labeled during vocabulary analysis (S3) Attack (74,422 registers) Out of specification (10,352 registers) Label Description # Registers Label Description # Registers LVL1 Undubituous 55,803 OOS1 Out of specification (RFC3986) 169 LVL2 Context dependent/scanning 8794 OOS2 Extended chars codification errors/not allowed chars 2021 LVL3 Codification attack 5515 OOS3 Initial ’//’ 6567 LVL4 Burst attack/DoS 4310 OOS4 Semantic errors/others 1595 Table 6 Directories, data format and file extensions of Biblio‑US17 Dir Fmt Ext Content RAW Raw .raw Full dataset: All valid registers after anonimization LABELS Label .lbl Labels assigned during sanitization, indexed by [RID] SID SID .sid Alerts generated by signature IDSs, indexed by [RID] CLEAN Raw .cl Registers considered clean indexed by [RID] ATTACK Raw .att Registers considered attack (only LVL1). PARTITIONS Raw .train .test .val Suggested partitions of clean requests for use in train/ test/validation. AUX Various – Auxiliar files (rules, scripts, tools) 3 Most of them—55,498—correspond to the use of ’%252525.... Excluding this attack, S3 added 15,076 new attacks. Page 16 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 sometimes led to the revision of the URIs where the word is found) allowed us to indentify the following: • a total of 36 LVL2 attacks, • 25 URIs did not comply with the specification and were labeled as OOS2 as the lack of compliance was due to encoding issues • 28 URIs semantically wrong. These URIs had the string *QUERYSTRINGPII_REMOVED* as key, presumably due to some kind of filtering applied by the server over the traces. As a result, we added a new category to the OOS label, OOS4 which was used to mark these URIs. It is worth nothing that the analysis of the keys x1 and login allowed us to detect attacks. In both cases, they are contained in bursts of requests received in June 1st and June 6th. Further details Our search for problematic characters only produced positive results for ’;’ and ’=’ in three situations: • The key amp;amp;amp;amp;amp;opción, which ended up being labeled as OOS2. • q=node/add was encoded as Q%3Dnode/add, which was labeled as OOS4. • 24 URIs did not encoded the character ’=’ at the end of the value field associated with a couple of suspicious keys. We labeled them as OOS2. Searching key words have allowed as to detect some attacks of type 2 in the following cases: • Query ’login=cmd’. It is found repeatedly with small variations in the path addressed to specific locations used by wordpress, which led us to classify it as a scanning. • URIS presumably also associated with wordpress using the format /sites/wdn4.mu.xs/files// wp-admin/[PAGE].php?x1 Path dictionary We carried out several analysis geared toward the detection of different types of anomalies in the vocabularies. First, and as a result of the findings during S2, we suppressed from subsequent analysis 18,394 stings that contained %252525. The URIs that contained these strings were directly labeled as LVL2 attack. Systematic analysis: delimiters We searched and found 101 words with delimiter characters, which are inspected manually. Of these, 2 are associated with LVL1 attacks, 29 with coding problems (OOS2) and 49 with semantic errors (OOS4). Use of percent encoding We detected issues related with the incorrect use of percent encoding. These issues were related to an Fig. 4 Temporal evolution of the vocabularies learned Table 10 Vocabulary details for (presumably) file names by extension (after phase 1) Ext. #Words #Ocurrences .png 2334 20,548 545 .css 1710 6,529,799 .pdf 1704 332,695 .jpg 1467 2,556,541 .html 1452 11,979 .js 1324 8 983,781 .asp(?) 353 2371 .php(?) 321 18,005 .htm 254 2311 .xml 126 749,907 .gif 123 1,031,787 .ppt(?) 134 7918 .xls(?) 111 5141 .zip 90 1158 .swf 70 1161 .pps(?) 60 3167 others 381 27,279,872 Total 12,041 172,619,409 Page 17 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 extensive use of encoded characters inside the words learned giving rise to excessive word sizes or erroneous or impossible encoding (e.g., %EF%BF). Thus, we extracted and analyzed all words that included percent encoding (10,738 words). To facilitate manual inspection, we carried out an iterative procedure that consists of substituting (or marking if applies) the different encoding found starting by the longest ones (encoding of a character using UTF with 3 elements) until there is no more words with encoding characters. On each case, we classify the substituted character as either as (associated with) attack, out of specification, or admitted. In the two first cases, we filtered the word and the corresponding URIs from the clean dataset. The manual inspection of some of these URI evidences attacks based on the incorrect or malicious use of encoding. As such, are labeled as LVL3 such as we did with the attack %2525.... An example of one of these register would be the following: On the other hand, some encoded text gives rise to characters that, although correct from the the specification point of view, are semantically inadequate regarding the content of the web application. For example: where %c3ndice is decoded as Ãndice rather than índice, which does not make sense. As a result, we identified 43 encodings to be excluded (e.g., %EF%81%BD, %C2%A0) and 5 (%3A, %3E, %7C, %3C and %ED) required additional supervision. From those to be excluded, 22 are associated with attacks LVL3 and 21 characters semantically inadequate (OOS4). The encodings requiring further supervision have 34 different words associated. Most of them, 32 words, ended [01-01-A016709] GET /comunicacion/noticias/cat%C3%83%C6%92%C3%82%C2%A1l ogofama? page=2 HTTP/1.1" 200 22376 [01-03-A065330] GET /arquitectura/noticias/%c3ndice-jcr-2014 HTTP/1.0" 200 135982 up labeled as OOS4 after inspection. In total, we identified 5518 URIs of type LVL3 and 396 of type OOS4. Search for patterns We identified two patterns followed by several words: • ’css_([a-Z]|_|-|[0-9]){40,}.css’, found in 1612 words, and • ’js_([a-Z]|_|-|[0-9]){40,}.js’, found in 1238 words. Both patterns were considered normal after the inspection of a significant number of words. Filenames by extension We extracted the text strings that preceded certain file extensions. We identified 16 file extensions with more than 50 occurrences (see Table10) that added up a total of 8745 different words. The remaining extensions added 273 different words. We inspected these words grouped by extension seeking anomalies or patterns. We did not find anomalies or words that required further investigation, but in the case of the extensions .php, .zip y .asp, we did investigated in further detail. In the examination of the words and URIs that correspond to these extensions, we found some requests that could intend to scan since they constitute a sequence of requests that access to known locations from other portals or tools through variations in the path aimed to confirm the existence of a resource. In some cases, they seem to be geared toward installing a tool or add-on. After confirming with the administrators that these requests Page 18 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 did not come from them, and due to the difficulty of determining all the burst intended to scan, we decided to label these burst of requests as LVL2. We also identified bursts of identical (valid) requests, which could be traced back to DoS attacks. In this case, we created a new category, LVL4, to label these sequence of identical requests. Like in the case of scanning requests, it was hard for us to identify all these bursts because they were frequently interleaved with normal traffic. Additional Details Fig. 5 Examples of LVL1 attack Fig. 6 Examples of LVL1 attack Page 19 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 We have identified 835 requests with queries that contained z3=[alfanumeric chars]%3d%3d and z4=z4=Lw%3d%3d that are included in bursts and are apparently associated to software updates. Appendix B: Examples ofattacks andOOS URIs Examples ofLVL1 attack identified duringS2 See Fig.5. Examples ofattacks manually identified duringS3 (LVL1) See Fig.6. Examples ofLVL2 attacks manually identified duringS3 See Fig.7. Example ofaburst attack See Fig.8. Appendix C: Main challenges faced Below, we outline some of the significant challenges we encountered and how we addressed them, providing practical guidance for the research community in applying our methodology to create new datasets from realworld data. Some of these aspects have been discussed in more detail throughout this article in the corresponding sections or appendices. • Data Volume: The large number of text records and successive processing phases necessitated the use of a relational database. To uniquely identify records and facilitate indexing, each collected URI was assigned a unique identifier. Processing was conducted according to labels associated with various properties (e.g., detection results from S-IDS), using different database tables. Fig. 7 Examples of LVL2 attack Fig. 8 Example of a burst attack Page 20 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 • Detection of Known Attacks: Two major challenges arose: some signature-based IDS require network traces as input, necessitating request generation and capture, which is computationally expensive and prone to artifacts. Additionally, some tools lack correlation between outputs and inputs, complicating the identification of alert-triggering records. We addressed these by developing InspectorLog, a tool that processes log files and implements detection techniques from online tools, ensuring clear mapping between outputs and input records via unique identifiers. • Preprocessing and Conditioning: Although data was provided by the operator from their log files, it required minor corrections due to errors such as incomplete lines, improper field delimitations, and inappropriate URI values. The database and unique identifiers were crucial for reviewing affected records without reprocessing all data. • Anonymization: Privacy concerns required extensive anonymization, guided by an external audit. We implemented an algorithm that modifies sensitive data while preserving features useful for detection (e.g., special characters, string frequencies). The process was intensive, requiring multiple iterations and independent verification before approval by the University’s legal and security teams. Evaluating the impact of anonymization on the results is also necessary. • Realism: We minimized modifications to preserve the realism of the data. All valid records are provided in the raw dataset, regardless of categorization. Despite normalization by Apache preprocessors, some acquired data lacked expected characteristics. We preserved URIs as stored in the trace file, reflecting the actual server-processed requests. Records with response codes outside the 200 range were excluded from the clean dataset but retained in the raw dataset. • Revision of Alerts: TP/FP Classification (Phase S2 of Labeling): We identified signature-based IDS rules generating TP or FP and analyzed all detected URIs. SID-based analysis facilitated this process by revealing the detection rationale. • Manual Supervision (Phase S3): The most challenging task was the manual supervision of the vocabulary and associated URIs. Supervising +47 M of individual URIs was impossible, but overseeing their constituent strings (around 80k) was feasible. We employed various tools, utilities, and Python scripts to assist with the analysis, search, and comparison of strings. Histograms and word/pattern lists, such as attack-associated strings (e.g., /etc/password) and valid strings (e.g., 32 consecutive hexadecimal characters followed by .css), were generated to expedite the process. In some cases, OSINT was needed to determine the nature of certain URIs. The effort paid off, as it significantly impacted the clean dataset by detecting more than 74k new attacks with various confidence levels. In summary, this work was a challenging task requiring significant expertise and resources. Acknowledgements We would like to thank the library of the University of Sevilla and, particularly, to Claudio Arjona, and professors Rafael Martinez Gasca and Angel Luis Varela for their support in this work, reviewing the anonymization process and help‑ ing to achieve administrative permissions. We also extend our gratitude to the anonymous reviewers. Their feedback has been important in enhancing this paper. Author Contributions JD‑V has conducted the research and has largely taken the work of sanitiza‑ tion and organization of the dataset. He also has reviewed and corrected the paper and written the first draft. RE has co‑conducted the research and has been active in organizing the paper content, supervising the anonymiza‑ tion job and the empirical study in section "Dataset description". He has also carried out and written the illustrative case study and revised the paper. AE has participated in the research and is the main writer of most sections of the paper. He has also supported some technical aspects during the empirical study. JM‑C has carried out the anonymization process, provided support dur‑ ing the sanitization, and has placed the dataset in a public repository. GM has provided technical support with the infrastructure used in the experiments and assisted in the sanitization process. Funding This work has been partly funded by the research Grant PID2020‑115199RB‑ I00 provided by the Spanish Ministry of Industry under the contract MICIN/ AEI/10.13039 /501100011033. Availibility of data and materials Labeled HTTP requests dataset: Dataset Biblio‑US17. https:// doi. org/ 10. 12795/ 11441/ 148254 (2023). [Online; accessed 29‑Jul‑2023]. Declarations Competing interests The authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non‑financial interest in the subject matter or materials discussed in this manuscript. Author details 1 Department of Signal Theory, Telematics and Communications, University of Granada, Granada, Spain. 2 Department of Telematics Engineering, University of Seville, Seville, Spain. Received: 7 March 2024 Accepted: 4 November 2024 References (2023) Cyber Security Statistics The Ultimate List Of Stats Data, & Trends For 2023 . https:// purpl esec. us/ resou rces/ cyber‑ secur ity‑ stati stics/# ZeroD ay. Accessed 19‑May‑2023 Abad S, Gholamy H, Aslani M (2023) Classification of malicious urls using machine learning. Sensors 23(18):7760 Agarwal N, Hussain SZ (2018) A closer look at intrusion detection system for web applications. Secur Commun Netw 2018 Page 21 of 21 Díaz‑Verdejoetal. Cybersecurity (2025) 8:38 Ageed ZS, Zeebaree SR, Sadeeq MM et al (2021) A survey of data mining implementation in smart city applications. Qubahan Acad J 1(2):91–99 Apps SC (2021) Cyberattacks 2021: phishing, ransomware & data breach sta‑ tistics from the last year. https:// spann ing. com/ blog/ cyber attac ks‑ 2021‑ phish ing‑ ranso mware‑ data‑ breach‑ stati stics/ Ben Martin CA, Denis Sinegubko RE, Pellegrini T (2022) 2022 website threat research report. https:// sucuri. net/ repor ts/. Accessed 28‑Aug‑2024 Betarte G, Giménez E, Martínez R et al (2018) Improving web application firewalls through anomaly detection. In: 2018 17th IEEE international con‑ ference on machine learning and applications (ICMLA). IEEE, pp 779–784 Brugger ST, Chow J (2007) An assessment of the darpa ids evaluation dataset using snort. UCDAVIS Dept Comput Sci 1:22 Chodak G, Suchacka G, Chawla Y (2020) EClog: HTTP‑level e‑commerce data based on server access logs for an online store Cretu GF, Stavrou A, Locasto ME et al (2008) Casting out demons: sanitizing training data for anomaly sensors. In: 2008 IEEE symposium on security and privacy (sp 2008). IEEE, pp 81–95 Díaz‑Verdejo JE, Estepa A, Estepa R et al (2020) A methodology for conducting efficient sanitization of http training datasets. Future Gener Comput Syst 109:67–82 Díaz‑Verdejo J, Muñoz‑Calle J, Estepa Alonso A et al (2022) On the detection capabilities of signature‑based intrusion detection systems in the context of web attacks. Appl Sci 12(2):852 Díaz Verdejo J, Estepa Alonso RM, Estepa Alonso AJ, Muñoz Calle FJ et al (2023) Labeled HTTP requests dataset: dataset Biblio‑US17. https:// doi. org/ 10. 12795/ 11441/ 148254. Accessed 29‑Jul‑2023 Diaz‑Verdejo J, Estepa R, Alonso AE et al (2024a) Insights into anomaly‑based intrusion detection systems usability. a case study using real http requests. In: European interdisciplinary cybersecurity conference, pp 82–89 Diaz‑Verdejo J, Estepa Alonso R, Estepa Alonso A et al (2024b) Biblio‑us17: a labeled real url dataset for anomaly‑based intrusion detection systems development. In: European interdisciplinary cybersecurity conference, pp 217–218 Díaz‑Verdejo J, Muñoz‑Calle J, Estepa Alonso R et al (2024c) Inspectorlog: a new tool for offline attack detection over web log trace files. In: Proceed‑ ings of the 21st international conference on security and cryptography— SECRYPT, INSTICC. SciTePress, pp 692–697. https:// doi. org/ 10. 5220/ 00127 64000 003767 Estepa R, Díaz‑Verdejo JE, Estepa A et al (2020) How much training data is enough? A case study for http anomaly‑based intrusion detection. IEEE Access 8:44410–44425 Etopen (emerging threats open) rule sets (2023) https:// commu nity. emerg ingth reats. net/. Accessed 17‑Nov‑2023 Forbes (2023) Top Website Statistics For 2023. https:// www. forbes. com/ advis or/ busin ess/ softw are/ websi te‑ stati stics/. Accessed 19‑May‑2023 Fung BC, Wang K, Chen R et al (2010) Privacy‑preserving data publishing: a survey of recent developments. ACM Comput Surv 42(4):1–53 Gao Y, Ma Y, Li D (2017) Anomaly detection of malicious users’ behaviors for web applications based on web logs. In: 2017 IEEE 17th international conference on communication technology (ICCT). IEEE, pp 1352–1355 Garcia‑Teodoro P, Diaz‑Verdejo J, Maciá‑Fernández G et al (2009) Anomaly‑ based network intrusion detection: techniques, systems and challenges. Comput Secur 28(1–2):18–28 Hajj S, El Sibai R, Bou Abdo J et al (2021) Anomaly‑based intrusion detection systems: the requirements, methods, measurements, and datasets. Trans Emerg Telecommun Technol 32(4):e4240 Hjerppe K, Ruohonen J, Leppänen V (2019) The general data protection regulation: requirements, architectures, and constraints. In: 2019 IEEE 27th international requirements engineering conference (RE). IEEE, pp 265–275 Husák M, Apruzzese G, Yang SJ et al (2021) Towards an efficient detection of pivoting activity. In: 2021 IFIP/IEEE international symposium on inte‑ grated network management (IM). IEEE, pp 980–985 Ibrahim IM, Ameen SY, Yasin HM et al (2021) Web server performance improve‑ ment using dynamic load balancing techniques: a review. System 19:21 Khraisat A, Gondal I, Vamplew P et al (2019) Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecurity 2(1):1–22 Landauer M, Skopik F, Wurzenberger M et al (2020) Have it your way: generat‑ ing customized log datasets with a model‑driven simulation testbed. IEEE Trans Reliab 70(1):402–415 Landauer M, Skopik F, Höld G et al (2022) A user and entity behavior analytics log data set for anomaly detection in cloud computing. In: 2022 IEEE international conference on big data (big data). IEEE, pp 4285–4294 Lara‑Romero A, Ternero‑Muniz J, Estepa R et al (2023) Http cyberattacks detec‑ tion through automatic signature generation in multi‑site iot deploy‑ ments. In: Proceedings of the 2023 European interdisciplinary cybersecu‑ rity conference. ACM, pp 65–70 Mahoney MV, Chan PK (2003) An analysis of the 1999 darpa/lincoln laboratory evaluation data for network anomaly detection. In: Recent advances in intrusion detection: 6th international symposium, RAID 2003, Pittsburgh, PA, USA, September 8–10, 2003. Proceedings 6, Springer, pp 220–237 Nemesida rules (2023) https:// rlinfo. nemes ida‑ secur ity. com/. Accessed 17‑Nov‑2023 Owasp modsecurity core rule set (crs) (2023), https:// github. com/ corer uleset/ corer uleset/ relea ses/. Accessed 17‑Nov‑2023 Pałka D, Zachara M (2011) Learning web application firewall‑benefits and caveats. In: Availability, reliability and security for business, enterprise and health information systems: IFIP WG 8.4/8.9 international cross domain conference and workshop, ARES 2011, Vienna, Austria, August 22‑26, 2011. Proceedings 6, Springer, pp 295–308 Paxson V (2004) Strategies for sound internet measurement. In: Proceedings of the 4th ACM SIGCOMM conference on internet measurement, pp 263–271 Reiter MK, Rubin AD (1998) Crowds: anonymity for web transactions. ACM Trans Inf Syst Secur 1(1):66–92 Salazar‑Hernández R, Díaz‑Verdejo JE (2010) Anonimización de payloads para el desarrollo de aids basados en protocolos. Actas de las IX Jornadas de Ingeniería Telemática (JITEL 2010) pp 260–267 Saxena A, Arora A, Saxena S et al (2022) Detection of web attacks using machine learning based url classification techniques. In: 2022 2nd inter‑ national conference on intelligent technologies (CONIT). IEEE, pp 1–13 Schulz H, Okanović D, van Hoorn A et al (2021) Context‑tailored workload model generation for continuous representative load testing. In: Pro‑ ceedings of the ACM/SPEC international conference on performance engineering, pp 21–32 Sharafaldin I, Lashkari AH, Ghorbani AA (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 1:108–116 Talos (vrt group) community rules, (2023). https:// www. snort. org/ talos. Accessed 17‑Nov‑2023 Uetz R, Hemminghaus C, Hackländer L et al (2021) Reproducible and adapt‑ able log data generation for sound cybersecurity experiments. In: Annual computer security applications conference, pp 690–705 Viegas EK, Santin AO, Oliveira LS (2017) Toward a reliable anomaly‑based intru‑ sion detection in real‑world environments. Comput Netw 127:200–216 Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in pub‑ lished maps and institutional affiliations.