Dynamic Agent Generation for Self-Adaptive Root Cause Analysis

Appendix – Dynamic Agent Generation for Self-Adaptive Root Cause Analysis SEAMS/ICSE 2026 Submission – Artifact Appendix 1 Overview This appendix accompanies the paper “Dynamic Agent Generation for Self-Adaptive Root Cause Analysis”, submitted to SEAMS 2026. It provides extended experimental results, ablation tables, parameter analyses, and dataset statistics. All scripts, and raw results are available in our public replication package: https://doi.org/10.5281/zenodo.17446399. Example Query and Buildspec Query Example. On March 4, 2021, between 18:30 and 19:00, a failure occurred. However, the root cause component, the exact time of the root cause occurrence, and the underlying reason for the failure are currently unknown. You are tasked with identifying the root cause component, the root cause occurrence datetime, and the root cause reason. Listing 1: Example Buildspec { "task_type ":"task_ 7", "date":"2021-03-04", "filename_date":"2021_03_04", "failure_time_range": { "start ":"18:30:00", "end":"19:00:00" }, "failure_time_range_ts": { "start ":1614853800, "end":1614855600 }, "failures_detected":1, "uncertainty": { "root_cause_time":"unknown", "root_cause_component":"unknown", "root_cause_reason":"unknown" }, "objective ":"Identify the root cause component , the exact root cause datetime and reason for the failure ", "filename_date_directory":"/ agentfactory / data /Bank / telemetry /2021_03_04", "absolute_log_file":"/ agentfactory / data / Bank / telemetry /2021_03_04/ log/ log_service.csv", "absolute_trace_file":"/ agentfactory / data /Bank / telemetry /2021_03_04/ trace / trace_span . csv ", "absolute_metrics_file":"/ agentfactory / data / Bank / telemetry /2021_03_04/metric /metric_container.csv" } 1 Table 1: Accuracy comparison across OpenRCA datasets–Telecom, Bank, Market Max. Agents Method Telecom Bank Market Partial Correct Partial Correct Partial Correct Partial Correct N/A RCA-Agent(Claude 3.5) N/A N/A N/A N/A N/A N/A 11.34 17.31 N/A RCA-Agent(GPT-4o) N/A N/A N/A N/A N/A N/A 8.96 17.96 N/A RCA-Agent(Gemini1.5 Pro) N/A N/A N/A N/A N/A N/A 2.69 6.87 N/A RCA-Agent(Llama 3.1 Instr.) N/A N/A N/A N/A N/A N/A 3.28 5.65 5 Claude 3.5 7.42 ±2.41 8.36 ±2.58 9.01 ±2.72 6.58 ±2.13 7.94 ±2.46 8.62 ±2.61 8.12 ±2.53 7.85 ±2.44 GPT-4o 6.85 ±2.27 8.11 ±2.49 8.78 ±2.65 6.24 ±2.08 7.63 ±2.42 8.21 ±2.56 7.75 ±2.45 7.52 ±2.38 Gemini 1.5 Pro 5.97 ±2.04 7.22 ±2.31 7.81 ±2.48 5.36 ±1.96 6.48 ±2.22 7.09 ±2.37 6.75 ±2.16 6.56 ±2.21 Mistral Large 2 3.54 ±1.88 4.86 ±2.07 5.12 ±2.14 3.11 ±1.74 4.25 ±1.96 4.73 ±2.05 4.30 ±1.99 4.23 ±1.95 Llama 70B 2.97 ±1.73 3.75 ±1.89 4.16 ±2.01 2.68 ±1.62 3.44 ±1.83 3.97 ±1.92 3.52 ±1.79 3.47 ±1.81 Llama 8B 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 Mistral 7B 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 0.00 ±0.00 10 Claude 3.5 35.41 ±2.12 29.86 ±1.73 33.25 ±2.05 27.63 ±2.05 32.44 ±2.41 29.18 ±2.27 33.70 ±1.97 28.89 ±1.82 GPT-4o 31.78 ±1.66 34.17 ±1.88 36.12 ±1.99 30.54 ±2.23 28.79 ±2.14 31.07 ±2.34 32.23 ±1.92 31.93 ±2.15 Gemini 1.5 Pro 28.92 ±1.89 27.51 ±1.67 30.08 ±1.84 22.47 ±1.74 25.36 ±1.88 26.92 ±2.16 28.12 ±1.87 25.63 ±1.86 Mistral Large 2 33.07 ±1.97 30.94 ±1.91 32.66 ±1.95 24.62 ±2.18 23.41 ±2.10 22.81 ±2.11 29.71 ±2.01 26.12 ±2.07 Llama 70B 26.38 ±1.52 25.03 ±1.59 27.47 ±1.63 19.85 ±1.91 21.64 ±2.02 20.54 ±1.97 25.16 ±1.72 21.81 ±1.83 Llama 8B 0.44 ±0.12 0.53 ±0.14 0.61 ±0.15 0.51 ±0.13 0.67 ±0.16 0.42 ±0.11 0.57 ±0.14 0.49 ±0.13 Mistral 7B 0.62 ±0.15 0.39 ±0.11 0.35 ±0.09 0.36 ±0.09 0.47 ±0.12 0.58 ±0.14 0.48 ±0.12 0.44 ±0.11 20 Claude 3.5 34.92 ±2.71 30.11 ±2.18 34.02 ±2.95 28.14 ±2.44 31.88 ±2.87 29.87 ±2.63 33.61 ±2.84 29.37 ±2.43 GPT-4o 32.43 ±2.54 33.28 ±2.33 35.76 ±2.81 31.08 ±2.69 29.21 ±2.25 30.92 ±2.77 32.47 ±2.53 31.76 ±2.60 Gemini 1.5 Pro 27.85 ±2.42 28.63 ±2.16 31.45 ±2.74 23.04 ±2.31 24.92 ±2.68 26.35 ±2.52 28.74 ±2.61 26.67 ±2.46 Mistral Large 2 34.12 ±2.85 29.83 ±2.29 31.88 ±2.79 25.17 ±2.47 22.96 ±2.41 23.64 ±2.63 29.65 ±2.68 26.21 ±2.46 Llama 70B 25.97 ±2.18 26.48 ±2.31 27.21 ±2.54 20.36 ±2.25 22.05 ±2.44 21.12 ±2.39 25.74 ±2.36 21.84 ±2.36 Llama 8B 0.53 ±0.21 0.49 ±0.19 0.57 ±0.23 0.47 ±0.18 0.59 ±0.22 0.51 ±0.20 0.56 ±0.20 0.49 ±0.19 Mistral 7B 0.48 ±0.19 0.57 ±0.24 0.43 ±0.18 0.55 ±0.23 0.41 ±0.17 0.46 ±0.21 0.44 ±0.18 0.49 ±0.22 (a) OpenRCA — Correct (Sequential) (b) OpenRCA — Correct (Asynchrone) Figure 1: Comparison of correct OpenRCA results for sequential (left) and asynchronous (rigth) execution mode. (a) OpenRCA — Partial (Sequential) (b) OpenRCA — Partial (Asynchrone) Figure 2: Comparison of partial OpenRCA results for sequential (left) and asynchronous (rigth) execution mode. 2 (a) Nezha-OB — PR@1 (b) Nezha-TT — PR@1 (c) Nezha-OB — PR@3 (d) Nezha-TT — PR@3 (e) Nezha-OB — PR@5 (f) Nezha-TT — PR@5 Figure 3: Comparison of precision for asynchronous execution modes of the Nezha-OB (left) and Nezha-TT (rigth). 3 (a) Nezha-OB — PR@1 (b) Nezha-TT — PR@1 (c) Nezha-OB — PR@3 (d) Nezha-TT — PR@3 (e) Nezha-OB — PR@5 (f) Nezha-TT — PR@5 Figure 4: Comparison of precision for sequential execution modes of the Nezha-OB (left) and Nezha-TT (rigth). 4 (a) Nezha-OB — PR@1 (b) Nezha-TT — PR@1 (c) Nezha-OB — PR@3 (d) Nezha-TT — PR@3 (e) Nezha-OB — PR@5 (f) Nezha-TT — PR@5 Figure 5: Precision@K vs number of adapted agents accross Nezha-OB (left) and Nezha-TT (right). (a) OpenRCA — Partial (b) OpenRCA — Correct Figure 6: Accuracy vs number of adapted agents OpenRCA Partial (left) and OpenRCA Correct (rigth). 5