Standards for transparent AI in Human Resource Management (TRANKI) – Research data

Standards for transparent AI in Human Resource Management (TRANKI) – Research data Katharina Simbeck*Yannick Kalff† HTW Berlin University of Applied Sciences Table of contents Disclaimer 2 Detailsontheinquiry............................................ 2 Detailsonthedata ............................................. 3 Using the data-set 3 Folderstructureandfiletree........................................ 3 Reading CSV, attaching variable names and labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Inspectingthedata-set ........................................... 4 Codebook 4 Items and indices 5 TechnologyAcceptanceModel ...................................... 5 AILiteracy.................................................. 6 Technophobia................................................ 7 Appendix 7 DashboardsExperiment1......................................... 7 DashboardsExperiment2......................................... 8 DashboardsExperiment3......................................... 9 Contact details 10 License 11 *Envelope[email protected] �https://orcid.org/0000-0001-6792-461X †Envelope[email protected] �https://orcid.org/0000-0003-1595-175X 1 Disclaimer The following data-set was collected in February 2025 for the research Project “TRANKI – Standards for transparent AI”, funded by the Hans-Böckler-Foundation 1 . The goal was to assess the effects of AI literacy on the interpretation of AI-enhanced user interfaces of HR software, and how explainable AI (XAI) elements affect interpretation results. The project’s research questions were: • What approaches at the user interface level help make AI systems in human resources transparent for users? • What differences in transparency requirements and perceptions exist between employees, managers, and HR experts? All items are included with their German labels and English translations. The translation process was automated, using LLM model mistral-medium-2508 , and proof-read by the authors. Please feel free to translate further as needed. The experiments presented the participants with mock-up screens of HR-software tools. The basis were real-life existing tools that advertised their AI components. In a first step, we researched tool vendors and screened their homepages for screenshots and demonstrators. Then, we recreated the user interfaces with slight derivations in two versions: a) baseline, b) with added XAI elements to explain the AI results. We used Moqups.2The dashboard screenshots are in the appendix. The experiments conducted each include an objective assessment of the correct or incorrect interpretation of information from an AI-supported dashboard with/without explainable AI elements. This data-set contains the transformed and evaluated items and accumulated points for correct answers. (Variables: Exp1_objscore_pos , Exp1_objscore_pre , Exp2_objscore_pos , Exp2_objscore_pre , Exp3_objscore_pos , Exp3_objscore_pre – where _pos indicates scores for XAI-enhanced dashboards, and _pre vanilla dashboards.) Details on the inquiry •Design: Experimental design, survey, cross-sectional study •Pretest: 15 participants in December 2024. •Sample: –Size: 𝑁 = 427 –Recruitment: ISO 20252:2019-certified sampling provider – Inclusion criteria: currently employed or employed within the last 12 months in the field of HR (any level) •Survey period: 2025-02-01 – 2025-02-28 •Ethics: Informed consent from participants before inquiry 1 URL: https://www.boeckler.de/de/suchergebnis-forschungsfoerderungsprojekte-detailseite-2732.htm?projekt=2022-797-2. (Grant No. 2022-797-2). 2We thank Thao Do for assisting us with the research and creation process of the experiments’ dashboards. 2 Details on the data We removed 17 implausible cases, retaining 𝑁 = 410 cases for analysis. Implausible cases demonstrated unrealistic relationships between age and work experience, for example: age 50 with 45 years of experience in HR, or outliers in reported income. We excluded participants where 𝑎𝑔𝑒 − 𝑤𝑜𝑟𝑘𝑒𝑥𝑝𝑒𝑟𝑖𝑒𝑛𝑐𝑒 was below 16 years, meaning any person who would have been younger than 16 years when they began working was excluded. Additionally, we removed income outliers below 300 € and above 10000 € per month. The provided data-set has been filtered accordingly and includes these 410 cases (cf. the following codebox). exclude <- df_o %>% filter(((2025 -Q6_4) -Q4_4) <16 |Q5_4 <300 |Q5_4 >10000)%>% pull(ID) df_o <- df_o %>% filter(!ID %in% exclude) The experimental group assignments for experiments 2 and 3 are: • IF: Important features (i.e., Feature Importance) • CF: What if (i.e., Counterfactuals) • MC: Model criteria Using the data-set Folder structure and file tree FOLDER-OPENTranki-data/ │ ├── FILE-ALTtranki_data.csv # CSV Data set ├── FILE-ALTvarnames.csv # Variable names German ├── FILE-ALTvarlabels.csv # Variable labels German ├── FILE-ALTvarnames_engl.csv # Variable names English ├── FILE-ALTvarlabels_engl.csv # Variable labels English │ ├── FILE-ALTLICENSE.md # License ├── FILE-ALTREADME.pdf # This file └── FILE-ALTCHANGELOG.md # Version history and changes Reading CSV, attaching variable names and labels Run the following R -code first to load the data and assign variable names and labels. Otherwise, use the provided SPSS file. 3 library(labelled) data <- read.csv("data/tranki_data.csv") # Please select: either varnames <- read.csv("data/varnames.csv")#for original German variable names varlabels <- read.csv("data/varlabels.csv")#for original German variable labels # or #varnames <- read.csv("data/varnames_engl.csv") #for translated English variable names #varlabels <- read.csv("data/varlabels_engl.csv") #for translated English variable labels # Add varnames for (i in 1:nrow(varnames)) { if (!is.na(varnames$variable_label[i])) { var_name <- as.character(varnames$variable_name[i]) if (var_name %in% names(data)) { attr(data[[var_name]], "label") <- varnames$variable_label[i] } } } # Add labels for (var in unique(varlabels$variable)) { if (var %in% names(data)) { var_labels <- varlabels[varlabels$variable == var, ] labels <- setNames(var_labels$value, var_labels$label) var_label <- attr(data[[var]], "label") data[[var]] <- set_value_labels(data[[var]], labels) if (!is.null(var_label) && is.null(attr(data[[var]], "label"))) { attr(data[[var]], "label") <- var_label } } } Inspecting the data-set Some example queries. print(data$Q1_1) summary(data$Q6_4) plot(data$Q4_4, data$Q5_4) Codebook To create a codebook, use the following syntax. 4 knitr::opts_chunk$set( warning = TRUE, message = TRUE, error = TRUE, echo = TRUE ) ggplot2::theme_set(ggplot2::theme_bw()) pander::panderOptions("table.split.table",Inf) library(codebook) library(ggplot2) library(summarytools) codebook <- dfSummary(data, varnumbers = FALSE, na.col = FALSE, style = "multiline", plain.ascii = FALSE, headings = TRUE, max.distinct.values = 5, tmp.img.dir = "./tmp") # Render the Codebook print(codebook, method = "render") Items and indices The questionnaire incorporates several third-party indices, particularly for assessing constructs related to the Technology Acceptance Model (TAM),AI literacy, and technophobia. The sections below evaluate the psychometric quality of these measures and provide references to the respective literature. Technology Acceptance Model Technology Acceptance Model follows the concept of Davies 1989 3 but with individually generated items for the four sub-indices attitude towards use (ATU),perceived usefulness (PU),perceived ease of use (PEOU), and external factors (EXT). Participants were presented with the respective items if they met either of the following criteria: 1. Their employer explicitly permitted the use of AI tools in work processes, or 2. They independently utilized AI tools in their work—regardless of employer consent. 3 Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319. https://doi.org/10.2307/249008 5 Table 1: Overview of the scale TAM Scale No. Items Cronbach’s 𝛼Mean (SD) 𝑚𝑒𝑎𝑛 (𝑟𝑖𝑡.𝑐𝑜𝑟𝑟 ) ATU 2 (3) 0.73 3.73 (1.25)0.58 PU 3 0.80 3.79 (0.96)0.64 PEOU 3 0.77 3.81 (0.89)0.60 EXT 3 0.70 3.48 (0.96)0.53 The item Q4_2r9 has to be inverted. However, the item statistics suggest that responses were ambiguous. Therefore, we decided to exclude it from the ATU sub-scale and further analysis. AI Literacy AI literacy was measured using the Scale for the Assessment of Non-Experts’ AI Literacy (SNAIL), developed by Laupichler et al. 4 The questionnaire items were drawn from the German translation provided in the appendix of Laupichler et al.5The original scale comprises 30 items across three dimensions: •Technical understanding (TU) •Critical appraisal (CA) •Practical application (PA) For this study, 15 items with the highest relevance to human resource (HR) management were selected to ensure contextual appropriateness. The index showed exceptional quality with high Cronbach’s 𝛼in all three sub-dimensions. Table 2: Overview of the scale AI Literacy Scale No. Items Cronbach’s 𝛼Mean (SD) 𝑚𝑒𝑎𝑛 (𝑟𝑖𝑡.𝑐𝑜𝑟𝑟 ) TU 5 0.92 2.85 (1.28)0.79 CA 5 0.90 3.29 (1.22)0.75 PA 5 0.89 3.10 (1.24)0.73 Further discussions of the quality of the index can be found in our publications.6 4 Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023). Development of the “Scale for the assessment of non-experts’ AI literacy” – An exploratory factor analysis. Computers in Human Behavior Reports, 12, 100338. https://doi.org/10.1016/j.chbr.2 023.100338 5Laupichler, M. C., Aster, A., Perschewski, J.-O., & Schleiss, J. (2023). Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment. Education Sciences, 13(10), 978. https: //doi.org/10.3390/educsci13100978 6 Kalff, Y., & Simbeck, K. (2025). Explained, yet misunderstood: How AI Literacy shapes HR Managers’ interpretation of User Interfaces in Recruiting Recommender Systems. In M. Kaya, T. Bogers, G. Bied, C. Johnson, & J.-J. Decorte (Hrsg.), Proceedings of the 5th Workshop on Recommender Systems for Human Resources (RecSys in HR 2025) (Version 1, Bd. 4046). CEUR. https://ceur-ws.org/Vol-4046/RecSysHR2025-paper_3.pdf 6 Technophobia The scale was originally developed by Sinkovics 7 to assess resistance toward emerging automated technologies, with a focus on the then-novel ATM machines. For this study, the scale was adapted to evaluate employees’ negative affect and apprehensions regarding the increasing integration of AI in HR workplace contexts. From the original 13 German-language items, six were selected based on their relevance to AI-specific anxieties in HR settings. Two items ( Q2_3r4 and Q2_3r5 ) require reverse scoring prior to analysis. The adapted scale demonstrated adequate internal consistency (Cronbach’s 𝛼 = .72 ), with a mean score of 𝑀 = 3.07 (𝑆𝐷 = 0.74). Appendix Dashboards Experiment 1 Figure 1: Baseline Figure 2: Variation 1: Important Features 7 Sinkovics, R. R. (2003). Technophobie. Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS).https://doi.org/10.6102/ZI S62 7 Dashboards Experiment 2 Figure 3: Baseline Figure 4: Variation 1: Important Features Figure 5: Variation 2: Counter Factuals Figure 6: Variation 3: Model criteria 8 Dashboards Experiment 3 Figure 7: Baseline 9