Supplementary_Material_Survey_Interview

Supplementary Material – Survey and Interview Instruments Survey Questions The following questions and their official response options were presented to survey participants. 1. Q1. Consent: Do you agree to take part in this survey? • Yes • No 2. Q2. What is your age group? • 18–24 • 25–34 • 35–44 • 45–54 • 55 or older 3. Q3. What is your highest level of education? • Bachelor • Master • PhD • Other 4. Q4. How would you rate your academic or technical background in Artificial Intelligence (AI)? • None (No background at all) • Basic (Heard of AI but haven’t studied it) • Intermediate (Studied or explored AI for learning or work) • Advanced (Academic or professional experience in AI) 5. Q5. How would you rate your academic or technical background in Natural Language Processing (NLP)? • None (No background at all) • Basic (Heard of NLP but haven’t studied it) • Intermediate (Studied or explored NLP for learning or work) • Advanced (Academic or professional experience in NLP) 6. Q6. Have you previously interacted with AI-based systems? • Yes • No • Not sure 7. Q7. Do you usually read user reviews before choosing healthcare services? • Yes • No • Sometimes 8. Q8. What is your preferred approach when reading patient reviews? • Careful reading • Skimming • Scanning for keywords only 9. Q9. On average, how much time do you spend reading patients’ reviews before making a healthcare-related decision? • Less than 1 minute • 1–3 minutes • 4–6 minutes • 7–10 minutes • More than 10 minutes 10. Q10. How helpful do you find reading or checking online patient reviews when making healthcare-related decisions? • 1(Not helpful at all) • 2 • 3 • 4 • 5 (Extremely helpful) 11. Q11. To what extent do you agree with the following statements about the usefulness of an AI system that analyses and summarises patient reviews, and explains the reasons behind its conclusions, in supporting healthcare decision-making? A. Saves time by expediting the review of large volumes of patient feedback. • Strongly disagree • Disagree • Neutral • Agree • Strongly agree B. Facilitates access to essential information without the need to read full reviews. • Strongly disagree • Disagree • Neutral • Agree • Strongly agree C. Supports decision-making based on priorities by organising comments according to specific aspects (such as waiting time, cleanliness, cost). • Strongly disagree • Disagree • Neutral • Agree • Strongly agree D. Reduces bias by providing standardised analysis of review content. • Strongly disagree • Disagree • Neutral • Agree • Strongly agree E. Enhances decision reliability through accurate summaries and clear explanations. • Strongly disagree • Disagree • Neutral • Agree • Strongly agree 12. Q12. How important is it for you to understand why a review was classified as positive or negative? • 1 (Not important at all) • 2 • 3 • 4 • 5 (Extremely important) 13. Q13. To what extent would knowing the reason behind the review’s classification increase your trust in the system? • 1 (Not at all) • 2 • 3 • 4 • 5 (To a great extent) 14. Q14. Which of the following explanation formats would best help you understand the system’s classification results? • Brief text • Key words • Graphical elements • Mixed 15. Q15. Have you ever used any applications or websites that analyse or explain user reviews? • Yes • No 16. Q16. If you answered 'Yes' to the previous question, please provide further details below: (Open-ended question – participants provided free-text responses) 17. Q17. What aspects should the proposed system have to make it useful or appealing to you? (Open-ended question – participants provided free-text responses) 18. Q18. What aspects of the system might discourage you from using it or make you hesitant to rely on it? (Open-ended question – participants provided free-text responses) 19. Q19. Would you be willing to be contacted for a short follow-up interview to further discuss your views on AI-based systems for healthcare reviews? • Yes, I’m happy to be contacted • No, I’d prefer not to be contacted 20. Q20. If yes, please leave your email address (optional): 21. Q21. Please use the space below to share any additional thoughts about the system or your experience with this survey (optional): (Open-ended question – participants provided free-text responses) Interview Questions The interview consisted of the following four open-ended questions: 1. Q1. What are the main technical challenges you have personally faced when developing or using explainable AI (XAI) systems? (Open-ended question – participants provided free-text responses) 2. Q2. What strategies or approaches do you suggest for adapting explanation methods in XAI systems so that they meet the needs and understanding of both technical and nontechnical users? (Open-ended question – participants provided free-text responses) 3. Q3. Which explanation techniques do you find most interpretable or trustworthy, and why? (Open-ended question – participants provided free-text responses) 4. Q4. Do you foresee any performance or scalability challenges with a multi-layered XAI approach, that starting with an initial explanation generated by a standard method (e.g., LIME, SHAP, or similar) and refining it via external post-processing? (Open-ended question – participants provided free-text responses)