Exploring Collaboration Models for Supporting Ideation with AI Agents - Supplementary Material

Exploring Collaboration Models for Supporting Ideation with AI Agents Supplementary Material Jérémy La Scala (https://orcid.org/0000-0002-8057-7787) Denis Gillet (https://orcid.org/0000-0002-2570-929X) École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland DOI: 10.5281/zenodo.17200232 Supplementary material complementing the report on the workshop titled "Exploring Collaboration Models for Supporting Ideation with AI Agents", organized at the 53rd SEFI 2025 Annual Conference, in Tampere, Finland, on the 17th of September 2025. Introduction survey and answers Below are the two questions that were asked in the introduction to the workshop using Framaforms. The survey was followed by a discussion on these questions. Question 1: In your opinion, what would be the main benefits of having AI-generated feedback in collaborative ideation? •- Have access to instant feedback (e.g. large class where it's not possible to have immediate feedback from TA) •Get feedback quickly. AI will respond more easily (students might be reluctant). Fostering AI Critical workhelp students understand how to work better with AI. AI doesn't get bored by giving the same feedback •Objective and quick (compared to teacher feedback) Question 2: In your opinion, what would be the main risks associated with AI-generated feedback? •- Potentially, users which focus too much on the AI feedback (which is not always correct). It can lead to a loss of creativity. •Students might prefer AI and avoid human feedback and interaction Students become overreliant Risk of hallucinations, poor feedback Sycophantic AI (unless you ask it to be critical) J. La Scala, D. Gillet 1 25/09/2025 Exploring Collaboration Models for Supporting Ideation with AI Agents - Supplementary Material Supplementary Material •The quality and interpretation of the idea sketches. Can it take into account the novice level, how define as teacher your expectations (difference between year-1 vs year-4 student.) Prompts Templates Template 1: Iteration enhancing Problem statement: {{ problem_statement }} Participant's idea: {{ current_response }} {% if previous_responses.size > 0 %} Previous ideas and feedback: {% for item in previous_responses %} - Idea {{ forloop.index }}: {{ item.response }} Feedback: {{ item.feedback }} {% endfor %} {% else %} No previous ideas have been submitted yet. {% endif %} Please provide constructive feedback on the participant's idea with a focus on its relevance to science and engineering education in the future. Use the following structure: 1. Strengths: [<DESCRIBE WHAT MAKES STRONG IDEAS>] 2. Opportunities and questions: [suggest one or two ways the idea could be further developed, and pose open-ended questions that help the participant elaborate, clarify assumptions, or explore new angles] {% if previous_responses.size > 0 %} 3. Relation to previous ideas: [explain how this idea complements, improves, or differs from earlier ideas and feedback] {% endif %} Template 2: Idea compliance Problem statement: {{ problem_statement }} Participant's proposed topic: {{ current_response }} {% if previous_responses.size > 0 %} Previous ideas and feedback: J. La Scala, D. Gillet 2 25/09/2025 Exploring Collaboration Models for Supporting Ideation with AI Agents - Supplementary Material Supplementary Material {% for item in previous_responses %} - Idea {{ forloop.index }}: {{ item.response }} Feedback: {{ item.feedback }} {% endfor %} {% else %} No previous ideas have been submitted yet. {% endif %} Please provide constructive feedback on the proposed poster ideas with a focus on its fit with <DESCRIBE YOUR TOPIC>. In your feedback, use the following structure: 1. **Fit with <TOPIC>:** - <DESCRIBE EACH CRITERIA> - Point out any missing or underdeveloped elements. 2. **Suggestions for strengthening the idea:** - Suggest concrete ways the participant could refine the topic to better address the definition <TOPIC>. - Offer one or two open questions that would help them expand or clarify their focus. {% if previous_responses.size > 0 %} 3. **Relation to previous ideas:** - Point out how this idea improves on earlier ones. - Note if any important aspects from earlier feedback have not yet been integrated. {% endif %} Prompts designed by the participants Group 1 Problem statement: {{ problem_statement }} Participant's idea: {{ current_response }} {% if previous_responses.size > 0 %} Previous ideas and feedback: {% for item in previous_responses %} - Idea {{ forloop.index }}: {{ item.response }} Feedback: {{ item.feedback }} {% endfor %} J. La Scala, D. Gillet 3 25/09/2025 Exploring Collaboration Models for Supporting Ideation with AI Agents - Supplementary Material Supplementary Material {% else %} No previous ideas have been submitted yet. {% endif %} Please provide constructive feedback on the participant's idea with a focus on its relevance to energy usage and sustainability. Use the following structure: 1. Strengths: [<DESCRIBE WHAT MAKES STRONG IDEAS>] 2. Opportunities and questions: [suggest one or two ways the idea could be further developed, and pose open-ended questions that help the participant elaborate, clarify assumptions, or explore new angles] {% if previous_responses.size > 0 %} 3. Relation to previous ideas: [explain how this idea complements, improves, or differs from earlier ideas and feedback] 4. Make sure the feedback is short, with a maximum of thirty words. 5. Give a picture related to the feedback. 6. If student's input is shorter than ten words, reply: Please use a full sentence to describe your idea. 7. Give all output in Dutch. {% endif %} Group 2 Problem statement: {{ problem_statement }} Participant's idea: {{ current_response }} {% if previous_responses.size > 0 %} Previous ideas and feedback: {% for item in previous_responses %} - Idea {{ forloop.index }}: {{ item.response }} Feedback: {{ item.feedback }} {% endfor %} {% else %} No previous ideas have been submitted yet. {% endif %} Please provide constructive feedback on the participant's idea with a focus on its relevance to science and engineering education in the future. Use the following structure: 1. Strengths: [a strong idea should be scientifically proven] 2. Opportunities and questions: [suggest 3 further ideas, based on the initial idea of the user] {% if previous_responses.size > 0 %} J. La Scala, D. Gillet 4 25/09/2025 Exploring Collaboration Models for Supporting Ideation with AI Agents - Supplementary Material Supplementary Material 3. Relation to previous ideas: [explain how this idea complements, improves, or differs from earlier ideas and feedback] {% endif %} Group 3 Problem statement: {{ problem_statement }} Participant's idea: {{ current_response }} {% if previous_responses.size > 0 %} Previous ideas and feedback: {% for item in previous_responses %} - Idea {{ forloop.index }}: {{ item.response }} Feedback: {{ item.feedback }} {% endfor %} {% else %} No previous ideas have been submitted yet. {% endif %} Please provide constructive feedback on the participant's idea with a focus on its relevance to science and engineering education in the future. Use the following structure: 1. Strengths: [<DESCRIBE WHAT MAKES STRONG IDEAS>] 2. Opportunities and questions: [suggest one or two ways the idea could be further developed, and pose open-ended questions that help the participant elaborate, clarify assumptions, or explore new angles] {% if previous_responses.size > 0 %} 3. Relation to previous ideas: [explain how this idea complements, improves, or differs from earlier ideas and feedback] {% endif %} 4. Brainstorm Raw Discussion Notes Feedback from the interactive demo •Principle is interesting. Feedback is promising. •Two participant mentioned the benefit of having a collaborative interaction with the AI. •AI feedback is too long, wordy •overly positive (sycophantic issue) J. La Scala, D. Gillet 5 25/09/2025 Exploring Collaboration Models for Supporting Ideation with AI Agents - Supplementary Material Supplementary Material Final open discussion •Preparing the activity (prompt design) was easier than expected •Flexible •threads and collaboration, building on each other ideas, distributed cognition •Participant mentioned that the worth is both in the tools that allow that, and the fact that it enables collaboration with the AI (here, as a feedback) •Benefit of having the possibility to define guidelines and prompt as a teacher. Participant mentioned it's not easy to do it without guidance/training •One participant mentioned that literature on education, facilitation is rich on how to give good feedback. The system could help teachers incorporate these guidelines in their prompt to obtain more effective feedback. •User Experience of configuring this system is tricky/not straightforward. •Tension between more flexible UX and benefit of simplified system with affordance that support focus on the task. •Finally, all participants agreed that feedback was overly positive. J. La Scala, D. Gillet 6 25/09/2025