Who's Calling? - Assessing The Effectiveness of Role Play in Developing Telephone Conversation Skills in Engineering Students

Research Paper Recommended citation: McKennedy, J., Beagon, U., & Harper, S. (2025). Who's Calling? - Assessing The Effectiveness of Role Play in Developing Telephone Conversation Skills in Engineering Students. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17631846. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. WHO'S CALLING? - ASSESSING THE EFFECTIVENESS OF ROLE PLAY IN DEVELOPING TELEPHONE CONVERSATION SKILLS IN ENGINEERING STUDENTS J. McKennedy a,1, U. Beagon b, S Harper c a Technological University Dublin, Dublin, Ireland, ORCID 0000-0003-0565-6116 b Technological University Dublin, Dublin, Ireland, ORCID 0000-0001-6789-7009 c Technological University Dublin, Dublin, Ireland, ORCID 0009-0008-3219-992X Conference Key Areas: Engineering skills, professional skills, and transversal skills Keywords: Communication skills, Interdisciplinary communication; role-plays; Telephone Skills; Professional Skills ABSTRACT The use of telephones as a means of conversation has diminished in recent years with the advent of alternative, less personal forms of conversation. Professional telephone skills have been identified by employers and students as a challenge for new graduates in the engineering work environment. As part of the ECO-GT research project, a telephone role-play was designed to use with undergraduate engineering students. In Technological University Dublin, there were two iterations of this workshop delivered to a group of eight 3rd year students, and subsequently to nineteen 2nd year students. The purpose of the exercise was to demonstrate and practice the skills required in conducting professional telephone skills when communicating with professionals across different disciplines. Quantitative feedback demonstrated that students perceived that the learning outcomes were met. This 1 Corresponding Author Initials Last name J McKennedy e-mail address [email protected] paper provides a template for this workshop to be used with engineering students in other contexts to introduce them to the importance of telephone skills. 1 INTRODUCTION 1.1 Literature Review Technical skills are vital for engineers to design solutions and problem solve in their working environment but if they cannot communicate these solutions effectively to their teams and customers, it will be difficult to find any solutions. In recent studies (Beagon et al., 2022; Ortiz-Marcos et al., 2020) employers have identified communication skills as key competences required for the graduate engineer of today. Oral communication is the preferred mode of communication for engineers and becomes more important as they progress through their careers (de Souza Almeida, 2019). In a survey of recent engineering graduates, Sageev and Romanowski (2001) reported a poor level of phone etiquette awareness. Scott et al. (2012) successfully developed mini modules to integrate professional skills and specifically, communication skills, to engineering undergraduates. As part of the PREFER project the KU Leuven Engineering programme used short 1 ECTS modules to incorporate transversal skills, including communication skills, to the curriculum (Craps, 2020). Role-play is an active learning pedagogy where students portray a persona in a given simulation of real world encounters (Agboola Sogunro, 2004). It is commonly and effectively used in teaching communication skills (Stokoe, 2011). According to the literature, role-play has been used successfully in teaching telephone communication skills in the realm of healthcare professionals (Hamilton et al., 2013; Rude et al., 2023) but there is no evidence from the literature that it has been used in a similar manner with engineering students. 1.2 Background to the project The ECO-GT project was a joint initiative between seven universities across Europe including Technological University Dublin in Ireland. Funded by the European Union, this Erasmus+ project’s principal aim was to develop a framework of communication competences for green transition engineers. As part of this project, this mini module was designed and piloted to teach students how to manage a telephone call in the workplace in a professional and effective manner. Focus groups were conducted with employers to elucidate what were the key communication skills they required for engineering graduates. There has been little research into the teaching of telephone skills to engineering students over the past 20 years. However, the focus groups reported that they required good telephone skills in graduates and currently, graduates did not demonstrate these skills to a sufficient level. The research questions addressed in this paper were (1) What are students’ perceptions on the effectiveness of role-play pedagogy in learning professional telephone skills? and (2) Can a role-play pedagogy be used to develop professional telephone competence? 2 CONTEXT AND PRACTICAL WORK 2.1 Ethical Approval An ethical approval application was approved by the Research Ethics and Integrity Committee, (Technological University Dublin) for the delivery and evaluation of the module in 8th February 2024. 2.2 Approach A role-play workshop was designed to demonstrate and allow students to practice professional telephone communication skills. The learning outcomes that were addressed in this session were: (1) Describe the importance of good communication by telephone (2) Identify aspects of good and poor communication techniques used in phone calls (3) Participate in a telephone role play scenario (4) Evaluate personal and peer performance in phone communication techniques After participation, students were asked to fill out a survey based on their experience of the workshop. It included both a Likert scale to which students could indicate agreement with particular statements and opportunities for open responses. The same survey was used to collect the data from all the mini-modules delivered across the ECO-GT project. A comprehensive lesson plan is detailed in table 1 below. In the first iteration of this workshop, it was difficult to engage the students at the beginning so the hands up tally was used to great effect in the second iteration. In a hands up tally a question is asked that has a yes or no answer. First students raise their hands if their response is yes and then they raise their hands if the response is no. This allowed the participants to take ownership and made it easier for them to share their particular experience. It allowed the facilitators to ask follow up open ended questions and led to engagement and meaningful discussion. A very simple script was devised by the two lecturers facilitating the workshop where one person was the experienced architect and the other was the graduate engineer. Initially, the two lecturers were belligerent and rude in order to emphasise how ineffective bad communication can be. In the middle of the conversation the lecturers stopped the role-play and asked the students to assess how the lecturers were communicating and suggest how they could improve. The lecturers took the feedback on board and demonstrated the roleplay again using an appropriate, professional communication manner. Table 1:Lesson Plan for delivery of Role-play workshop Timing Content/Activity Introduction: 15 minutes Introduction The facilitators, project and lesson will be introduced. Hands up tally: Who would be comfortable making a professional conversation? Who would not? Open discussion: What would you find difficult about that situation? One of the facilitator gives context to the lesson: Students returning from work placement express difficulty with managing telephone calls in an engineering office environment. The aim of the workshop is to allow students to observe, critically review and practice making telephone calls. 10 minutes: Initial Lecturer role-play demonstration Two lecturers will act out a number of telephone call conversations based on a telephone conversation between two different disciplines (for example, an experienced architect and a graduate engineer). The role play will describe a problem and offer a solution to be negotiated between both disciplines. In the initial conversation, both lecturers will be as belligerent, rude and unhelpful as possible. This scenario will be stopped mid conversation and students will be asked to comment on what is wrong with this scenario and how both lecturers could improve their communication style. The lecturers will then demonstrate how a good telephone conversation should be conducted by both the architect and the graduate engineer. 20 minutes: Students will practice the role-play Students will be paired up and invited to practice a telephone call role play scenario with another student. They will be given a task such as “Describe to the Architect the impact of putting a green roof on a house extension project”. This will not be scripted and the students will be invited to engage in the role play and then give each other feedback after their role play. Lecturers will circulate and assist during the practice sessions. 10 minutes: Feedback and wrap up. Lecturers will field any questions that students may have and wrap up the session. There were 27 participants in total across two iterations of the workshop. In the first instance, there were eight undergraduate 3rd year civil engineering students and in the second instance there were 19 undergraduate 2nd year civil engineering students. There were 24 responses to the survey. The surveys collected a range of qualitative data in the form of open ended questions relating to what had been learned, what was good and what could be improved. A large range of quantitative data was also collected around the complexity level, pace and relevance of the workshop. 3 RESULTS AND INSIGHTS The timing of the workshop worked well. One hour provided the participants with time to engage with the material, to share their own personal experiences of telephone communications. During the workshop, one student shared that he had a speech impediment and asked how he could manage this in a professional phone call scenario. The demonstration of the phone conversation made him more comfortable to address this question and he was advised and agreed that if he took his time, and explained to the person who he was speaking to initially, that he would be professional in his communication. In the opening delivery of the second iteration, the lecturers started the role play conversation relating back to the last time they had met. Upon the completion of the demonstration, a student asked how this phone call could be started if they hadn’t known each other. The lecturers answered this by rerunning the role-play where they had not yet met and the phone call was their first oral communication, giving students examples of how to build rapport. A survey using a 5-point Likert scale was used to gather quantitative data about the students’ perceptions of the workshop. Students were not required to interact with all the statements presented and some students didn’t respond to all the statements posed. Some of the responses related to the appropriateness of the workshop format and content for the workshop are presented in table 2. Analysis of the results showed that the median was “strongly agree” for all statements except “The activities on the module contributed to my learning” where there was the same frequency for “strongly agree” and “agree”. These data show that students agreed or strongly agreed with the statements presented with the exception of one respondent who provided a “neutral” response to the statement “The type of delivery suited the topic”. The respondent who was neutral on the suitability of the topic answered “I don’t think so” to the qualitative question “If you were to change anything about this session, what would it be? What about the course / session could be improved?” so it is difficult to interpret the neutral response. Table 2: Student Responses on Telephone conversation role-play workshop Feedback Strongly Agree Agree Neutral Disagree Strongly Disagree Median The type of delivery suited the topic 6 (31.6%) 12 (63.2%) 1 (5.3%) 0 0 Strongly agree The activities on the module contributed to my learning 10 (50.0%) 10 (50.0%) 0 0 0 Agree – strongly agree I feel what I learned will be very relevant to my work as an engineer. 13 (68.4%) 6 (31.6%) 0 0 0 Strongly agree I feel confident in using what I learned today in my engineering studies. 10 (52.6%) 9 (47.4%) 0 0 0 Strongly agree Table 3 shows the students responses to how effective the workshop was in achieving the learning outcomes. A 5-point Likert scale was also used with these statements where the responses ranged from very effective to very ineffective. Analysis of the data showed that all of the medians were “very effective” showing that the participants perceived that the learning outcomes were met. Again, there were no respondents who said that the session was somewhat ineffective or very ineffective. All respondents state that the role-play was a somewhat effective or very effective method to achieve all the learning outcomes with the exception of one respondent who said that the learning outcome “Evaluate personal and peer performance in phone communication techniques” was neither effective nor ineffectively achieved with the role-play session. Table 3:Responses to effectiveness in achieving the Learning Outcomes Learning outcome Very effective Somewhat Effective Neither effective nor ineffective Somewhat Ineffective Very Effective Median Describe the importance of good communication by telephone 13 (68.4%) 6 (31.6%) 0 0 0 Very effective Identify aspects of good and poor communication techniques used in phone calls 10 (58.8%) 7 (41.2%) 0 0 0 Very effective Participate in a telephone role play scenario 11 (61.1%) 7 (38.8%) 0 0 0 Very effective Evaluate personal and peer performance in phone communication techniques 9 (50.0%) 8 (44.4%) 1 (5.6%) 0 0 Very effective The qualitative data that was collected in this workshop were based on the 3 open ended questions: 1. What were the main three things that you learned in the session/course? 2. What did you particularly like about the course / session? 3. If you were to change anything about this session, what would it be? What about the course / session could be improved? In answering the first question outlined above, several students used the words “polite”, “professional”, “patient” and “take notes”. Specific responses included: “Proper introduction is important, try to be informative and helpful even if you don't know the answer, be patient”; “Take notes when on phone, be polite and always tell the person your name”; “How to be nicer when making a phone call, how to manage a rude behaviour on a phone call, how to recognise whether the other person is being fair or not on a phone call”. One response specifically noted the importance of being aware of conversations between senior and junior members of staff: “Communication is a two sided thing and all participants in the conversation play a role in how well the conversation goes”. Sageev and Romanowski (2001) suggested that poor telephone etiquette was an issue for graduate engineers and these results are indicative that this has been addressed by this lesson. Students were able to recognise that professionals remain polite and patient when on the phone and that it is helpful to take notes of what was discussed. When analysing the responses to the second question related to what the participants liked about the course, there were a number of responses that demonstrated satisfaction with the roleplay methodology: “my favourite part was when we had to interact with each other over the phone”; “Practice to give clarity”; “Interesting and had to do it myself”; “Experiencing what it would be like in that situation”; “Discovering my abilities about this topic, as I had never tested myself before about it” and “The attempt role play, to try get through the anxiety of performing”. Although some of the students were reluctant to practice their own telephone call, they seemed to enjoy the opportunity once they overcame their anxiety. To the question “If you were to change anything about this session, what would it be? What about the course / session could be improved?” ten respondents said there was nothing they would change. There were four useful suggestions for how the workshop might be improved. Students suggested: “Maybe let the students act out the role play at the start”; “I'd say to make it a bit harder. Once I did a workshop where people were meant to do a speech in a limited time, from 2 minutes to 10 seconds. But fitting the same necessary information”; “Different scenarios of answering and calling” and “More on site examples”. All of these critical suggestions are useful and viable options to use in future iterations. In particular incorporating more examples, which was suggested by two of the respondents. The quantitative data presented in table 2 demonstrates the effectiveness of the roleplay pedagogy used in this scenario. Student feedback indicated that they liked the role-play activity and found it effective. Table 3 data clearly showed that the students perceived that the learning outcomes had been achieved. The open ended responses from student feedback showed that the words the students used to describe their learning were related to the learning outcomes required to conduct a professional telephone conversation. 4 CONCLUSIONS AND IMPLICATIONS Both the qualitative and quantitative data address the research questions posed in this paper. As previously identified by Stokoe (2011), this work has demonstrated the effectiveness of using role-play pedagogy to effectively teach engineering students how to conduct a professional telephone conversation in the work environment. This can easily be used with other undergraduate engineering students with potential for incorporating some of the suggestions that were put forward by the respondents in this iteration. It is recognised however that the contextual and cultural differences