Effectiveness of the inclusive design wheel in practice: Lessons from five pilot projects

Effectiveness of the Inclusive Design Wheel in practice: Lessons from five pilot projects J. Goodman-Deane1(), Mike Bradley1, Boris Lazzarini2, Elisabet Roca Bosch2, Sam Waller1, and P. John Clarkson1L 1Engineering Design Centre, Department of Engineering, University of Cambridge {jag76, mdb54, sdw32, pjc10}@cam.ac.uk 2Institute for Sustainability Science and Technology, Universitat Politècnia de Catalunya {boris.lazzarini, elisabet.roca}@upc.edu Abstract: The Inclusive Design Wheel (IDW) is an established inclusive concept design process, emphasizing iteration and early-stage evaluation. However, a structured evaluation of its effectiveness in practice has not been previously conducted. This paper describes how the IDW process was adapted for use in the design of digital mobility services. It was then trialled on five pilot projects in Italy, Spain, Belgium and the Netherlands. Questionnaires were completed by participants (both pilot partners and end-users) and interviews were held with pilot partners at the end of the project. Initial insights from the pilot experiences and the evaluation are described. The pilot teams found the IDW helpful and particularly valued its help in structuring the design process and offering clear steps and tools. Insights for improving the IDW focus on the importance and role of co-creation, issues with recording iteration and the balance between structure and flexibility. Implications are highlighted for improving the Inclusive Design Wheel and for inclusive design processes more generally. 1 Introduction There is a growing recognition of the diversity in the population and the consequent importance of inclusive design, i.e., “the design of mainstream products and/or services that are accessible to, and usable by, as many people as reasonably possible” (British Standard Institution, 2005). However, designing inclusively can be difficult in practice. Effective and practical inclusive design processes and tools are needed to help designers and other stakeholders to put inclusive design into practice. 2 The Inclusive Design Wheel (IDW) is one such process (Cambridge Engineering Design Centre, undated). It was developed based on over ten years of research and knowledge transfer to help designers to structure the inclusive concept design process within product design (Clarkson et al., 2007; Waller et al., 2015). The IDW is compatible with other design models such as the Waterfall, Double Diamond, V and Spiral models (Clarkson and Eckert, 2005; Design Council, 2007). However, it is more specifically focused on concept design and emphasizes iteration and early-stage evaluation (Waller et al, 2015). It also directly includes specific activities that are important for designing more inclusively. The IDW is also compatible with co-creation methods (e.g., Cruikshank et al, 2013), which can be used within the different phases and activities of the IDW. The IDW is composed of four main phases as shown in Figure 1 (Waller et al, 2015; Cambridge Engineering Design Centre, undated). Successive iterations of these phases are used to generate: 1. a clearer understanding of the needs that the product or service should meet; 2. better solutions to meet those needs; 3. stronger evidence that the needs are met. Fig 1. Overview of the IDW process (from Cambridge Engineering Design Centre, undated) The IDW provides a selection of activities within each phase, such as Create stakeholder map, Observe users, Develop concepts and Test with users. A more detailed diagram listing all the activities can be found in (Waller et al, 2015; Cambridge Engineering Design Centre, undated.). The IDW process and associated tools have been used on projects from a wide variety of domains and industry sectors (e.g., Centre for Business Innovation, undated), but a structured evaluation of its effectiveness in practice has not previously been conducted. This paper reports on the use and evaluation of an adapted version of the IDW in five pilot projects across Europe. Section 2 explains how the IDW was adapted for use in the design of digital mobility services and Section 3 3 briefly describes the pilot projects. Section 4 describes how the work was evaluated and presents initial results and insights from the evaluation. 2 Adaptation of the Inclusive Design Wheel The IDW process was originally intended for use in product design, but it can be adapted for use in different sectors, and for different types of applications. The overall process (shown in Figure 1) stays the same, but individual activities can be adapted and additional activities added to better match the characteristics and needs of the sector. The process can also be adapted for use in a particular project or company to fit better with an established design process or activities. As part of the EU funded DIGital traNsport In and for socieTY (DIGNITY) project (DIGNITY project, undated), the IDW was adapted for use in the design of digital mobility services (Bradley and Goodman-Deane, 2021), i.e., transport services that use digital interfaces to provide information or functionality to the user. Examples include websites and apps that provide information about travel, online ticket booking services and digital interfaces for ride sharing and vehicle hire. The digital mobility sector differs from traditional product design in various ways. For example, digital mobility services operate within larger transport ecosystems that involve many different services, providers and commercial, governmental and public stakeholders. Funding and management for public transport services may also differ from those in traditional product design. Adaptations were also made to the IDW process to fit better with the other activities in the DIGNITY project and to incorporate co-creation methods within the Create phase. A diagram of the adapted IDW process is shown in Figure 2. Examples of the changes in the IDW include replacing Observe users and Describe user journeys with Examine customer journeys. This was done to tie in with other work on the DIGNITY project that uses customer journey mapping to understand users’ experiences and needs (van Apeldoorn and Herpen, 2022). Another example is the addition of Make storyboards to the Create phase. This was included because storyboards are often more suitable than prototypes for describing concepts for a mobility service. A key issue in adapting the IDW was deciding what to include as an explicit activity in the process diagram (the small circles in Figure 2). There was only room for a limited number of such activities. As a result, some activities were included as part of a more general activity. For example, co-creation workshops are a key part of Stimulate ideas but are not directly visible in the process diagram. Care was also needed in trying to describe each activity in a few words so that people viewing the diagram could get a good impression of what was involved. 4 Fig 2. A diagram showing the adapted version of the IDW for the design of digital transport services and for the DIGNITY project. This diagram shows the individual activities within each design phase. Note that it is not necessary to do all of the activities within each of the phases. When advocating the use of a design process, it is helpful if that process is accompanied by a tool for guiding people through the process and capturing the outputs from activities and the rationale for design decisions. The tool can take a variety of forms, such as a website, a bespoke piece of software or a file in standard software, e.g., Microsoft Office. For the adapted IDW, a design log template (see Figure 3) was developed in PowerPoint. PowerPoint was chosen because most professionals already know how to use it and it allows users to easily add combinations of text and images. It also aids in the presentation of information to others throughout the design process. The design log included slides for each of the activities, with brief guidance on the activity and templates to fill in with notes and findings, e.g., different types of stakeholders or ideas generated from a co-creation workshop. 5 Fig 3. Example template from the Inclusive Design Log (v1.0) used in the DIGNITY project. The custom navigation bar at the top of the screen was developed for this project. 3 Pilot Projects The adapted IDW was piloted in five pilot projects (referred to as ‘pilots’ in this paper) across four cities/regions in Europe: one each in Ancona (Italy), Barcelona (Spain) and Flanders (Belgium), and two in Tilburg (Netherlands). All of the pilots involved the local authority for the city or region plus other partners such as local transport providers and consultancies with expertise in transport and mobility. More information on the pilots can be found in Goodman-Deane and Bradley (2022). The pilots used the adapted IDW to develop concepts, prototypes and recommendations for more inclusive mobility services in their regions. Different pilots had different specific aims and developed different outputs, as detailed in Table 1. This pilot work was conducted by the pilot teams with guidance and an initial evaluation of their concepts provided by the research team from the University of Cambridge. The work started in summer 2021 and this paper reports on work conducted up to May 2022. During this period, all of the pilots did substantial work on the Manage, Explore and Create phases of the IDW. Four of the five pilots did some Evaluate activities as well, thus completing at least one iteration of the adapted IDW process. However, as with any iterative process, the teams are continuing work to further improve their outputs. 6 Table 1. Summary of the five pilots Pilot Aims Outputs Ancona • To improve the accessibility of the current public transport app and website • To add non-digital solutions to include users who face digital exclusion • Prototype of a new version of the app • Suggestions for improving the accessibility of the website • Seven (mostly non-digital) recommendations for the transport system in the region Barcelona • To promote the use of demand-responsive transport (DRT) among groups with low digital competence • To make the DRT smartphone app simpler, more inclusive and more user-friendly • Three (mostly non-digital) concepts for improving the DRT service • Proposals for improving the DRT app Flanders • To develop a central point of contact (Hoppincentrale) for end users, which can be accessed via an app, website or call centre. This will provide public transport information and help end users to plan trips. • Five concepts for improving the public transport system in the region • A prototype of an app for accessing Hoppincentrale • Proposal for a training programme to equip public transport personnel in assisting people with disabilities. Tilburg pilot 1 • To develop an intervention to help digitally excluded people (particularly those who are older) to get from A to B • Concept for a central travel information phone number • Five storyboards describing how people could to find out about and get access to this phone line Tilburg pilot 2 • To develop a concept for a socially and digitally inclusive bike-sharing scheme, with special attention to the needs of migrant women. • Guidelines for bike sharing schemes • High level description of a concept for a new bike sharing service • Recommendations for an equitable shared cycling system in the region The implementation of the adapted IDW varied between pilots, depending on factors such as the scale and type of the problem being addressed and whether the team was adapting an existing solution or building a new one. There were differences in the exact activities conducted, in their implementation and in the type and level of detail of the outputs. Nevertheless, a key part of all the pilots were the co-creation workshops (see F, held as part of the Stimulate ideas activity. All pilots held at least one such workshop with target users for their project. The participants usually included some 7 from user groups at increased risk of digital mobility exclusion (such as older people or those with disabilities). All the workshops included elements that identified and explored users’ transport needs, as well as ones that elicited ideas for meeting those needs. Most of the workshops were held in person, but one pilot had to run theirs online due to COVID-19 restrictions. To balance this, they also held some telephone interviews with users at higher risk of digital exclusion. Fig 4. Photos from some of the co-creation workshops 4 Evaluation The pilot work on the adapted IDW was evaluated using a range of methods: - a questionnaire addressed to participants (end-users) in the co-creation workshops (part of the Stimulate Ideas activity), conducted at the end of or soon after the workshops; - a questionnaire addressed to pilot partners, conducted partway through the IDW process (usually somewhere in the Create phase); - a questionnaire addressed to pilot partners, conducted at the end of the IDW work examined in this paper; - a semi-structured interview with pilot partners, also conducted at the end of the IDW work examined in this paper. In addition, throughout the process, the University of Cambridge team recorded feedback from the pilots, any issues that arose and their own reflections on the experiences during the pilots. The questionnaire and interview responses indicated that the pilot partners considered the adapted IDW process to be helpful overall. They particularly valued the help it provided in structuring the design process and offering clear steps and tools. Nevertheless, the experience on the pilots highlighted a range of issues and 8 lessons to learn for improving the IDW. The analysis of the evaluation results is still ongoing but some of the initial lessons are described below. 4.1 Co-creation workshops All the pilots found the co-creation workshops helpful. For example, one of the pilot teams said, ‘[It was helpful] in giving insights directly from the target group about the target group.’ The participants in the workshops also valued them – the great majority of them found them useful, enjoyed them and felt they generated useful ideas. Furthermore, the observation of the pilots indicated that the co-creation workshops often provided key ideas and insights, both for the Explore (understanding users’ needs) and the Create (coming up with ideas for solutions) phases. This indicates that these workshops are extremely important for the IDW process. However, they do not appear directly in the process diagram in Figure 2 because they are part of the Stimulate ideas activity. The name of this activity does not convey anything about co-creation or user involvement and, as a result, the cocreation workshop could be easily overlooked. It would help to emphasize the importance of this activity if it appeared directly in the process diagram. Furthermore, the workshops provided important information for both the Explore and Create phases. However, they are only located in the Create phase in the process diagram. Further work is needed to consider how the relevance of the workshops to both phases can be communicated. 4.2 Iteration Iteration is an inherent aspect of design in general (Wynn and Clarkson, 2018) and of the IDW in particular (see Section 1). As well as the macro iteration of the Explore, Create and Evaluate phases, teams often engaged in micro iteration cycles where they made rapid, small changes to their insights and concepts as they discussed and tried out things together. However, some of the teams found it difficult to record this iteration within the PowerPoint design log: ‘[The design log] could be improved by organising better the addition of new inputs … to record the evolution of the pilot as it is [a] continuous process’. Some of the pilot teams came up with their own method of tracking iteration within the log, by annotating new material with a coloured circle with the word 'NEW' and the date. Some teams kept both the old and new versions of slides within the design log. However, keeping old versions makes the log more difficult to navigate, and it was still hard to capture and communicate how the content varied over time. 9 Capturing and communicating successive and iterative updates to multiple interlinked components is a difficult challenge that occurs in other domains as well. A rigorous and structured approach to version numbering can help, but excessive rigour and structure may be onerous and could stifle creativity. Bespoke project and file management software might be able to help, but this requires teams to adopt and learn new software, thus creating an extra barrier to its use. An alternative might be to create custom buttons within standard software (such as PowerPoint) to streamline the process of tagging material with the date. Further work with design teams would be required to explore and refine solutions to this issue. 4.3 Flexibility Some teams felt that the IDW process was a bit constraining. In the final evaluation questionnaire, one of the teams said, ‘[it] seems like you have to complete all the activities in order which is counter intuitive’. Another commented, ‘we used some elements but not the whole IDW method.’ The adapted IDW was intended to be structured and yet flexible, with multiple iterations, each containing a subset of the activities in Figure 2. However, this was not adequately conveyed in the process diagram. Furthermore, the use of PowerPoint for the design log meant that the slides for the different activities were presented in a linear order. Alternative tools, such as Adobe Illustrator and Miro, offer some potential to break away from this linear presentation. They allow boards to be laid out and moved around spatially within a workspace. This might fit better with the intended paradigm for completing the activities in the IDW. However, the availability of and prior experience with these tools is lower than PowerPoint, so further work is needed to determine the best environment in which to implement the design log and/or whether PowerPoint can be better customised to suit a more flexible process. 5 Conclusions and Further Work This paper has reported on initial results from the evaluation of five pilot projects that used an adapted version of the Inclusive Design Wheel in the design of mobility services with a digital element. The pilot teams found the adapted IDW process helpful and particularly valued its help in structuring the design process and offering clear steps and tools. Particular issues identified in the evaluation include the importance of the cocreation workshops and the need to highlight this better in the process diagram. The importance of these workshops also underlines the importance of early-stage user involvement and co-creation in inclusive design processes more generally.