Deliverable D 6.1 Impact Assessment Framework

Project number: F-DUT-2022-0088 Driving Equitable and Accessible 15 Minute Neighbourhood Transformations WP6. Impact assessment and evaluation T6.1. Impact assessment framework Deliverable D 6.1 Impact Assessment Framework Version: 2.0 Date: July 4th, 2025 Responsible partner: Vrije Universiteit Brussel (VUB) Authors: Charlotte van Vessem (VUB) & Imre Keserű (VUB) 2 DOCUMENT CHANGE RECORD Version Date Status Author Description 0.1 10/02/2025 Draft for internal review Charlotte van Vessem (VUB) First draft version 0.2 26/02/2025 Draft for internal review Charlotte van Vessem (VUB) & Imre Keserű (VUB) Second draft version with internal VUB comments 0.3 03/03/2025 Draft for internal review Charlotte van Vessem (VUB) & Imre Keserű (VUB) Third draft version with internal VUB comments 0.4 06/03/2025 Draft for internal review with request for input before the Paris consortium Charlotte van Vessem (VUB) & Imre Keserű (VUB) Draft version circulated to consortium in preparation of the Paris consortium meeting 0.5 28/03/2025 Draft for internal review Charlotte van Vessem (VUB) & Imre Keserű (VUB) Added clarification on the tables in the Living Lab data collection plans 1.0 23/05/2025 Draft for internal review by partners Charlotte van Vessem (VUB) & Imre Keserű (VUB) Finalised version to be reviewed by partners BOKU and UGE 1.1 30/05/2025 Internal review Roman Klementschitz (BOKU), Charalampidou Georgia (BOKU), Yusak Susilo (BOKU) Provision of internal review on draft 1.0 1.2 12/06/2025 Internal review Alain L’Hostis, Heythem Adjeroud (UGE) Provision of internal review on draft 1.1 2.0 04/07/2025 Final draft Charlotte van Vessem (VUB), Imre Keserű (VUB) Finalisation of the Deliverable to be published on the project website 3 EXECUTIVE SUMMARY This deliverable presents the Impact Assessment Framework (part of Task 6.1) developed for evaluating the Living Lab interventions implemented across the six Living Labs of the DREAMS-project: Brussels, Budapest, Munich, Paris, Utrecht and Vienna. As a starting point, this Deliverable reflects on the reasons to conduct evaluation, explaining the different types of evaluation that will be used in the DREAMS project and their most important concepts. We look into different timings of evaluation and what type of data needs to be collected, when and why, per evaluation type. The deliverable also explains the uses of stakeholder evaluation, Stakeholder-Based Impact Scoring (SIS), and what is needed to conduct such an analysis. Then, an overview is given of the establishment of the DREAMS Evaluation Framework, as well as how the data collection plans per Living Lab were created. These data collection plans detail each Living Lab's objectives, interventions, expected impacts, as well as indicators and target. This also includes an overview of what data needs to be collected, how, by whom and when. Furthermore, it includes a first overview of the stakeholder assessment (part of Task 6.2) to be conducted in each Living Lab. For this, the Deliverable includes an overview of each Living Lab’s relevant stakeholders, their goals, and the interventions alternatives. This Deliverable serves as a both a guideline and a report for the Living Lab’s interventions: it is expected to regularly receive updates as the interventions take place. These changes are reflected in the document change record. 4 TABLE OF CONTENTS List of figures ............................................................................................................................. 6 List of tables ............................................................................................................................... 6 1 List of abbreviations ............................................................................................................ 8 2 Introduction ........................................................................................................................ 9 2.1 Purpose of the Deliverable and Work Package and interaction with other tasks ........................ 9 2.2 Structure of the Deliverable ................................................................................................... 10 3 Evaluation tasks in the dreams-project .............................................................................. 10 3.1 The goals and scopes of the DREAMS-evaluation .................................................................... 10 3.2 Impact evaluation ................................................................................................................. 11 3.3 Stakeholder-based evaluation (T6.2) ...................................................................................... 12 3.4 Process Evaluation (T6.3) ....................................................................................................... 14 3.5 Timing of data collection ....................................................................................................... 14 4 The DREAMS Evaluation Framework .................................................................................. 15 4.1 Impact assessment and process evaluation.................................................................................. 15 4.2.6 Data collection for stakeholder-based assessment .............................................................. 21 5 Development of Indicators for the Living Lab interventions ................................................ 23 5.1 Identification of LL indicators for the DREAMS Evaluation Framework .................................... 23 5.1.1 Stage overview of framework creation ............................................................................... 23 5.1.2 Vienna Symposium workshop (October 2024) ..................................................................... 23 5.2 Development of the LL Indicator template ............................................................................. 24 6 Data collection plan per Living Lab .................................................................................... 28 6.1 Overview of the Living Labs ................................................................................................... 28 6.2 Living Lab Brussels ................................................................................................................. 29 6.2.1 Short overview .................................................................................................................. 29 6.3 Living Lab Budapest ............................................................................................................... 34 6.3.1 Short overview .................................................................................................................. 34 6.4 Living Lab Munich .................................................................................................................. 36 6.5 Living Lab Paris ...................................................................................................................... 41 6.6 Living Lab Utrecht .................................................................................................................. 47 6.7 Living Lab Vienna................................................................................................................... 50 7 Conclusion and next steps .................................................................................................. 54 8 Acknowledgements ........................................................................................................... 55 9 References ........................................................................................................................ 55 10 Appendices .................................................................................................................... 56 5 10.1 Appendix I. Template for LL leaders ....................................................................................... 56 10.2 Appendix II. Template for local stakeholders .......................................................................... 59 10.3 Appendix III. Poster templates for the Vienna Symposium Day ............................................... 62 10.4 Appendix IV. Poster templates Vienna – pictures of the workshop session .............................. 68 10.5 Appendix V. Stakeholder-based Impact Scoring guide by Te Boveldt, G. .................................. 76 6 LIST OF FIGURES Figure 1 The different stages of the evaluation process and the intervention…………………………………..11 Figure 2 Steps in the impact evaluation process………………………………………………………………………….....12 Figure 3 Visualisation of the different steps in a Stakeholder Impact Scoring……………………………….…13 Figure 4 Visualisation of the DREAMS Evaluation Framework.………………………………………………………………………………………………………………………………….16 Figure 5 Example of the workshop posters…………………………………………………………………………………....24 Figure 6 Visualisation of the order of the DREAMS Impact Assessment…………………………………………..26 LIST OF TABLES Table 1: : Example of an empty Living Lab objectives, indicators and measurement information template ……………………………..………………………………………………………………………….........................................28 Table 2: Mobitwin objectives, indicators and measurement information…………………………………………30 Table 3: Cozywheels objectives, indicators and measurement information……………………………………...31 Table 4: Cargobike objectives, indicators and measurement information……………………………………31-32 Table 5: Stakeholders Brussels………………………………………………………………………………………………………32 Table 6: Stakeholder objectives Brussels………………………………………………………………………………………..32 Table 7: Baseline and alternatives Mobitwin……………………………………………………………………………..32-33 Table 8: Baseline and alternatives Cozywheels………………………………………………………………………..……..33 Table 9: Baseline and alternatives Cargobike…………………………………………………………………………….…...33 Table 10: Living Lab timeline Brussels…………………………………………………………………………………………..33 Table 11: Budapest intervention objectives, indicators and measurement information………….………..34 Table 12: Stakeholders Budapest…………………………………………………………………………………………………..35 Table 13: Stakeholder goals Budapest…………………………………………………………………………………………...35 Table 14: Baseline and alternatives Budapest………………………………………………………………………………..35 Table 15: Living Lab timeline Budapest…………………………………………………………………………………………36 Table 16: : Micro-mobility points’ objectives, indicators, and measurement information ………………..37 Table 17: Rewards program for public transport and shared mobility’s objectives, indicators and measurement information……………………………………………………………………………………………………….…..38 Table 18: Stakeholders Munich…………………………………………………………………………………………………38-39 Table 19: Stakeholder objectives Munich………………………………………………………………………………..39-40 Table 20: Baseline and alternatives for micro-mobility points in Wolfratshausen.…………………………40 Table 21: Baseline and alternatives for micro-mobility points in Geretsried …………………………………40 Table 22: Baseline and alternatives for the rewards program for public transport and shared mobility………………………………………………………………………………………………………………………………………41 Table 23: Living Lab timeline Munich…………………………………………………………………………………………..41 Table 24: New mobility systems objectives, indicators and measurement information…………………..42 7 Table 25: Discounts on shared mobility objectives, indicators and measurement information……….43 Table 26: T12 signage objectives, indicators and measurement information……………………………43-44 Table 27: Stakeholders Paris……………………………………………………………………………………………………….44 Table 28: Stakeholder goals Paris ………………………………………………………………………………………….44-45 Table 29: Baseline and alternatives Paris……………………………………………………………………………….47-48 Table 30: Living Lab timeline Paris ……………………………………………………………………………………….46-47 Table 31: Discounts for shared bikes objectives, indicators and measurement information………….47 Table 32: Solar station objectives, indicators and measurement information……………………………….48 Table 33: Stakeholders overview Utrecht…………………………………………………………………………………...48 Table 34: Stakeholder goals Utrecht…………………………………………………………………………………………...48 Table 35: Baseline and alternatives Utrecht………………………………………………………………………………..49 Table 36: Living Lab timeline Utrecht……………………………………………………………………………………49-50 Table 37: Flexible activity hub objectives, indicators and measurement information……………....51-52 Table 38: Bike sharing station objectives, indicators and measurement information……………….52-53 Table 39: Stakeholders Vienna…………………………………………………………………………………………………..53 Table 40: Stakeholder goals Vienna……………………………………………………………………………………………53 Table 41: Baseline and alternative flexible hub…………………………………………………………………………..53 Table 42: Baseline and alternatives bike sharing station…………………………………………………………….54 Table 43: Living Lab timeline Vienna………………………………………………………………………………………....54 8 1 LIST OF ABBREVIATIONS 15mC: 15-minute city BME: Budapesti Műszaki és Gazdaságtudományi Egyetem / Budapest University of Technology and Economics BOKU: Universität für Bodenkultur Wien / University of Natural Resources and Life Sciences, Vienna CBA: Cost-Benefit Analysis DAT: DREAMS Accessibility Tool DDST: DREAMS Decision Support Tool DREAMS: Driving Equitable and Accessible 15 Minute Neighbourhood Transformations DUT: Driving Urban Transitions Dx.x: Deliverable number per Work Package HU: Hogeschool Utrecht IKP: Interactive Knowledge Platform KPI: Key Performance Indicator LL: Living Lab MCA: Multi-Criteria Analysis SIS: Stakeholder Impact Scoring Tx.x: Task number per Work Package TUM: Technische Universität München / Technical University of Munich TUW: Technische Universität Wien / Vienna University of Technology UGE: Université Gustave Eiffel UT: Universiteit Twente / Twente University VUB: Vrije Universiteit Brussel WP: Work Package 9 2 INTRODUCTION 2.1 Purpose of the Deliverable and Work Package and interaction with other tasks This report is Deliverable 6.1 (D6.1) of the DREAMS (Driving Equitable and Accessible 15 Minute Neighbourhood Transformations) 1 project, which aims to explore how co-created and user-centric mobility services, mobility and flexible activity hubs can actively contribute to creating accessible, sustainable, and inclusive 15-minute City (15mC) neighbourhoods in the urban outskirts 2 of European cities and regions. This deliverable is part of Work Package 6 (WP6), titled “Impact Assessment and Evaluation”. Coordinated by the Vrije Universiteit Brussel (VUB), this WP assesses and synthesises the impacts of co-created scenarios and services in the Living Labs covering tangible (climate, accessibility, economic, health, liveability), non-tangible (governance, equity) and stakeholder-specific impacts. Work Package 6 consists of the following three Tasks: • Task 6.1: Impact assessment framework, led by Vrije Universiteit Brussel (VUB). This task is active in project months 14 until 18 (2025). • Task 6.2: Impact assessment of 15mC interventions and scenarios, led by the University of Natural Resources and Life Sciences, Vienna (BOKU). Active in project months 18 until 32 (20252026). • Task 6.3: Evaluation Synthesis, led by VUB. Active in months 32-34 of the project (2026). This Deliverable is the result of the corresponding Task 6.1 (T6.1), “Impact Assessment Framework”. The goal of this Task was to develop an impact assessment framework for the various Living Lab (LL) scenarios, part of WP5 that is concerned with the running of all LLs in the project, and interventions on the different layers of the Dreams Decision Support Tool (DDST). This tool (currently in development) will offer several ‘layers’ of best practices, methodologies and proximity calculations to help urban planners and other stakeholders design 15mC neighbourhoods. Work Package 6 is specifically linked to layers 3, 4 & 5 of the DDST (part of Work Package 3 and Deliverables 3.1, 3.2 and 3.3, all forthcoming). Additionally, this Deliverable (D6.1) indexes the measurement methods and defines the data collection schedule and data quality requirements for each LL intervention, to function as a data collection guide throughout the interventions. It also includes explanations of the stakeholder assessment method Stakeholder Impact Scoring (SIS) that will be used as part of T6.2, “Impact assessment of 15mC interventions and scenarios”, led by the University of Natural Resources and Life Sciences, Vienna (BOKU). Finally, it includes a first evaluation framework, which will be used in T6.3, the “Evaluation Synthesis” in which the implementation of all LL interventions will be evaluated. WP6 is scheduled to produce two Deliverables, the first one being this current Impact Assessment Framework (D6.1). The second Deliverable will be the Synthetic evaluation report of all Living Labs, which is scheduled for month 34, at the end of the project in late 2026. As the work of WP6 is closely related to the LL’s interventions for all Tasks of WP6, it is important to ensure a good connection to WP5, which entails the Living Labs set-up and coordination, led by BME (Budapest University of Technology and Economics). For this reason, common monthly meetings for both WP5 and WP6 have been set-up since April 2025. Additionally, the Impact Assessment Framework and the Assessment of T6.2 will also connect to various layers of the WP3 Dreams Decision Support Tool, especially to T3.5, “Strategic objectives and societal impacts”, led by UGE (Université Gustave 1 A list of all abbreviations can be found at the beginning of this Deliverable. 2 In the DREAMS-project, urban outskirts are understood as “Medium-sized areas or neighbourhoods in the near context of an urban area, where high car dependency is present and a strong economical and functional connection to the city is observed, primarily through daily work commutes” (Deliverable 2.1, DREAMS-project, 2024). 16 Figure 4: Visualisation of the DREAMS Evaluation Framework. In order to make Figure 4 easier to understand, the next sections will outline the definition of objectives; indicators and specifically indicators; targets; possible data collection methods for these indicators and KPIs; and the principles for the data collection plans for the DREAMS LLs. 4.2 Key elements of the DREAMS Evaluation Framework 4.2.1 Objectives In the DREAMS Evaluation Framework, objectives are the overarching goals of an intervention or project. As such, they can refer to either an output or an outcome, meaning that they either connect to a LL-specific goal (output), or a project-related one (outcome). To give an example of each: an outcome objective could be ‘fostering 15-minute accessibility to key destinations in urban outskirts’, or ‘supporting modal shift to sustainable modes’. An example of an output objective could be ‘increasing uptake of shared bikes’ or ‘better understanding of the area’s business potential’. Objectives can be linked to one or several indicators (see next section) and they can also be linked to stakeholders in SIS evaluations. 4.2.2 Indicators Once the objectives of the project and/or the intervention have been established, it is time to continue to set indicators. Indicators are measurable elements that are used to track the progress, performance or quality of a project, process, or system. They can be both qualitative and quantitative in nature. Within (transport) projects, indicators are used to gain insights in a project’s current performance, and to take evidence-based decisions regarding the project’s future directions. Indicators can be categorised into two main types: descriptive indicators and performance indicators. Descriptive indicators assess the state of a system at a given point in time, allowing for cross-sectional evaluations. Performance indicators, on the other hand, facilitate comparisons of descriptive indicators over different periods, enabling the assessment of progress toward specific targets (Chakhtoura & Pojani, 2016). In a top-down approach, a set of indicators is created based on literature research or interviews with experts, keeping in mind the main scope(s) of the project and its interventions. When choosing this approach, Doran asserts that indicators should adhere to the SMART criteria, meaning they must be (a) specific, (b) measurable, (c) achievable, (d) relevant, and (e) time-bound (Doran, 1981 in Bjerke & Renger, 2017, p. 125). Similarly, Gillis et al. (2015) argue that indicators should be (a) specific, (b) comprehensive, (c) precisely measurable, (d) sensitive, (e) technologically neutral, and (f) scalable. 17 When selecting indicators in a bottom-up manner, the project partner and relevant stakeholders create a set of indicators based on their planned interventions and what they would want to achieve and assess in their specific context. The selection of indicators should consider the temporal and spatial dimension of the evaluation. Additionally, the choice of indicator sets should be guided by the scale of the evaluation, the contextual factors—including environmental, geographical, and socioeconomic considerations—the timing of the assessment, and the availability of relevant data (Gudmundsson, 2004). 4.2.3 Indicators for the 15-minute city in urban outskirts The 15-minute City (15mC) is an urban planning concept that emphasises proximity-based accessibility, ensuring that neighbourhood residents can meet their daily needs—such as work, education, healthcare, shopping, and leisure—within a 15-minute walk or bicycle ride from their homes (Moreno et al., 2021). While much of the existing research and application of this model have focused on dense urban cores, its implementation in urban outskirts presents unique challenges, particularly regarding transport infrastructure, interconnectivity, and accessibility of amenities. As the goal of the DREAMSproject is to “explore how co-created and user-centric mobility services, mobility and flexible activity hubs can actively contribute to creating accessible, sustainable, and inclusive 15mC neighbourhoods in the urban outskirts of European cities and regions”, this indicates a need for a specific set of indicators focused on these core elements of the project. So far, no standard indicators dedicated to sustainable 15mC in urban outskirts have been identified. However, the DREAMS-project proposal did indicate six key impacts areas that are essential to assess the project’s interventions. These key areas are : Accessibility, Carbon Emissions, Liveability, Health, Economic Impact, and Equity. A more in-depth analysis and explanation of these themes and the KPIs that will be connected to these domains will be included in D3.3, which includes the results from T3.5, “Strategic Objectives and Societal Impacts”. This task will define the overall strategic objectives of the project, which will as such not be discussed in detail in this Deliverable (6.1). However, as D3.3 is only scheduled to be published in late 2025, we give a short overview of the main themes down below. Designing a coherent and context-sensitive indicator framework for 15mC evaluation in urban outskirts requires both theoretical grounding and practical consideration of what is measurable across diverse local contexts and conditions. T3.5 contributes to this effort by identifying a core set of accessibility indicators (to be integrated into the DREAMS Accessibility Tool (DAT)), and establishing a broader set of additional indicators (feeding into the Interactive Knowledge Platform (IKP)) that would allow a comprehensive project-level impact assessment. Together, these tools aim to support practitioners in selecting appropriate metrics based on their specific planning priorities. The indicator selection process involved rigorous evaluation of each potential indicator's sensitivity to LL activities, relevance to individual LLs, and measurability across different contexts. This screening process, outlined in more detail in the forthcoming comprehensive indicator analysis in Deliverable 3.3, has led to the identification of a refined set of indicators that is designed to ensure both practical applicability in the project LL contexts and a strong alignment with the project's strategic objectives. While research on the 15mC concept in connection to urban outskirts is scarce, D2.1 of the DREAMSproject, titled "Exploring the 15-minute city concept for urban outskirts: a systematic literature review”, written by the Universiteit Twente (UT), has highlighted six key areas that need to be considered in this context. These key areas are: Density, Diversity, Design, Human perspectives, Governance, and Business. In the section below, these areas have been matched to indicators. We have also chosen to add a seventh area on Environment and Sustainability, as this is a key component of a 15mC strategy (Moreno et al., 2021). It is important to note that not all impact areas are equally represented in the final indicator set, as this reflects the reality that not all LLs are conducting activities to which certain indicators would be sensitive. 18 1. Density First, a fundamental aspect of the 15mC concept is proximity to essential services, which refers to the accessibility of amenities such as healthcare, education, retail, and recreational areas within a 15-minute walking or cycling radius. For example, Chiaradia et al. (2024) propose measuring service intensity within a given area to assess spatial inequalities in accessibility. This approach helps identify gaps in service distribution and supports urban planning strategies aimed at reducing spatial disparities. This aspect of the 15mC also entails the presence of (essential) amenities. The filtered indicator set in the DREAMS-project includes: • Travel times: Average travel time to essential services • Service intensity index: Number and distribution of essential services within a 15-minute radius • Proximity to daily needs: Percentage of population within x-minute radius of amenities • PT catchment: Percentage of population within x-minute radius of a transit stop • Availability of shared mobility options: Number of shared bikes/scooters/cars available per resident or within x-minute radius • Resident satisfaction: Scores regarding accessibility to different amenities 2. Land use diversity Land use diversity is measured through indicators of mixed-use development. A balanced mix of residential, commercial, and institutional land uses can help foster local economic activity and reduce the need for long-distance travel (Zhang et al., 2022). This is particularly relevant for the DREAMSproject, as we specifically consider the effectiveness of mobility interventions in peri-urban areas, which depends on the availability of services within a short travel distance. The provision of public spaces, recreational and green areas is another key aspect of the 15mC model. Studies emphasise the importance of access to parks and recreational areas in promoting physical activity, social cohesion, and overall well-being (Chiaradia et al., 2024). Evaluating the proportion of residents with access to (highquality) green spaces within a 15-minute radius can provide insight into the liveability of urban outskirts. Although the filtering process revealed limited sensitivity to current LL activities and challenges related to measurability, the following indicators were retained in the final set because they can be reliably measured through the DREAMS Accessibility Tool (DAT): • Public spaces & recreation: areas accessible within an x-min radius • Green space accessibility: % of population within an x-min radius of parks and green space • Sport & recreation: % of population within an x-min radius of fields, playgrounds, ana other recreational amenities 3. Design The design dimension mainly focuses on the physical infrastructure that enables 15mC functionality through the monitoring of metrics such as the walkability index that helps assess the quality of pedestrian infrastructure, and considers factors such as sidewalk availability, pedestrian safety, and route directness (Chiaradia et al., 2024). However, traditional walkability and cyclability indicators showed limited sensitivity to the specific LL interventions being tested across the Living Labs. There’s clearly a need for future research to develop design indicators that better capture the impact of interventions like the implementation of new innovative mobility services on the quality of infrastructure in urban outskirts. 19 4. Human perspectives Social equity and inclusion are also central to the evaluation of 15mC initiatives. Equity indicators assess whether all demographic groups—including women, low-income households, older adults, and people with disabilities—have equal access to essential services and sustainable mobility options. By integrating these indicators into the evaluation framework, the project can ensure that 15mC principles benefit all segments of the population, rather than reinforcing existing spatial inequalities (Zhang et al., 2022). Our current filtered indicator set includes: • Diversity: Percentage of female and elderly service/activity users • Reported accessibility to amenities for target groups • Accessibility of the service/activity for target groups • Travel cost: Estimated average travel cost per trip or period 5. Governance Governance represents a critical yet often underexplored dimension of 15mC implementation, particularly in urban outskirts where traditional top-down planning approaches may prove insufficient. Effective governance in this context requires collaborative and participatory processes that engage a broad range of stakeholders, including public authorities, local businesses, residents, and especially underrepresented and underserved communities, in the co-creation of mobility solutions and urban services (Pozoukidou & Chatziyiannaki, 2021). The governance framework for 15mC in urban outskirts must address several key challenges: coordinating across multiple administrative boundaries that often characterize peri-urban areas, ensuring inclusive participation in planning processes, and establishing mechanisms for adaptive management of various interventions. However, governance frameworks enable long term strategies and are difficult to assess within the relatively short evaluation periods of most urban experimentation projects., Our indicator screening process reflected these difficulties: governance-related indicators often lack direct sensitivity to Living Lab (LL) activities or present measurement challenges. While indicators such as "Number of public events organized by public authorities" and "Resident satisfaction scores regarding communication, decision-making" were identified in the comprehensive analysis, only one key indicator is currently retained: • Percentage of target-area residents who attended workshops, gave feedback or participated in a survey 6. Business Economic vitality is an important factor in evaluating 15mC projects. Indicators such as local business activity, employment rates, and commercial diversity within a 15-minute radius can offer valuable insights into the economic sustainability of urban neighbourhoods (Chiaradia et al., 2024). However, these indicators showed limited alignment with current LL activities, with only one indicator retained in the filtered set: • Willingness to pay 7. Environment and sustainability Environmental impact assessments within the 15mC framework typically focus on reductions in carbon emissions due to increased accessibility and decreased reliance on private vehicles. Measuring transport-related emissions before and after interventions can provide insight into the environmental benefits of shifting towards localized mobility solutions (Zhang et al., 2022). By integrating these 20 indicators into the evaluation framework of the DREAMS project, we ensure that the assessment extends beyond conventional transport-focused KPIs to capture the broader socio-spatial impacts of 15mC interventions. This approach aligns with recent research emphasizing the need for multidimensional evaluation methods in 15mC planning (Chiaradia et al., 2024; Zhang et al., 2022). However, as of the current situation in the LLs, few environmental indicators got through the screening process because of the measurability challenges. the filtered indicator set only includes: • Reported number of car trips (for specific target groups in the LL area) • Uptake of shared mobility: Including shared mobility usage, number of trips, number of new users, and distance covered It is important to note that, regardless of their current categorisation within the evaluation framework, certain indicators may demonstrate relevance across multiple thematic categories. This is due to the inherently overlapping nature of impacts across dimensions such as accessibility, sustainability, human experience, and economic viability. Moreover, the current set of indicators reflects the present state of Living Lab activities and should be considered a foundational layer, open to refinement and expansion as interventions evolve. This approach ensures that the evaluation framework remains grounded in measurable outcomes while maintaining alignment with the broader theoretical understanding of 15mC principles in urban outskirts contexts. 4.2.4 Targets After deciding what indicators are relevant to measure outcome and output, it is be necessary to formulate what qualifies as a success for this indicator and objective. Setting targets plays a crucial role in evaluation, as they enable the ex-post assessment of how well the objectives have been achieved. Targets are directly tied to the project’s main objectives and the indicators. In many cases, failing to meet key targets can result in the project being deemed unsuccessful, even if it performs well on other criteria or scores well overall based on the full evaluation framework (Keseru et al., 2016). Targets set a quantifiable threshold that indicates direction (i.e., an increase or a decrease) and a number or percentage (i.e., increased uptake of the service with 10%). Similarly to indicators, targets should also comply with the SMART criteria outlined above (Bjerke and Renger, 2017; Hyllenius et al., 2009). This entails the following elements: Specific: targets should be clearly defined, preferably in quantifiable terms. For example, a target could be to reduce peak-hour car traffic in a suburban neighbourhood by 15%. This target assumes that baseline traffic data is available and that traffic can be measured both before and after the project. If quantification is not possible, a qualitative assessment can be used, with terms like ‘improvement’ or ‘positive’ impact. Measurable: targets can only be monitored if both the baseline and post-project conditions are measurable. In our example, it should be possible to measure peak-hour traffic before the project begins and after its completion. Ambitious and Accepted: ambitious targets motivate progress throughout the project. Additionally, targets should be supported by both decision-makers and the project team. Realistic: while targets should be ambitious, they must also be achievable. For instance, if public transport usage declines by two percent annually, maintaining the same level of public transport use as the previous year may be a realistic target. Time-bound: the timeframe within which targets should be achieved should be defined. In our example, the target to reduce peak-hour car traffic between 2025 and 2027 would be time-bound. Of course, targets should be ambitious, but they should also feasible. This is why it is important to have access to baseline data regarding the target indicator, so that the target can be set at an appropriate level. It is also important to note that in the DREAMS-project, targets serve as an ambition, but failing to meet them will have no negative consequences on the LL partners (beyond the lack of success of their 21 intervention). These targets are strictly meant to provide an aim for the intervention. In the interviews planned for T6.3, we will then discuss if the targets were met, and the reasons for success or failure to meet them. 4.2.5 Data collection methods Impact assessment is based on the systematic collection of quantitative and qualitative data. Data collection must ensure that the results are reliable and representative of the target population of the project. The choice of the data collection method depends on the budget available, availability of external data, expertise of the project management team and timescale of the project. The following methods can be used to collect data depending on the type of the indicator: Direct measurement of indicators: the most reliable way of collecting indicator data is the direct measurement of the indicators using impact measurement devices (e.g. NHx, PM2.5, noise), tracking equipment (GPS) or carrying out counts or surveys (traffic data, passenger numbers, etc.). At the same time, it is the most resource-intensive way of data collection as these measurements are expensive and require the involvement of experts. Surveys: some indicators cannot be measured directly as they are qualitative (e.g. perception of safety, satisfaction with route planning services, quality of urban space, socio-political acceptance). Such indicators require a survey of citizens and/or transport users. Estimation: in some cases, direct measurement of indicators may not be possible because of lack of resources or the complexity of such measurements. It is especially true for such cost-intensive activities as air quality monitoring, noise measurements and modelling. In these cases, it is possible to estimate the impacts through other proxy indicators. Air pollution and noise are, for example, directly related to the number of kilometres travelled by different transport modes. Monitoring the number of passenger or vehicle kilometres travelled and using average emission factors these indicators can be estimated. Modelling: Indicators that should be collected from several locations are often modelled. Such indicators include noise, traffic and air pollution. Using a computer model to estimate them reduces the cost of data collection. It, however, requires expertise or the involvement of an external consultant. Official statistics: some indicators may be available from official national, regional, local or operators’ statistics. Typically, indicators like the number and severity of accidents, public transport passengers etc. can be derived from such statistics. Very often, however, the spatial extent and the time period covered by official statistics are different to the project’s geographic and temporal scope. The appropriate data collection method will be identified for each DREAMS LL intervention and indicator in the individual LL data collection plans (see below) 4.2.6 Data collection for stakeholder-based assessment This section outlines the key elements of the stakeholder assessment method ‘Stakeholder Impact Scoring’ (SIS), which will be used in all LLs as part of T6.2. In order to conduct a SIS evaluation, the data needs per key step (described in Section 2.2) are as follows: 1. Data collection o Gathering structured and unstructured data from multiple sources. This can be from sources such as literature reviews, but can also come from expert consultations, and stakeholder inputs. Stakeholder inputs and consultations can be organised in different ways; for example, a workshop day can be held or a template can be made to be filled out by the expert or stakeholder (similar to the template that can be seen in Appendix II). o Data: the data used can also come from different sources. Reports, statistical datasets, expert opinions, and policy documents can all function as data for a SIS. 22 o The data collection will be done by each LL individually. The LL leader(s) will have to organise the data collection of their own relevant local stakeholders. As the LLs each have their own topic and scope, the decision on who are relevant stakeholders or experts will differ per LL. A first overview of the LLs local stakeholders can be seen in Section 6. 2. Data processing and integration o This entails standardising and integrating data to create a coherent dataset for analysis. o Data: in order to be ready to use, the data should be cleaned and compiled into formatted datasets. 3. Analytical modelling and scenario development o Employing qualitative and quantitative methods to analyse trends, relationships, and potential outcomes. o Data: Forecast models, scenario simulations, and comparative assessments. 4. Visualisation and communication o Presenting results in an accessible manner to inform decision-makers and stakeholders. o Data: Graphical representations, dashboards, and summary reports. It is important to note that each LL will be responsible of conducting their own SIS. As the LLs vary in their scope, the final choices as to who are relevant experts or stakeholders, as well as what type of data will be used as input will vary between the various LLs. Each LL leader has the final say over their specific SIS analysis. As the creators of the method, the VUB will assist the LLs in performing their SIS analysis. 4.2.7 Stakeholder perspectives and identification of baselines and alternatives Next to the objectives, potential impacts, indicators, targets, and measurement methods and frequencies, the stakeholder template described above also included questions regarding stakeholder preferences as well as the identification of scenarios and alternatives. These last two concepts are key elements of the SIS analysis used in T6.2. The stakeholder templates were distributed by the VUB to the LL leaders in November 2024, who in turn distributed the template to their relevant local partners. The selection of local stakeholder was made autonomously by each LL leader. The distribution itself happened in multiple ways: some partners held a meeting with the local stakeholder(s) to discuss their objectives, filling out the template together, while others had the stakeholder fill out the template by themselves. The full template can be found in Appendix II. In any case, this template represents a first exploration of the various stakeholder perspectives present throughout the different LL and their interventions. It serves as a starting point for T6.2, but it will have to be enhanced with more data collection regarding the various stakeholder perspectives, objectives, and preferences. Additionally, it was necessary to establish a baseline as well as alternatives for each intervention. These sections on scenarios and alternatives were part of the LL leader stakeholder templates that were distributed by VUB to the LL leaders in the same timeframe as the stakeholder templates. This template included a section in which the LL leaders were asked to identify a baseline and alternatives for each intervention. The baseline is a situation without any intervention and alternatives are different versions of the interventions’ implementation. These may include different designs, locations, policies, or approaches. An example of this could be: Baseline: there are no shared bikes in the Living Lab location [current situation]. Alternative A: shared bikes are placed in a central location. Alternative B: shared bikes and cargo-bikes are placed in a central location. Alternative C: shared bikes are placed in a central location and throughout the neighbourhood. 23 Alternatives can be very similar to each other, such as A-B-C above, but they can also be less similar: Alternative D: shared cars are placed in a central location Alternative E: we host a café for neighbours. Section 6 provides the contacted local stakeholders as well as their goals and the baseline and alternatives drafted by each LL leader. Stakeholder involvement is often a long-term process, where often multiple meetings are necessary to clarify the goals and viewpoints of the stakeholders. Arranging such meetings can also prove difficult with certain stakeholder groups. As such, this is an ongoing process in most Living Labs, where changes or updates are expected throughout the course of the project and the specific interventions. Such changes will be reflected in this Deliverable, primarily in Section 6. 5 DEVELOPMENT OF INDICATORS FOR THE LIVING LAB INTERVENTIONS After having established the overarching objectives of the project, it is time to consider each Living Lab’s individual goals and interventions. This was a part of Task 6.1, which was carried out by the VUB. This process had several stages, which are explained in the following sections. 5.1 Identification of LL indicators for the DREAMS Evaluation Framework 5.1.1 Stage overview of framework creation First, the LL locations and local intervention were selected by local partners (the university partners and their local stakeholders). This happened during the first months of the project, starting from March 2024 until November 2024. Each LL team could decide these for themselves, and these were partially predefined before the start of the project (during the proposal writing stage). 5.1.2 Vienna Symposium workshop (October 2024) Once the plans for the LLs were known, the VUB organised a workshop during the DREAMS Vienna Symposium day on October 11th, 2024. The goal of this workshop was to have a first exploration of the possible goals and objectives of each LL. Together with partners from each city plus members of the local audience, the workshop created a first list of expected impacts for each LL location, as well as possible KPIs and their measurement method and frequency. the only exception to this process was the Paris LL, as there was no project partner present to represent their Living Lab. The three research questions of this workshop session were: • What impact would you expect from the intervention of the Living Lab (positive or negative)? • What indicators would you use to measure these impacts? • How would you measure these impacts? Figure 5 shows an example of the Brussels poster mid-session, with answers for the first two questions stated here. 24 Figure 5: Example of the workshop posters, in this case the Brussels Living Lab (photo credits: VUB) The other posters, as well as the templates that were used in this workshop can be found in Appendices III and IV. 5.2 Development of the LL Indicator template Based on these first results from the Vienna workshop, the VUB created a first thematic overview of the reported impacts, indicators and measurement methods. These were then used to craft two templates: one for the Living Lab leading partner, and one to be filled out by local stakeholders (with the help of the LL leader(s)). These templates were used to indicate objectives, expected impacts, indicators and possible measurement methods of each individual LL location. These templates can be found in Appendix I. The creation of these templates was done in November 2024, and the period for filling out the templates initially took place during December 2024. However, as it proved difficult for some partners to organise a meeting with their local stakeholders, this timeline was extended into January and February of 2025. 5.2.1 Template collection and analysis These individual templates were analysed by the VUB for viability and for common themes. As all LL’s varied in their interventions and approaches, not many commonalities were found. These common themes were discussed in a WP6 partner meeting, as well as at the Paris Consortium meeting in March 2025. After this meeting it, was decided to follow up on the templates separately for each Living Lab and develop LL-specific indicators, with the more overarching set of indicators being part of Task 3.5 25 instead. It was decided that instead, each indicator in the data collection plan would be classified as either referring to an outcome (i.e., related to the project) or output (i.e., related to the intervention). A full list of output indicators is being developed as part of Task 3.5, which will be presented in a forthcoming deliverable. 5.2.2 Data collection plan finalisation Based on the variety of interventions and their impacts, the decision was made to create a template that would be overarching but still allowed to tailor to each individual Living Lab and intervention. Per Living Lab, this framework includes information on the interventions, as well as a description of the data needs for the output and the intervention outcome. Depending on the exact contents of the intervention per LL, this can entail a mix of data to be collect ex-ante, ex-post and continuous collection. These plans were ‘finalised’ as a first draft after Paris Consortium meeting in March 2025, after the VUB held meetings with all LL leaders separately in March and April 2025. 5.3 Measurement Approaches The evaluation of interventions across the different Living Labs employs a combination of qualitative and quantitative methodologies. The measurement framework follows an ex-ante, ex-durante and expost evaluation structure. Considering the varying objectives and expected impacts of each LL and intervention, the measurement approaches are tailored to specific contexts while maintaining common elements across locations. These common elements include shared themes and methodologies, such as the use of surveys, tracking data from intervention-related vehicles, and a mix of qualitative and quantitative approaches. As shown in Figure 6 below the measurement process is structured in four key stages: 1. Planning: defining objectives, selecting indicators (KPIs), and identifying appropriate data collection methods. 2. Baseline measurement: conducting ex-ante data collection to establish a reference point before intervention implementation. This step was part of the LL leader template (explained above). 3. Pilot/measure implementation: gathering ex-post data and comparing it to baseline measurements to assess impact. 4. Reflection: interpreting results and formulating recommendations based on findings. The Reflection stage will be the topic of Deliverable 6.2, which will be the Evaluation Synthesis of the DREAMS-project, so it will not be discussed here yet. Deliverable 6.2 will be published at the end of the project (late 2026). 5.4 Measurement timeline and methods This section outlines the specific data collection methods used in each stage of the measurement process, ensuring clarity in the alignment of the different qualitative and quantitative approaches used in the LLs with the overall DREAMS Evaluation Framework. As the interventions do not all have the same duration and are not scheduled to take place at the same moment across LLs, a general timeline in months cannot be given here. For this, we refer to the timeline per Living Lab in Section 6. The general stages that all LL interventions will go through in visualised in Figure 6 below. 32 positively or very positively 1,2,3, 4 Improved service in the area Finding a suitable location for a station Outcome 1 station identified per bike Qualitative methods (survey/interview/fo cus group) One-time, expost Mpact 1,2 Improved visibility Increased recognition/interes t by pilot users Output 10 users registered per bike Qualitative methods (survey/interview/fo cus group) One-time, expost Mpact Table 4: Cargobike objectives, indicators and measurement information 6.2.3 Stakeholder-based assessment Name (organisation) Stakeholder group Cambio Project-supporting Brussels Mobility Decision-maker Table 5: Stakeholders overview Brussels Objectives/local goals: Stakeholder Goals for the intervention(s) Cambio Provide feedback Share data where appropriate Provide material support Participate in workshops Provide material support (e.g. a cargo bike) Brussels Mobility Study the potential of cargo bikes for modal shift Gain insights from the sharing of Cambio vehicles via Mobitwin for the work carried out by the PRM working group within the framework of the Green Deal Inclusive Carsharing Provide a more comprehensive documentation of carsharing between individuals or between individuals and organisations Table 6: Stakeholder goals Brussels Mobitwin Baseline (please state): There is already a small number of rides (without a shared car) from the LL location + (assumption) there are already informally organised rides between neighbours Alternative A: We enhance the local antenna in the LL locations by offering shared vehicles to the volunteers. 33 Alternative B: Other transport options + other organisational options (TBD after survey) Alternative C: We do not develop the local antenna further or find local volunteers. Instead, we explore the local interest in the service. Table 7: Baseline and alternatives Mobitwin Cozywheels Baseline (please state): There is no organization that is currently sharing its fleet with locals in the LL locations. Alternative A: We explore the potential for companies in the LL locations to share their vehicles with locals outside of business hours. Table 8: Baseline and alternatives Cozywheels Cargobike Baseline (please state): There is no presence of shared cargo-bikes in the LL locations Alternative A: We set up cargo-bikes in private locations (station-based) Alternative B: We set up cargo-bikes in public locations (station-based) Alternative C: We set up free-floating cargo-bikes Table 9: Baseline and alternatives Cargobike 6.2.4 Brussels timeline The timeline of the interventions of the Brussels Living Lab is as follows: Intervention name Task Scheduled timeframe Project months Partner responsible Cozywheels Carrying out interviews with potentially interested local businesses November 2024February 2025 11-14 Mpact Mobitwin Service pilot October – November 2025 21-22 Mpact Cargobike Bike-sharing pilot September– October 2025 20-21 Mpact Table 10: Living Lab timeline Brussels 34 6.3 Living Lab Budapest 6.3.1 Short overview The Budapest Living Lab aims to provide an effective alternative to the reliance on private car usage and low-frequency public transport in the Rákosmente district of Budapest. By encouraging more sustainable transport modes, the living lab seeks to reduce the reliance on single-occupancy vehicles, thereby fostering a greener and more accessible urban environment. The living lab's core focus is strengthening connections within the Rákosmente district. Enhancing these connections facilitates better movement across neighbourhoods and promotes a sense of community among residents. With improved accessibility, residents are more likely to engage with local services, boosting the local economy and encouraging a vibrant neighbourhood atmosphere. 6.3.2 Impact assessment 6.3.2.1 Interventions MOL Bubi is Budapest's community bike-sharing system, which has only been available in the city centre until now. In the DREAMS pilot, we want to test the service in an "island operation" on the outskirts of Budapest (the planned service area is not connected to the Budapest downtown service area). • Objectives and Goals (1) Test bike sharing scheme in suburban circumstances Obj. Expected impact Indicator Output or outcome Target Measurement method Measurement frequency Data collection 1 New users for the bikesharing system nr. of new users Output 50 new users nr. of newly registered users Continuous BKK 1 Integration of the Mobi points into the urban space nr. of micromobility tools in Mobi points Output 2 tools / Mobi points personal observation of the status of the tools Ex-durante Rákosmente, BME 1 Awarenessraising on the service in the LL area nr. of local residents reached in any channels Output 500 local residents social media pages and groups, local media reports Continuous Rákosmente 1 Enhanced usage of bikesharing in the LL area nr. of trips with MOL BuBi within the district Output 100 trips data collection of bike-sharing service provider Continuous BKK duration of trips with MOL BuBi within the district Output 5 min (average) data collection of bike-sharing service provider Continuous BKK utilisation rate of the MOL BuBi bikes Output 1 usage / day data collection of bike-sharing service provider Continuous BKK bike availability within the district Outcome 1 bike / Mobi point data collection of bike-sharing service provider Continuous BKK 1 Financial viability of the service in the LL area total cost per number of users Outcome Cost breakeven nr. of registered users compared to the total cost of the pilot and calculated cost of per trip in other areas Ex-post Rákosmente, BME Table 11: Budapest objectives, indicators and measurement information 35 6.3.3 Stakeholder-based assessment Primary stakeholders + stakeholder groups: Name (organisation) Stakeholder group BKK Centre for Budapest Transport Decision-maker KTI Hungarian Institute of Transport Science and Logistics Advisory role Municipality of Rákosmente Decision-maker Bikeride association Advisory role Citizens Advisory role Table 12: Stakeholders overview Budapest Stakeholder Goals for the intervention(s) BKK Increase the modal share of sustainable transport modes Develop and test the current shared micromobility system in an outskirt district of Budapest KTI Provide better accessibility of local public transport Better capacity sharing between public transport and sustainable individual transport modes in the pilot area Rákosmente Offer alternative transport modes for citizens Increase the usage of local services Bikeride association Decrease the car dependency of the pilot area Change the transport choices of the inhabitants Citizens Have access to alternative transport modes Possibility to combine transport modes and reduce travel time Table 13: Stakeholders goals Budapest Scenarios and alternatives per intervention: Baseline (please state): There is no micromobility hub infrastructure and bike-sharing service in Rákosmente. Alternative A: Bike-sharing service in Rákosmente. Alternative B: Mobi micromobility points in Rákosmente. Table 14: Baseline and alternatives Budapest intervention 36 6.3.4 Budapest timeline Once provided, the timeline of the interventions of the Budapest Living Lab will be presented here. Intervention name Task Scheduled timeframe Project months Partner responsible Table 15: Living Lab timeline Budapest 6.4 Living Lab Munich 6.4.1 Short overview Munich’s Living Lab centers on the neighboring cities of Geretsried and Wolfratshausen in Upper Bavaria, situated approximately 43 km and 38 km south of Munich, respectively. Geretsried, the larger and more recently founded of the two, has a population of approximately 26,000 and was established after World War II as a planned settlement for displaced persons. It exhibits a dispersed urban form characterized by post-war residential developments and light industrial zones. In contrast, Wolfratshausen, with around 19,000 inhabitants, features a compact historical center with medieval street patterns positioned at the confluence of the Loisach and Isar rivers. According to the 2022 German Census, both cities exhibit relatively high proportions of older adult residents—29.31% in Geretsried and 28.64% in Wolfratshausen—as well as notable shares of migrants, comprising 19.23% and 16.54% of the population, respectively. Therefore, the Living Lab aims to develop strategies to reduce inequalities in access to essential services within 15-minute neighborhoods, focusing on supporting older adults and individuals with migration backgrounds. Consequently, it will assess the potential of emerging micro-mobility hubs and ride-sharing initiatives in collaboration with SIXT Share, E-Oberland, and MVV. As part of these efforts, it envisions developing a reward program that benefits users of local public transport and shared mobility services, enhancing the engagement and accessibility of residents. 6.4.2 Impact assessment 6.4.2.1 Interventions Within Munich’s Living Lab, two primary interventions are envisioned to be tested (it is still to be determined if they will be on-site investigations or theoretical): 1. Micro-Mobility Points: to explore the potential and feasibility of establishing local, decentralized mobility points with bus and bike/car-sharing offers to serve residents within walking and biking distance. 2. Rewards program for public transport and shared mobility: to explore the potential business of implementing a point-based system that rewards users of public transport and shared mobility services. These points collected could be redeemed for discounts on local goods or services, for example. 37 1. Micro-Mobility Points Objectives: As previously outlined, the Micro-Mobility Points initiative seeks to assess the potential and practical feasibility of establishing local-scale, decentralized mobility hubs offering bike and car-sharing services connected to local bus routes. This intervention investigates suitable site selection for these hubs based on demographic density and transport needs, the modalities of integration with existing public transport systems, and the necessary supporting infrastructure. The overarching objective is to reduce firstand last-mile mobility gaps and improve local accessibility (particularly for socially vulnerable groups, such as older adults and migrants, who often face compounded barriers due to limited digital literacy, physical mobility, and language skills). By integrating the public transport offers with shared mobility services and supporting infrastructure, this intervention aims to: 1. Identify suitable locations for the micro-mobility points based on demographic data, current infrastructure, and public transport offers. 2. Examine integration models between shared mobility services and the existing public transport network. 3. Assess usability and accessibility of micro mobility points’ interfaces and infrastructure for older adults and migrants. 4. Evaluate community uptake and satisfaction among target user groups, focusing on frequency of use, ease of access, and perceived benefits. 5. Explore the potential for replicability and scalability. The following KPIs and measurement methods are connected to the objectives: Obj. Expected impact Indicator Output or outcome Target Measurement method Measurement frequency Data collection 1, 3 Enhanced accessibility and coverage of mobility services for residents % of the area with improved accessibility Output At least 20% of the area had its accessibility improved GIS mapping and spatial accessibility analysis One-time, to define the suitable location of the mobility points TUM 2 Increase of shared mobility options’ availability Number of available shared mobility options integrated to public transport Output At least one sharedmobility service is available at the mobility point Counting of the shared-mobility services incorporated One-time, after the definition of the location of the mobility points TUM, with the support of the local partners 2, 3, 4 Increase in the usage of shared mobility options in first/last-mile connectivity Number of rides by different population groups Outcome At least 10 rides per day are done with shared mobility services Number of rides Weekly, during the intervention period MVV, SIXT, and E-Oberland 4 The community is satisfied with the micromobility points and is willing to keep using them in the long term Respondents’ satisfaction and willingness to continue using the services Outcome Most respondents indicated being satisfied and willing to continue using the services Qualitative method (e.g., survey or workshops with focus groups) One-time, ex-post TUM, with the support of the local partners 5 Deeper insight into the areas’ potential for shared-mobility business models Operational costs and revenues Outcome Cost breakeven Number of trips and revenue + operation costs One-time, ex-post MVV, SIXT, and E-Oberland, supported by TUM Table 16: Micro-mobility points’ objectives, indicators, and measurement information. 38 2. Rewards program for public transport and shared mobility Objectives: As previously outlined, the rewards program for public transport and shared mobility seeks to assess the feasibility and potential value of implementing a point-based incentive system that encourages sustainable transport modes. Under this scheme, users would accumulate points through regular use of public transport and shared mobility services (e.g., bike-sharing, car-sharing), which could then be redeemed for discounts on local goods and services, reinforcing sustainable travel behavior and support for the local economy. The overarching objective is to encourage modal shifts away from private car use, promote low-emission transport alternatives, and foster inclusive urban mobility systems that address environmental and social sustainability goals. By implementing the rewards program, this intervention aims to: 1. Assess the attractiveness and user acceptance of a points-based rewards system for public transport and shared mobility users. 2. Measure the impact of incentives on travel behavior, particularly regarding modal shifts away from private car use toward sustainable options. 3. Evaluate the social inclusiveness of the rewards scheme, with a focus on participation by older adults, migrants, and individuals with limited digital skills. The following KPIs and measurement methods are connected to the objectives: Obj. Expected impact Indicator Output or outcome Target Measurement method Measurement frequency Data collection 1 User engagement and satisfaction with the rewards system - % of users enrolled who actively use and redeem points - Satisfaction and willingness to continue using the services Outcome There are at least 50 active users in the first 3 months - System usage data (e.g., logins and reward claims) - Small surveys to assess satisfaction levels One-time, ex-post Municipalities of Geretsried and Wolfratshause n 2 Increased use of public transport and shared mobility in place of private vehicles % increase in public/shared mobility transport use Outcome 20% increase in public/shared mobility transport use Number of rides Weekly, during the intervention period MVV, SIXT, and EOberland 3 Increased involvement of migrants and older adults % of migrants and older adults using the rewards program Outcome At least 20% of registered users are migrants or older adults Demographic data from registration process of the rewards program Weekly, during the intervention period Local partners, supported by TUM Table 17: Rewards program for public transport and shared mobility’s objectives, indicators and measurement information. 6.4.3 Stakeholder-based assessment Primary stakeholders + stakeholder groups: Name (organization) Stakeholder group City of Wolfratshausen Advisory role 39 City of Geretsried Advisory role MVV (public transport operator) Project supporting SIXT-Share (shared-mobility: cars) Project supporting E-Oberland (shared-mobility: cars) Project supporting Table 18: Stakeholders’ overview Munich. In the following table, hypothetical goals (not yet confirmed with the stakeholder) have been marked in italics. Stakeholder Goals for the intervention(s) City of Wolfratshausen Provide feedback Share data when appropriate Provide supporting material Participate in the workshops Mobility of the future that combines inclusion, sustainability, and participation Improving and simplifying accessibility and mobility Sustainable urban development Switching to public transport and shared mobility offers Public acceptance and effective information dissemination City of Geretsried Provide feedback Share data when appropriate Provide supporting material Participate in the workshops Promote mobility for all, including persons with reduced mobility, language barriers, and educational impairments Strengthening active forms of mobility Reduce private car usage Reduce parking spaces Actions/interventions should have high public acceptance Bring employers on board to finance better and sustainable commuting choices (instead of the use of private cars by their employees) MVV Provide feedback Share data when appropriate Provide supporting material Participate in the workshops Improve first and last-mile connection Testing new business models for shared mobility that can be rolled out in other similarly structured areas in the MVV area Reduction in travel time through new shared mobility offers Enabling more people to choose alternatives to private car usage SIXT-Share Provide feedback Share data when appropriate 40 Provide supporting material Participate in the workshops E-Oberland Provide feedback Share data when appropriate Provide supporting material Participate in the workshops Leveraging potential in all business areas in the cross-network Creating a sensible, cost-effective alternative to a private second vehicle Reducing company car fleets in small to medium-sized companies through reliable car-sharing Increase user rate and profitability Table 19: Stakeholders’ goals Munich. Scenarios and alternatives per intervention: 1. Micro-Mobility Points in Wolfratshausen: Baseline (please state): One train station in Wolfratshausen (S7) connects to Munich, and 13 regional and 2 express bus routes connect within the city and the region (buses running every 20 minutes). The train station contains 482 bike parking spaces. Alternative A: Bike sharing is part of the mobility hub at Wofratshausen train station. Alternative B: Car sharing is part of the mobility hub at Wofratshausen train station. Alternative C: Bike and car sharing are part of the mobility hub at Wofratshausen train station. Alternative D: Bike sharing is integrated along the express bus stops in Wofratshausen. Alternative E: Car sharing is integrated along the express bus stops in Wofratshausen. Alternative F: Bike and car sharing are integrated along the express bus stops in Wofratshausen. Table 20: Baseline and alternatives for micro-mobility points in Wolfratshausen. 2. Micro-Mobility Points in Geretsried: Baseline (please state): There are 8 regional and 1 express bus routes connecting the city and the region (buses running every 20 minutes during weekdays). There are also 122 bike parking spaces distributed throughout the urban form. Alternative A: Bike sharing is integrated along the express bus stops in Geretsried. Alternative B: Car sharing is integrated along the express bus stops in Geretsried. Alternative C: Bike and car sharing are integrated along the express bus stops in Geretsried. Table 21: Baseline and alternatives for micro-mobility points in Geretsried. 41 c. Rewards program for public transport and shared mobility: Baseline (please state): No mobility rewards programs are running in Geretsried and Wolfratshausen. Operating next year: My-Radl (if you use public transport, you can ride MVV shared bikes for free) Alternative A: If you walk XX kilometers, you get an incentive (TBD). Alternative B: If you ride XX kilometers by bike, you get an incentive (TBD). Alternative C: If you have XX car-sharing rides, you get an incentive. Table 22: Baseline and alternatives for the rewards program for public transport and shared mobility . 6.4.4 Munich timeline Once provided, the timeline of the interventions of the Munich Living Lab will be presented here. Intervention name Task Scheduled timeframe Project months Partner responsible Table 23: Living Lab timeline Munich 6.5 Living Lab Paris 6.5.1 Short overview The primary aim of the Paris Living Lab is to study how 15-min accessibility within the Tram 12 corridor could be improved, through the implementation of innovative mobility solutions. This involves addressing last-mile connectivity gaps by deploying new shared mobility stations at strategic locations near key T12 stations. These stations will integrate bikes and scooters. The Living Lab also aims to improve wayfinding and user experience by upgrading signage at major stations at Évry-Courcouronnes. To analyse the adoption dynamics of shared mobility services, the project will introduce discounted memberships and free-trial offers. Discounts mainly target students, while free-trial programs will provide users with their first rides free of charge. These measures aim to reduce cost barriers and promote sustained adoption of shared mobility options. By leveraging the existing cycling infrastructure, the Living Lab will further encourage active mobility, reducing reliance on private vehicles and contributing to environmental sustainability. The Living Lab also seeks to foster collaboration among stakeholders to ensure the successful implementation and scalability of its interventions. Key partners include Pony as the shared mobility provider, the municipality of Évry-Courcouronnes, the Essonne Department, Ile-de-France Mobilités (PT authority), and community organisations. 48 1 New users for the bike-sharing system Number of new users Output Increase the uptake of e-bikes by 10% for the workers commuting by private cars in six months. DOTT app (quick question before worker starts trip) Companies disseminate small survey Once after six months UT and HU Table 31: Discounts for shared bikes objectives, indicators and measurement information 2. Design and implementation options for a pilot solar charging shared e-bike station • Short Description: design a pilot off-grid solar charging bike station that will help reduce the cost for DOTT bikes to be distributed as the company will save logistics costs (less battery swapping tasks to be conducted). • Primary objectives of the intervention: 1. decrease cost for mobility providers, leading to decreasing the cost for users and thus, increasing the uptake of shared bikes because they are offered at cheaper prices due to reduced logistics costs. 2. Reducing energy dependence of the mobility provider. Obj. Expected impact Indicator Output or outcome? Target Measuremen t method Measurement frequency Data collection 1 Reduction of battery swapping tasks Number of tasks reduced Output Reduce logistical cost by 20% Interview DOTT Once six months after intervention UT and HU 2 Reduction in grid energy dependence. Euros saved in energy cost Output Reduce energy consumption by 5% Data from energy company Once six months after intervention UT and HU Table 32: Solar station objectives, indicators and measurement information 6.6.3 Stakeholder-based assessment Primary stakeholders + stakeholder groups: Name (organisation) Stakeholder group Dott Decision-maker Utrecht Municipality Decision-maker Fietserbond Local stakeholder Community centre Local residents/ employees of companies Project O NGO and owner of the location of solar charging station Companies Owners of the companies Table 33: Stakeholders overview Utrecht Stakeholder Goals for the intervention(s) Dott Gain the mobility provider tender by reducing the cost in the outskirts and being a sustainable service. Utrecht Municipality Decrease mobility poverty Fietserbond Make cycling an inclusive option of transport Community centre Increase accessibility for different services Project O Improve opportunities for residents in the neighbourhood 49 Companies Increase healthy wellbeing of workers and reduce parking space costs. Table 34: Stakeholder goals Utrecht Scenarios and alternatives per intervention: Baseline: Current number of workers that commute by private car Intervention #1 Alternative A: 20% discount for DOTT shared e bikes for workers commuting trips Intervention #1 Alternative B: 30% discount for DOTT shared e bikes for workers commuting trips Intervention #1 Alternative C: Free DOTT shared e bikes for workers commuting trips Baseline: There is no off-grid solar charging shared e-bike station or knowledge about it Intervention #2 Alternative A: Generate the knowledge about the possible implementation of the solar charging station. Intervention #2 Alternative B: Obtain a feasible design and showcase it to the municipality to raise awareness of this solution as a viable option. Intervention #2 alternative C: Implement the solar charging station. Table 35: Baseline and alternatives per intervention 6.6.4 Utrecht timeline The timeline of the interventions of the Utrecht Living Lab is as follows: 50 Intervention name Task Scheduled timeframe Project months Partner responsible Solar charging docking system for Dott ebikes Development & design of a solar charging docking system for Dott ebikes February – July 2025 14-19 HU is leading, although TU is still the formal lead so responsible for final results of the deliverable Building & piloting the solar charging docking system September 2025 – July 2026 21-31 idem Reduction on trip price for employees of companies Interviews with companies in Overvecht – to identify needs and desires May – July 2025 17-19 idem One month campaign ‘Everyone in Overvecht Cycles to work’ September 2025 21 idem Continuation of reduced fees for companies with that interest October – April 2026 22-28 idem Table 36: Living Lab timeline Utrecht 6.7 Living Lab Vienna 6.7.1 Short overview The case study area of the Vienna Living Lab is the Wiener Flur and its surrounding neighbourhood, situated in the 23rd District known as Liesing. Liesing is located in the southwest of Vienna and is the fifth largest district in terms of area, covering 32.29 km2. The 23rd District has a relatively high average age, which is steadily increasing, and compared to the city as a whole, it has lower educational levels but a higher average income (Stadt Wien -Liesing in Zahlen). The Vienna Living Lab investigates how a flexible activity hub can contribute to the 15-minute city concept and how the development of this flexible activity hub can be linked to mobility services like bike sharing and demand responsive transport. 6.7.2 Impact assessment 6.7.2.1 Interventions In the Vienna LL, there will be two interventions tested.: 1. Flexible activity hub: To transform underutilized spaces, we plan a flexible activity hub through a co-creative approach. By reactivating open areas in Wiener Flur, these spaces can serve as dynamic hubs for (community) events, cultural offers, groceries. A co-creative setup encourages community input in designing and programming the space, ensuring it meets local needs and fosters engagement. 51 2. Bike Sharing Station: To enhance accessibility, a light bike sharing station will be established near the flexible activity hub. This station will provide a designated area for bike sharing, accessible through the WienMobil App, eliminating the need for physical docking infrastructure. By positioning the station close to the flexible activity hub, we can facilitate easy bike pick-up and drop-off, promoting sustainable urban transportation options. There is also a demand responsive transport (DRT) available in the LL area. WienMobil Hüpfer offers a demand-responsive bus service. It will be determined if the user data can be used to evaluate if the flexible activity hub has an impact on its operation. The introduction of a flexible activity hub in this area could further enhance the on-demand bus’s effectiveness. 1. Flexible activity hub With the intervention “flexible activity hub” we aim to: 1. Set up a flexible activity hub with different offers (e.g. open-air cinema, theatre, workshops) (time span mid-June until mid-August 2025) 2. Invite residents and local initiatives to get actively involved in the set-up of flexible activity hub. 3. Create ideas/ concepts for activating vacant/ underutilised spaces that involve local actors and stakeholders. 4. Assess the impact of flexible activity hubs on mobility patterns and statisfaction level in the neighbourhood. 5. Discuss and explore business models for sustainable space use that can continue to exist after the research projects have been completed. Obj. Expected impact Indicator Output or outcome Target Measurement method Measurement frequency Data collection 1 Improved access to events and offers Visitors during cultural events (cinema & theater) Output 100 visitors per event Manual visitor counting Ex-durante TU Wien/ BOKU/ Stadtland 2 Improved involvement and networking of local actors Involved organisation s/ initiatives in the preparations of flexible activity hub Output 10 organisatio ns/ iniatives involved Manual counting/ qualitative assumption One time, ex-post TU Wien/ BOKU/ Stadtland 1 Improved offers in the LL area Visitors at the flexible activity hub (e.g. during workshops) Output At least 10 visitors on days of activity Manual visitor counting Ex-durante TU Wien/ BOKU/ Stadtland 4 Improved quality of events and offers in LL area Reported feedback on quality new offers at flexible Outcome High satisfactio n rate (rating 4 out of 5) Interviews or feedback forms Ex-durante. ex post TU Wien/ BOKU/ Stadtland 52 activity hub 1, 3 Improved number of offers in the LL area Number of activities and events hosted at the hub Output 2 events/ offers per week (= 24 in total) Manual count Ex-durante, expost TU Wien/ BOKU/ Stadtland 1, 4 Improved access to events and offers Proximity for inhabitants Outcome Distance to cultural offers reduced? GIS analysis, isochrone Ex-post TU Wien/ BOKU (with support by TUM) 2? Sustainable travel mode to the hub Travel mode Outcome 80% of vistors come by foot or bike Interviews or feedback forms Ex-durante TU Wien/ BOKU/ Stadtland Tbd. Improved usage of DRT in the LL area Change in DRT use/ numbers of trips booked output Data from the DRT provider/ discussion Ex-post TU Wien/ BOKU/ Stadtland Table 37: Flexible activity hub objectives, indicators and measurement information 2. Bike Sharing Station With the intervention “Bike Sharing Station” we aim to: 1. Understand local demand for decentral bike sharing stations 2. Collect customer feedback to assess the usability 3. Exploring the potential of new bike-sharing stations’ in reducing car dependency and CO2 emissions. Obj. Expected impact Indicator Output or outcome Target Measurement method Measurement frequency Data collection 1 Increase of bike sharing usage in the LL area Number of bike dropoffs and pickups at new bike sharing station Output 5 drop-offs during events Data from the bike-sharing provider Ex-durante, ex-post TU Wien/ BOKU 1 Increase of bike sharing usage in the LL area Number of rides; Distance travelled (tbd) (proxy: pick-up and drop-off stations) Outcome Increase in number of rides during the pilot by 25%; 50% of drop offs were trips within the district Data from the bike-share provider; interview Ex-durante, ex-post TU Wien/ BOKU 2 Better understanding of the need for and quality of new bike sharing station Satisfaction with new station Outcome High satisfaction rate (4/5) Interviews or feedback forms Ex-durante, ex-post TU Wien/ BOKU 3 Shorter distance travelled to event or offers Travel time/ travel distance Outcome Decrease in distance by 50% Interviews or feedback forms (proxy: usual visited places for Ex-durante, ex-post TU Wien/ BOKU 53 cinema, theater etc.) 3 Reduced reliance on private vehicles Reduction in carbon footprint through bike usage/ walking Outcome ? Interviews or feedback forms (proxy: usual mode for to visit cultural sites etc.) Ex-durante, ex-post TU Wien/ BOKU Table 38: Bike sharing station objectives, indicators and measurement information 6.7.3 Stakeholder-based assessment Name (organisation) Stakeholder group Statdtland/ Lokale Agenda Neighbourhood initiative Morgenjungs/ imGrätzl Social business Table 39: Stakeholders overview Vienna Stakeholder Goals for the intervention(s) Statdtland/ Lokale Agenda Foster participation of local inhabitants Improve access to (outdoor) space in the neighbourhood Create (cultural) offers and activities Morgenjungs/ imGrätzl Create visibility of vacant space Promote local and cooperative (economic) activity in the neighbourhood Table 40: Stakeholder goals Vienna Flexible hub Baseline (please state): There is usually no use of the public space (in terms of cultural events and similar offers) at the LL location. Alternative A: We set up a flexible activity hub with cultural offers Alternative B: We set up a flexible activity hub with cultural offers and groceries/ 54 food-related offers Table 41: Baseline and alternatives flexible hub Bike sharing station Baseline (please state): There is no bike-sharing station at the LL location. Alternative A: We set up one bike sharing station in the LL area Alternative B: We set up two bike sharing stations in the LL area Table 42: Baseline and alternatives bike sharing station 6.7.4 Vienna timeline Once provided, the timeline of the interventions of the Vienna Living Lab will be presented here. Intervention name Task Scheduled timeframe Project months Partner responsible Table 43: Living Lab timeline Vienna 7 CONCLUSION AND NEXT STEPS While the 15-minute city concept is becoming increasingly well-known and studied in transportation research, its application and viability in urban outskirts has received less attention so far. The DREAMSproject aims to study 15-minute cities in this context. This deliverable provides an overview of the Key Performance Indicators of the project, divided per Living Lab location (located in Austria, Belgium, France, Germany, Hungary and the Netherlands). Per location, the deliverable gives an overview of the interventions planned, their expected impacts and attached indicators. It also provides a first data collection framework and timeline per location, and for the project as a whole. As the nature of the interventions varies per location, the indicator selection was done using a bottom-up approach, by asking each Living Lab leader and relevant stakeholders to reflect on what their specific intervention(s) would need. A larger, overarching set of KPIs for the entire project, that will take into the topics of 55 Accessibility, Carbon Emissions, Liveability, Health, Economic Impact, and Equity will be developed in Work Package 3, as part of Task 3.5 and Deliverable 3.2 (forthcoming), of which a first version has been included in this Deliverable. The KPIs defined in this current deliverable will be implemented and tested in the following years in the six Living Labs as part of Work Package 5 of the project. Additionally, the Deliverable also includes an explanation of the forthcoming Impact Assessment of Task 6.2. It also includes an overview of the local stakeholders, their objectives, and the scenarios and alternatives for each intervention and LL. As next steps, there will be the continuation of the bi-monthly meetings to ensure proper data collection, as well as continuing T6.2 in the coming months. Where T6.1 only entailed the setup of the DREAMS Impact Assessment Framework, T6.2 will also allow for a cross-LL comparison through the SIS analyses that will be performed in 2025-2026. Starting in the latter half of 2025, the first steps in the process evaluation will be conducted to examine how interventions are being implemented across the six Living Labs, as part of Task 6.3. This evaluation will focus on identifying enablers and barriers to implementation, understanding contextual differences, and assessing the replicability of interventions in different urban outskirts. Data will be collected through interviews, surveys, and observational methods, ensuring a robust analysis of the Living Lab dynamics. 8 ACKNOWLEDGEMENTS With thanks to the reviewers: Roman Klementschitz (BOKU) Georgia Charalampidou (BOKU) Yusak Susilo (BOKU) Heythem Adjeroud (UGE) Alain L’Hostis (UGE) We gratefully acknowledge the Brussels-Capital Region - Innoviris (Brussels Public Organisation for Research and Innovation) for financial support under grant number ‘RBC/2023-EU-DUT-11’. 9 REFERENCES Allam, Z., & Newman, P. (2022). Redefining the smart city: Culture, metabolism and governance. Springer. Bjerke, M., & Renger, R. (2017). Being smart about writing SMART objectives.. Evaluation and program planning, 61, 125-127 . https://doi.org/10.1016/j.evalprogplan.2016.12.009 C40 Cities. (2022). How to build back better with a 15-minute city. https://www.c40knowledgehub.org/s/article/How-to-build-back-better-with-a-15-minute-city Chakhtoura, C., & Pojani, D. (2015). Indicator-based Evaluation of Sustainable Transport Plans: a framework for Paris and other large cities. Transport Policy, 50, 15–28. https://doi.org/10.1016/j.tranpol.2016.05.014 Chiaradia, F., Lelo, K., Monni, S., & Tomassi, F. (2024). The 15-Minute City: An Attempt to Measure Proximity to Urban Services in Rome. Sustainability, 16(21), 9432. https://doi.org/10.3390/su16219432 56 Gillis, D., Semanjski, I., & Lauwers, D. (2015). How to Monitor Sustainable Mobility in Cities? Literature Review in the Frame of Creating a Set of Sustainable Mobility Indicators. Sustainability, 8(1), 29. https://doi.org/10.3390/su8010029 Gudmundsson, H. (2004). Sustainable transport and performance indicators. In The Royal Society of Chemistry eBooks (pp. 35–64). https://doi.org/10.1039/9781847552211-00035 Hyllenius, P., Rosqvistq, P. S., Haustein, S., Welsch, J., Carreno, M., & Rye, T. (2009). MaxSumo - Guidance on how to plan, monitor and evaluate mobility projects: Integrated project: 6.2 Sustainable Development, 1.6.2 Sustainable Surface Transport Objective, 3.1.1.1.3 Advancing Knowledge on innovative measures in urban transport. Keseru, I., Bulckaen, J., & Macharis, C. (2016). Sustainable, Participatory and Practical: The NISTO Evaluation Framework for Urban and Regional Mobility Projects. Transportation Research Procedia, 13, 134–144. https://doi.org/10.1016/j.trpro.2016.05.014 Linnan, L., & Steckler, A. (2002). Process evaluation for public health interventions and research. JosseyBass. Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., … Baird, J. (2015). Process evaluation of complex interventions: Medical Research Council guidance. BMJ, 350, h1258. https://doi.org/10.1136/bmj.h1258 Moreno, C., Allam, Z., Chabaud, D., Gall, C., & Pratlong, F. (2021). Introducing the "15-minute city": Sustainability, resilience and place identity in future post-pandemic cities. Smart Cities, 4(1), 93-111. https://doi.org/10.3390/smartcities4010006 Pozoukidou, G., & Chatziyiannaki, Z. (2021). 15-minute city: Decomposing the new urban planning eutopia. Sustainability, 13(2), 928. https://doi.org/10.3390/su13020928 Saunders, R. P., Evans, M. H., & Joshi, P. (2005). Developing a process-evaluation plan for assessing health promotion program implementation: A how-to guide. Health Promotion Practice, 6(2), 134–147. https://doi.org/10.1177/1524839904273387 te Boveldt, G., Keseru, I., & Macharis, C. (2022). When monetarisation and ranking are not appropriate. A novel stakeholder-based appraisal method. Transportation Research Part A: Policy and Practice, 156, 192– 205. htps://doi.org/10.1016/j.tra.2021.12.004 Zhang, S., Zhen, F., Kong, Y., Lobsang, T., & Zou, S. (2022). Towards a 15-minute city: A network-based evaluation framework. Environment and Planning B: Urban Analytics and City Science, 50(2), 357–375. DOI/10.1177/23998083221118570 10 APPENDICES 10.1 Appendix I. Template for LL leaders Task 6.1: Living Lab objectives and KPIs template for LL leaders 1. General Information • Living Lab: (Budapest, Brussels, Munich, Paris, Utrecht, Vienna 57 • Intervention ID/Name: (Unique name or code for the intervention, i.e. ‘Neder-Over-Heembeek MobiTwin pilot, to identify the separate interventions per LL. It can also be the address.Whatever you think works the best, but not or ‘Paris A, Paris B’ etc as this is too general.) • Short Description: (Briefly describe the intervention) • Primary objectives of the intervention: 2. Local Context and Objectives Please identify the project alternatives or options you are aware of. List all possible options for the project or indicate if only one option exists. These may include different designs, locations, policies, or approaches. This can for example be: Alternative A: shared bikes are placed in a central location. Alternative B: shared bikes and cargo-bikes are placed in a central location. Alternative C: shared bikes are placed in a central location and throughout the neighbourhood. Alternatives can also be less similar to each other: Alternative D: shared cars are placed in a central location Alternative E: we host a café for neighbours. These are always connected to a baseline: the situation without any intervention. Please fill out any alternative per baseline scenario. If you are involved in multiple interventions, you can copy + paste the table per intervention. Baseline (please state): Alternative A: Alternative B: Etc. 3. Impacts, KPIs and measurements • Baseline data collection: (Explain what type of baseline data is necessary, what it will be about, and how it will be collected) • Expected Impacts: (List 4 to 5 specific impacts, e.g., improved air quality, increased walkability, economic growth) Then fill out the rest per expected impacts in the table below: 64 65 66 67 68 10.4 Appendix IV. Poster templates Vienna – pictures of the workshop session 69 70 71 72 73 80 Here you enter the weights that indicate the importance of each factor for each stakeholder. This is usually based on input from the stakeholders themselves. Assign a score between 10 (maximum importance) and 0 (not important). A common approach is to start by identifying the most important factor and then rating the other factors relative to that one. It is possible for multiple factors to have the same weight. It is also possible that no factor receives the maximum weight. Depending on whether you selected “Yes” or “No” under “Use individual representatives for stakeholder groups?” (tab 1 ‘Start’), you will assign weights either for the stakeholder group as a whole or for each individual representative of the group. Tab: Compare Results Based on your input, this tab displays two graphs for comparing the options: Total relative impact score, breakdown per stakeholder: This shows the extent to which stakeholders are positively or negatively affected by each option. Total relative impact score, breakdown per factor: This shows the extent to which each factor contributes positively or negatively to the total effect. In both graphs, 100% represents the maximum possible (positive or negative) effect an option could have across all factors and all stakeholders. The layout of the charts can be customised. For clarity, it is advisable to use one consistent colour per stakeholder, regardless of whether the impact is positive or negative. Tabs: Results Option 1, 2, 3, … 81 Each of these tabs provides a more detailed analysis of the individual options (interventions) using two graphs that show how a specific intervention could be adjusted to address the needs or concerns of stakeholders: Absolute impact per stakeholder: This shows the impact of the option on each factor and displays the stakeholders for whom the impact is relevant. Stakeholders are shown on the horizontal axis. On the vertical axis, a score of 1 corresponds to the maximum (positive or negative) impact for one stakeholder on one factor. Therefore, the more relevant factors there are per stakeholder, the higher the potential impact score. Relative impact per factor: This shows the impact of the option on each factor and the stakeholders for whom that impact is relevant. 100% represents the maximum possible impact per factor. 82 SIS software Start Go to www.crispa-sis.eu. After signing up, choose the option ‘SIS’ at the intro screen. In the next screen (My projects), choose ‘+ Add new project’ Give your project a name, goal (optional) and choose an evaluation option, i.e., the granularity of the scale of the performance rating (step 4). 83 You then arrive in the actual SIS process. The tabs and subtabs represent the different steps of SIS. Tab: Basics Options: Enter the names of the options (interventions). Only enter the options that are not businessas-usual scenarios (minimum 1). Unlike in multi-criteria analysis, SIS results are still meaningful even if you have only one option. Stakeholder groups: Enter the names of the stakeholder groups and their members. If you only have stakeholders that are not groups with members, leave the ‘member name’ box blank. By entering the email addresses, you enable them to provide their input from their own computer. 84 Impact Factors (indicators): Enter the factors that can make an intervention beneficial or detrimental to one or more stakeholders. Aim for minimal ambiguity and overlap between factors. 85 Tab: Performances The options and impact factors you entered earlier are automatically filled in. You still need to provide the following inputs: Describe the effects of the option: In the designated cells, describe the effects of each option for each factor. How does the intervention differ from doing nothing? Be as factual as possible—use quantitative data if available, or qualitative descriptions otherwise. The description has no effect on the calculation, but you can show this screen for the sake of transparency. How positive or negative is the effect?: Indicate the degree to which the described effects are negative, positive, or neutral (compared to the do-nothing scenario), by selecting from the dropdown menus. For the calculation of impact scores, these qualitative labels (negative–positive) are converted into numerical values ranging from -1 to +1. Tab: Surveys This tab is optional. Here you can request individual members of stakeholder groups to participate in the process by email. The module allows you to select individual recipients and set expiration dates. 86 Tab: Weights Here you or the stakeholders themselves can assign weights to each of the impact factors reflecting their relative importance. Assign a score between 10 (maximum importance) and 0 (not important). A common approach is to start by identifying the most important factor and then rating the other factors relative to that one. It is possible for multiple factors to have the same weight. It is also possible that no factor receives the maximum weight. Tab: Results Under this tab you find the impact scores and various visualisations of these results. There is a subtab ‘Compare options’ and one subtab for one option each for more detail. Subtab: Compare options Based on your input, this tab displays two graphs for comparing the options: Total relative impact score, breakdown per stakeholder: This shows the extent to which stakeholders are positively or negatively affected by each option. 87 Total relative impact score, breakdown per factor: This shows the extent to which each factor contributes positively or negatively to the total effect. In both graphs, 100% represents the maximum possible (positive or negative) effect an option could have across all factors and all stakeholders. The layout of the charts can be customised. For clarity, it is advisable to use one consistent colour per stakeholder, regardless of whether the impact is positive or negative. Option-specific subtabs The option-specific subtabs allow you to see, for one option, what its (dis)advantages are and for which stakeholder they are relevant. Absolute impact score per stakeholder, breakdown by factor: This shows the impact of the option on each factor and displays the stakeholders for whom the impact is relevant. Stakeholders are shown on the horizontal axis. On the vertical axis, a score of 1 corresponds to the maximum (positive or negative) 88 impact for one stakeholder on one factor. Therefore, the more relevant factors there are per stakeholder, the higher the potential impact score. Relative impact per factor: This shows the impact of the option on each factor and the stakeholders for whom that impact is relevant. 100% represents the maximum possible impact per factor. 89 References te Boveldt, G., Keseru, I., & Macharis, C. (2022). When monetarisation and ranking are not appropriate. A novel stakeholder-based appraisal method. Transportation Research Part A: Policy and Practice, 156, 192–205. https://doi.org/10.1016/J.TRA.2021.12.004