Using the critical incident technique for qualitative process evaluation of interventions : The example of the “Let's Move It” trial

Contents lists available at ScienceDirect Social Science & Medicine journal homepage: www.elsevier.com/locate/socscimed Using the critical incident technique for qualitative process evaluation of interventions: The example of the “Let's Move It”trial Katri Kostamo a,∗ , Piia Jallinoja b , Kari Mikko Vesala a , Vera Araújo-Soares c , Falko F. Sniehotta c , Nelli Hankonen a,b a The University of Helsinki, Helsinki, Finland b The Tampere University, Tampere, Finland c Newcastle University, Newcastle, United Kingdom ARTICLE INFO Keywords: Finland Behavior change Physical activity Sedentary behavior Youth Critical incidents Qualitative analysis ABSTRACT Rationale: Trials evaluating interventions to promote health behavior change rarely embed investigations that assess participant perceptions of crucial triggers of change. Objective: The "Let's Move It”(LMI) randomized trial evaluated a theorybased whole school system intervention aiming to increase physical activity (PA) of adolescents attending vocational schools. This article serves two main purposes: to describe how to use the critical incident technique (CIT) to conduct in qualitative process evaluation to identify events, including intervention elements, which LMI trial participants perceived to enable or support behavior change. Method: Semi-structured interviews (n = 34) conducted immediately post intervention from intervention and control arms were analyzed using the CIT. Results: The analysis identified altogether 39 critical incidents. Most of the critical incidents were related to the LMI in the intervention arm and the findings are partly aligned with the LMI intervention theory. Analysis revealed several critical incidents also in the control arm, including gaining insights regarding PA and mere measurement effects, illustrating challenges facing real-world trials. Conclusion: The CIT seems a promising approach for directing analysis towards potentially crucial intervention elements as described by the participants themselves, helping in focusing and limiting the text corpus to accounts relevant to change. Qualitative evaluations in trials may add valuable understanding to complement quantitative assessments. 1. Introduction Evaluation of health behavior interventions can improve our understanding of behavior change processes as well as further develop health promotion efforts (Craig et al., 2013). Effectiveness evaluations alone are not helpful in learning what works and what does not: standalone outcome assessments mask what happens in the “black box”. This weakness of the randomized control trial (RCT) design is well-known (Deaton and Cartwright, 2018). However, process evaluations of complex interventions could explain (in)effectiveness and this understanding can be used to improve interventions (Moore et al., 2015). Although process evaluations may also involve implementation fidelity and context, the present study focuses specifically on the evaluation of mechanisms of impact. We show how to use the critical incident technique (CIT) as a key methodology in behavior change intervention process evaluations, with our analysis of the trial of the Let's Move It (LMI) physical activity (PA) school-based intervention as an illustrative example. Assessing intervention effects on the hypothesized mechanisms of change can lead to a more detailed understanding about these mechanisms and participants’interactions with intervention contents. A causal modeling approach is useful in exploring these questions: An a priori model, describing how the intervention is assumed to achieve its effects, is tested. Causal modeling permits identifying which intervention contents are essential and which are not (Moore et al., 2015). In turn, interventions can be better optimized before wider implementation. It can also contribute to theory (Rothman et al., 2006). We posit that the behavior change process can be examined by utilizing the CIT (Butterfield et al., 2005) as an analytic viewpoint. The CIT is an analysis method where critical incidents are identified from https://doi.org/10.1016/j.socscimed.2019.05.014 Received 27 November 2018; Received in revised form 10 April 2019; Accepted 14 May 2019 ∗ Corresponding author. Faculty of Social Sciences, Unioninkatu 37 (P.O. Box 54), 00014 University of Helsinki, Finland. E-mail address: katri.kostamo@helsinki.fi(K. Kostamo). Social Science & Medicine 232 (2019) 389–397 Available online 15 May 2019 0277-9536/ © 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/BY-NC-ND/4.0/). T data and categorized inductively. In previous studies, respondents have often been directly asked to name and depict important incidents (Chell and Pittaway, 1998). Instead of specific rules guiding the analysis, the CIT method allows the analysis principles to be modified case by case. Frequently, a critical incident is expected to include information considering (1) the background (how things were before the incident), (2) the experienced event, and (3) the consequences (what happened or did not happen because of the event). The results may be reported as a few general categories or several specific categories, depending on which level is assessed to be more practical (Butterfield et al., 2005). Critical incidents can be defined as events that participants identify as significant (Niska et al., 2011). Thus, behavior change may be identified as a series of such incidents that are construed as significant in interviewees’talk. As individuals differ as to the beliefs, attitudes, motivations, and existing skillsets at baseline, interventions delivered to entire cohorts need to cater for various needs. For example, whereas some individuals may already be motivated, but need a boost in their PArelated self-efficacy, others may have not found a personally meaningful reason to try to increase PA. An evaluation of idiographic change paths can shed light into the unique turning points and inspirations. The purpose of the current empirical study was to explore what kind of events low-active adolescents in vocational education perceive as promoting changes in their activity behaviors (including both PA increase and reductions of sedentary behavior, e.g., sitting) during and after the PA intervention. Instead of examining change accounts as realistic descriptions, we considered change experiences as material that humans use when they interpret reality –approaching interview talk as possible experiences (e.g., Hammersley, 2003). To our knowledge, few studies have explored those processes, events, and insights that adolescents themselves relate to PA increase during interventions. 2. Context for applying the CIT 2.1. Intervention The LMI whole school system intervention is a theory and evidencebased program developed to increase moderate-to-vigorous PA and reduce sedentary behaviors, such as sitting, among Finnish vocational students. Intervention design was based on behavioral theories relevant to youth PA, original empirical research in the target population, and co-design with end users and experts (Hankonen et al., 2016,2019). As previous literature suggested that classroom-based interventions or environmental changes alone are not sufficient in creating change, the intervention incorporated elements at both individual and environmental levels. The intervention consisted of several components, including changes to the school classroom environment, teachers' behaviors, and students’motivation and self-regulation skills. The face-toface intervention program consisted of six 60-min interactive group sessions that were included in the normal school curriculum and delivered by trained facilitators. A key feature of group sessions was an accepting, positive, and supportive environment. This part of the intervention was one part of the school system intervention that also included a poster campaign, a website, and a teacher-delivered intervention to decrease sedentary behavior in normal classrooms (e.g., activity breaks for sitting reduction), as well as changes in wider environments (Hankonen et al., 2016). The intervention was developed and optimized for targeting those with insufficient levels of PA. Table 1 shows an overview of intervention content and the linked theoretical determinants of the intervention theory. The intervention theory hypothesized that increased levels of PA will be achieved through the use of key active ingredients targeting behavioral determinants. Such active ingredients include changes in the environment and a set of conscious motivational and self-regulation processes targeted during group sessions, as well as other materials (e.g., posters). In pair and group discussions, students were encouraged to reflect upon and select personally suitable leisure time PA types and settings. This approach, including stepwise, self-guided experimentation, and prompting practice with PA was used to increase behavior and its maintenance (Hankonen et al., 2016,2019). 2.2. Trial design and data collection In a cluster-randomized trial (n= 1120 students), we evaluated the included intervention school arm and control schools following standard curriculum. Standard curriculum in the Finnish schools contains a basic course on health education and physical education. However, control participants may have taken these at slightly different times and in different forms, as the Finnish national curriculum provides freedom on how to implement this module in schools. Therefore, the form and content of the standard curriculum is not entirely uniform (compared to the precision of the LMI intervention). The co-primary outcome of the trial was total PA increase, conceptualized as changes in moderate-to-vigorous PA, overall sedentary time, and breaks in sedentary time. These were measured with sevenday accelerometry and self-report (Hankonen et al., 2016). Additionally, background and process variables, as well as secondary outcomes, were assessed by self-report in both arms, and body composition (secondary outcomes) was objectively measured using a bioimpedance device. 2.3. Aims of the present study As part of the process evaluation, this study aimed to identify intervention contents and potential mechanisms of impact that the LMI intervention school students perceived as either inspiring or crucial to increase their activity and contributing to changes in meaning-making. Targeting LMI trial participants with self-reported low or moderate PA, the study analyzed: 1. How do the students associate LMI intervention with their own PA changes, sitting reduction, and thoughts about PA or sitting? 2. When comparing intervention and control group students' accounts about critical incidents, what similarities and differences can be identified? Our aim was to evaluate participants’interpretations of important factors that they interpreted as leading to PA change and, by comparing to accounts from the control arm, to facilitate making conclusions about experiences and meaning-making during and after the intervention. The control arm was included to estimate whether critical incident categories associated with the LMI intervention were specific only to the intervention arm. Thus, the aim is also to provide researchers with a clear guidance on how to use the CIT in process evaluations of behavior change interventions. 3. Method 3.1. Study participants Individual interviews were conducted among a subsample of students from intervention and control arms (n= 34; intervention n= 21; control n= 13) immediately post-intervention (October 2015/ December 2015/March 2016/May 2016). In recruiting participants, we used survey data to identify adolescents who had reported low or moderate levels of PA at baseline. Other inclusion criteria were: (1) being a student in a vocational program, (2) voluntary signed consent for the study, (3) age: 15–19 years, (4) Finnish language skill: Self-assessed, minimum 4/5, (5) no competitive sports background, and (6) attended at least 4–5/6 LMI sessions. Participants’mean and median age was 17 years; 21 were practical nurse students and 13 hotel, restaurant, and catering students (batches 3–6 of the trial; Hankonen et al., K. Kostamo, et al. Social Science & Medicine 232 (2019) 389–397 390 2016). 3.2. Procedure All interviews were conducted using the same interview guide (see Appendix 1). There were four topics discussed with the interviewees: individual PA behavior, strategies to manage one's activity behaviors (including one question about sitting reduction), thoughts about PA, and thoughts about the LMI intervention. The interviews took place during students' free time on school premises. Participants were given an information sheet explaining the research topic and a consent form. As a compensation for their time, participants received a movie ticket. Interviews (which ranged from approximately 20 to 80 min; nine to 22 sheets as transcribed) were audio-recorded, independently transcribed verbatim, and de-identified prior to analysis. Broad and open-ended questions would be ideal to elicit storytelling, however, in this study, we used a more detailed topic guide as young participants might not be used to describing their experiences in detail. Relevant to theoretical underpinnings of the intervention, theme 2 included questions about PA self-regulation (e.g., using, planning and self-monitoring techniques) and during theme 4 (only for intervention arm), intervention participants were directly asked whether elements of the intervention had facilitated changes. Interviewers used follow-up or clarifying questions, according to a pre-specified interview strategy (Appendix 1), thus specifying the points participants had made. The ethical committee of the Hospital District of Helsinki and Uusimaa (367/13/03/03/2014) assessed the research plan and procedures. 3.3. Data analysis Transcribed interviews were analyzed using the CIT (Butterfield et al., 2005). As our aim was to identify important intervention contents, we did not limit critical incidents to only those accounts that included a detailed description of each phase in the process (i.e., the background, the event, and the consequences). In this study, critical incidents were simply defined as events associated with change in participants’accounts. The first author, KK, conducted the analysis, generating critical incidents iteratively from the data while being blinded to the quantitative outcomes. Change processes were identified, classified, and summarized, and thereafter, critical incidents were identified and categorized inductively. Table 2 summarizes the analysis process; thereafter, it is described in more detail. First, KK read and re-read all transcripts for familiarization and discussed potential issues with KMV and NH (Step 1). Secondly, all extracts describing change were marked down (Step 2). Next, a table was created to understand what was occurring in adolescents' narratives: whether the personal changes were illustrated as change in PA behavior, PA-related thoughts, sitting reduction, or thoughts related to sitting. Then the focus was on examining described changes. Each interview was summarized in own words in a table as a categorization of accounts of change in behavior and thinking as concerning background, triggering factors, consequences, and experiences or present state (Step 3). As the analysis of change processes occurs through interpretation of the interview talk, it can require understanding and reading the interview as a whole. In other words, these parts of processes may be impossible to be located or coded from a few neighboring sentences only. Based on these summaries, critical incidents were identified and presented as triggers for experienced changes and short descriptions of each incident were drafted (Step 4). All these summaries were read both as independent texts and compared to each other and the transcripts. The purpose of writing short stories is to elucidate the researchers’interpretations of the process, which makes the interpretation of the critical incident transparent. An example of a short critical incident description appears in Table 3. Next, the critical incidents were classified according to identified similarities and differences and were discussed with the team (Step 5). Table 1 Overview of intervention theory regarding student PA: Theoretical determinants and intervention activities targeting them. Theoretical determinant Intervention content linked to determinant (examples) Self-efficacy •Graded tasks (i.e., emphasis on moderate goal setting, including the principle “Any movement is good! Any activity is better than nothing”) •Verbal persuasion of capability •Skill provision Student sessions, workbook, posters, table stands Outcome expectations •Information about consequences of physical activity, framed positively, incl. principle “Any movement is good! Any activity is better than nothing”,“Sitting sucks” •Behavioral experiments (sessions and homework) Student sessions, workbook, posters, table stands Autonomous motivation (including self-concept) •Autonomy supportive style across sessions and materials •Principle “Your own choice: Whether you are active, and how”(across sessions and materials) •Emphasis on selecting personally important reasons (principle “Know what moves you”) •Emphasis on selecting autonomous goals (incl. principle “Goal is well-being, not fatless body”) •Identification of oneself as a physically active person •Behavioral experiments Student sessions, workbook, posters, table stands Descriptive norms •Information about others' behavior and attitudes toward PA Student sessions, workbook, posters, table stands Self-regulation •Goal setting, goal review •Action planning, coping planning (problem solving) •Self-monitoring Student sessions, workbook, posters, table stands Environmental opportunities in school class •Environmental changes in classroom (physical equipment) PA equipment, e.g., gym balls, standing desks, gym sticks, pilates cushions •Teacher activity in classroom (e.g., activity breaks) Teacher workshops Environmental opportunities in school •Better access to school PA facilities Improved access or improved awareness (informed via leaflets in student sessions) Environmental opportunities at home and neighborhood •Home workout videos •Better access to neighborhood PA opportunities Provision of home workout videos Improved awareness of existing opportunities at home, online, environment Arrangement of low-cost PA deals with community PA providers (informed via leaflets in student sessions) K. Kostamo, et al. Social Science & Medicine 232 (2019) 389–397 391 Finally, the analysis proceeded to a more comparative and interpretative phase. KK systematically compared preliminary findings by reference to participants’arm allocation. At first, only accounts considering the first three themes (PA behavior, behavioral self-management, and thoughts about PA) were explored, leaving the fourth theme (LMI intervention) under separate examination (Step 6.) Afterward, KK discussed preliminary results with all co-authors. Parts of the results were first discussed with KMV and NH and some amendments were made. Then, all authors reviewed the results, after which some categories were combined. 4. Results Participants were asked to talk about those changes that had happened since their latest school period started. About two-thirds of all participants described either a change in their activity, their thoughts, or both. Most reported either PA increase or maintenance. Similarly, most interpretations considering meaning-making described either no change or change towards more positive thoughts about PA and were mostly associated with PA increase. Only some described explicit changes in their sitting behavior or thoughts. Some participants talked about the intervention only when they were explicitly asked to comment on it. Theme 4 was especially designed to evoke talk about contents that participants might recall as helpful in increasing PA in the future. However, the participants illustrated some new critical incidents even in this phase. Moreover, most of critical incidents were presented during discussing PA in general (themes 1–3), and only some in theme 4. When discussing the LMI intervention (theme 4), even some of the intervention arm participants who had explicitly not described sitting change reported use of sitting reduction equipment in classrooms: “well they were nice, that you could like sit on the gym ball every now and then”or “if you weren't here I don't think that they [the teachers] would give permission to be jumping in the classroom”. These accounts were excluded from the analysis as they were not associated with PA change in participants' talk. Altogether, 31 critical incidents were identified in 14 intervention arm participants' talk (intervention n= 21) and, of these, they related 23 to the intervention. Eight critical incidents were identified in seven control arm participants' talk (control n= 13). Table 4 presents an overview of identified critical incidents. Next, we illustrate what critical incident categories intervention arm Table 2 Steps of analysis of change and critical incidents in participants’interview talk. Steps of analysis Description 1. Getting acquainted with data - Read the stories without any particular perspective. 2. Outlining data to find change descriptions - Identify extracts describing change in behavior or thinking. If the data has not been collected by asking participants directly about critical incidents, it may be challenging for the researcher to decipher what changed. In these cases, it may be helpful to look for verbs, such as “I decided”,“Someone suggested”,“Something happened”. 3. Summarizing participants' descriptions of changes with regard to outcomes In this Step, researcher creates a table that classify change descriptions according to outcome types and extracts phases of critical incidents into the same table. a) Using the extracts identified in step 2, create a table for classification of changes into the outcomes of interest, such as the behavior change targets of the intervention (e. g. change in PA behavior, PA-related thoughts, sitting reduction, or thoughts related to sitting). This helps checking whether the extract describing change refers to only one type of change, or multiple changes. Changes are usually intertwined. For example, a participant may describe the intervention triggering changes in both thoughts and intentions regarding PA and PA behavior. b) Write short process descriptions including, if possible, the following phases, and place them into the table: (1) background (e.g., Previously I did not think about sitting. I used to walk to school), (2) triggering factors, i.e., the core of the critical incident (e.g., During LMI sessions I realized that sitting is not good for me. The winter comes soon so it is dark, cold and rainy), (3) consequences (e.g., I have been thinking about sitting reduction), and (4) experiences/present state (e.g., Now I go to school by bus but hop offone stop earlier to walk more. At school I now sit on the gym ball, if it is available). In most CIT analyses only phases 1, 2 and 4 are present. Note. Depending on the research question, a critical incident can also be an event hindering change, e.g., I was going to start going to the gym in the spring (1), but then school work load was too big and took too much time (2), and in the end I couldn't start regular gym (4). In these cases, the phase 4 refers to non-change. 4. Identifying critical incidents - Write short descriptions of each critical incident, thus summarizing the phases 1–4 outlined above. See example in Table 3. These can be reported as a table, and a different table than in Step 3, because one change may associate with several critical incidents, and one critical incident may be linked to several changes. For example, in the LMI analyses, the critical incidents were classified according to (1) the trigger (intervention or some other critical incident), and to (2) outcome (e.g., thinking or behavior). 5. Categorization of critical incidents - Group the identified critical incidents. Depending on the research question, this categorising can happen in a) data-driven manner, or b) based on a theory, e.g., program theory of the intervention. 6. Comparison of change processes and critical incidents - Identify similarities and differences according to intervention and control arms CIT = Critical incident technique. LMI = Let's Move It (intervention). Table 3 An example of critical incident descriptions in the analysis. Thoughts about PA, related to PA increase; LMI and self-regulation interpreted as a critical incident When I was a kid I used to play soccer, but nowadays I do not have a sports hobby. I walk to school and climb stairs at school. At LMI meetings they gave us tips how to increase PA. I might write myself a PA plan and keep a PA diary. LMI = Let's Move It (intervention). Table 4 Distribution of identified critical incidents in participants’talk. Identified critical incidents PA increase and new thoughts about PA Sitting reduction and new thoughts about sitting Total Intervention arm participants (total): 23 8 31 Associated with the LMI intervention 17 6 23 Associated with other issues 6 2 8 Control arm participants (total) 71 8 K. Kostamo, et al. Social Science & Medicine 232 (2019) 389–397 392 participants related to the intervention content. Secondly, we present what other critical incidents they highlighted. Thirdly, we describe control arm participants' accounts about critical incidents and compare them to intervention arm participants’talk. 4.1. Critical incidents associated with the LMI intervention content in intervention arm participants’talk The analysis produced three critical incident categories: (1) gaining insight into own behavior, (2) learning self-regulatory skills and behavior change skills, and (3) getting deeper knowledge on PA and sitting. In the following, we present some examples to illustrate each of these categories. As critical incidents are frequently considered as consisting of various phases (Butterfield et al., 2005), we have striven to present comprehensive extracts of participants’accounts. 4.1.1. Category 1: gaining knowledge on PA and sitting New understanding of PA. Several intervention arm participants interpreted that knowledge about PA, acquired during group sessions, had contributed to change in their PA behavior or thoughts about PA. One participant described how becoming aware of the importance of muscular endurance, balance, and functional exercise had led to the adoption of a more varied range of exercise. I: …if you could describe in your own words how your physical activity habits have changed …you can elaborate a little. P30: Well, for example this, that I bike the fitness bike a little more, I actually got a fitness bike into my room so I have used that more, then I have done more muscle excercises whereas before I used to think that aerobic excercise is more important, but for example along with the Let's Move It, like, it has come up that you should do muscle toning too, or like have a balance, and increase the everyday physical activity, that you take the stairs and maybe walk the shorter distances and occasionally you might, what comes to mind, if the train is packed you don't have to go sit but you can stand the, like, 10-min ride, 12-min ride. Knowledge of PA benefits. Some reported that knowledge of PA benefits had increased their motivation to participate in PA. In interview talk, information about the benefits of minor moves and small changes in PA behavior was interpreted as having affected change in one's PA beliefs. I: …Do you think there was something in these sessions, something that helped, you to increase physical activity? P5: [The instructor] said that a little plus is, even a little bit of physical activity is a plus. So, it has given me a huge amount of motivation and, that thing. These extracts illustrate how cognitive change may be helpful for behavior change. If participants who do not meet PA guidelines associate PA only with vigorous exercise, they may perceive PA adoption as out of their reach. Light exercise, in contrast, may not motivate them unless its benefits are understood. Knowledge of the disadvantages of sitting. Some participants mentioned that they had not heard about the harmful effects of sitting until LMI sessions. On the contrary, other participants noted that this was not new information, however, they had not paid attention to it until the intervention. Only in-depth knowledge about sitting, acquired during the intervention, has induced actual understanding of its significance. I: Have you always thought that you should maybe reduce your own sedentary behavior a little? P31: No. I have not thought about it, except for when this Let's Move It thing came, there it was said that sitting is not that good of a thing. I have not really thought about it before. I have thought that it might not be good to be sitting down all the time, but then the Let's Move It thing was like hey this really is not good. These comments highlight the usefulness of refined health messages. General information about sitting may not be as influential as specificdescriptions of what harmful effects sitting may cause. 4.1.2. Category 2: learning self-regulatory skills and behavior change skills Practical tips for PA increase and maintenance in one's daily life. In addition to knowledge, participants illustrated the intervention's advice on various concrete ways for how PA can be increased and maintained in practice as their own PA change prompt. One participant reported getting offthe bus one stop earlier; another described stair climbing and walking short distances. Yet another reported how the intervention inspired her to do knee bends at home. I: …so it has occurred to you more that everyday activities are good for you, so did it influence your physical activity habits in any way? P33: Well maybe I have started like for example if there is a really short distance, so should you take a bus or walk, you then walk the distance and in that way I have started to think more about the everyday physical activity being good for you, it already does a lot of good if you walk and not take the bus and stuff. I: …Can you tell in more detail what was the start for this change? P33: Well maybe those lessons- …Like that even a little is good for you, or like even a small thing is good for the body and like that yeah. Practical knowledge may inspire PA increase, particularly among those who are not accustomed to regular exercise. Knowledge about strategies to reduce sitting in one's daily life. Three participants presented becoming aware of various sitting reduction strategies as a critical incident. For example, they described actual behavior change as sitting on gym balls in the classroom, standing up during lessons and going out on breaks at school. I: And was it so that now you've had thought that you could do it at school too? P14:…If teachers say that ok you can go on a break, or let's have a 5-min break to strech our leg-well I would sit in the classroom and look at the phone, but now I feel like I have to step outside even for a little while to catch some fresh air and if I have to I can take the phone with me and look at the messages there so, I have tried to go walk a little during every break. … I: Yes. Yes. Has anything happened since this period started that would have influenced your thoughts? P14: Well, these Let's Move It lessons maybe have helped to increase, or like if you are doing something increase some of these small breaks, like in school and at home where you sit and, in other ways helped too. Adopting self-regulation procedures. One participant highlighted how PA planning and self-monitoring had become a habit that helps maintain her PA routines. This participant described how her intention to begin gym exercising resulted from the intervention. Even if the program was identified as a critical incident already in the beginning of the interview, it was not until the end when she described how the intervention work book and tasks given at the group sessions prompted her to begin regular exercise. I: …how did the going to the gym happen, where did it start from? P13: Well when there was that Let's Move here then we like went through those things …nothing really prevents you from being physically active like, I have thought about going to the gym previously too but I just haven't had the energy, but now that for example there has been that Let's Move in the school, it like motivates K. Kostamo, et al. Social Science & Medicine 232 (2019) 389–397 393 me more, and then I went and started. … P13: Yes, when I started going to the gym I made myself, like, a schedule or kind of a plan. …so on Monday we do this, on Tuesday this and so forth. … I: Where did you get the idea from to make the schedule? P13: Well on these Let's Move lessons we have discussed it and that is how the planning came up. So then I thought that what if I, for example, tried for that first week, then if i feel like it or if it is going better I would keep it up. … I: Yes. Do you write them down on a piece of paper or? P13: Yes or I like, make like, like a timetable that Monday Tuesday and so on and then I write on it. In the previous extract, new knowledge was crucial for starting to exercise at home, whereas in this interpretation, PA planning and selfmonitoring have become a habit that helps to maintain those PA routines that the participant has chosen. 4.1.3. Category 3: gaining insight into own behavior A perception of freedom to make one's own choices. Two participants stressed that voluntariness emphasized in the intervention had become a motivating factor in PA. The following extracts shed light on the atmosphere in LMI sessions and how it is reflected in participants' thoughts. The objective of the intervention was seen to offer alternatives and learn to be active in a variety of self-chosen ways instead of encouraging everyone to exercise at a gym. As PA is one's own choice, the voluntariness implies freedom to choose something else as well. I: …what …was like so important that it stuck to your head when she said …about the different ways? P6: …usually people are like let's say you are overweight or something, so that they like instantly arelook at you like you should lose weight, or go to the gym …but yeah, the Let's Move It people, they did not see it like that, …they just pointed out that you can be active in different ways, …it is not like that, that we come from an angle where we try to get everyone to exercise and to go to the gym but that you can do small things and they will show us. … that it is our option whether we do or don't. That they won't judge. Instead of encouraging to conform to other people's advice or expectations, the intervention encourages to follow one's own preferences. Becoming aware of one's own sitting behavior. One participant highlighted awareness of one's own excessive sitting as a critical incident. When explicitly asked whether she has always been striving to reduce sitting, the participant answered that she had never before even thought about it. Only during the sessions had she realized that she, in fact, sits very much and that reducing sitting could be worth trying. I: Did you independently try any new lines of sports or ways to reduce sitting during the Let's Move It program? P19: Well I did not do that try-out, I forgot. Well I have, tried to sit down less but it very difficult for me. Specifically because of indolence. (laughs) And yes. I do so much by sitting down. That somehow it has been diffficult to be doing something standing up. I: Yes. Have you had it in your head nevertheless or like that you should like- (P19: yes) … P19: I have thought about having to reduce but have not yet done it. I: Yeah. Had you like had this thought like before or like that you should sit less? Was it like that orP19: Before that I did not think about it at all, that I sit so much, but when it came up on these lessons I realized that I sit a lot. Instead of reasons or means for sitting reduction in general, she described having learned something personal, that is, knowledge of her own behavior. Interestingly, this participant did not highlight this new awareness of her sitting behavior until theme 4. In the following extract, one participant illustrates PA change as a process. Here, intervention contents act as a trigger for behavior change (e.g., getting up and stretching). Having noticed how the change affects one's well-being (such as feeling better after getting up and stretching) strengthens repetition of this behavior. In interview talk, critical incidents form a developmental path that ends in paying more attention to sitting reduction. I: Yeah. Are the classes spent mostly sitting down? P30: Yes and I have noticed myself that when, that you should like stand up to do some stretches in every half an hour but you don't know how the teachers would react if you disturbed the lesson like that, since we have the forty-five minuted and then a break. I: …if you were told that you should restrict the amount of sitting in you life, what thoughts would it evoke? P30: I think it is really good because I these days I have noticed that when you reduce sitting, or rather when you don't reduce sitting, after that you have no energy and you are tired and like when you stand up then your blood starts to flow and. (laughs) Like you do a little bit of stretching and feel better. I: Yeah. Have you always thought this way? P30: Maybe not as often as now that the matter has surfaced a lot so, I have paid attention to it more. … I: …where like the cons of sitting still and stufflike that are discussed. P30: Oh, well in the Let's Move It stuffusually. (chuckles). I: Yeah. O.K. Did you have anything else …have you heard something from the media or something, or was it mostlyP30: Maybe in passing but we have never had anything, like a place or anything where we would have really digged deep into it, and properly been told about the cons and about the ways how you can prevent it and stuff. I: Was it mostly like Let's Move It (P30: yeah, yeah) -that caused it? P30: Yeah. So I did know about it beforehand, but still when I was told about it more, I went much more deeply into the matter. Also in this case, cognitive change is triggered by increasing knowledge not only about sitting, but about sitting reduction practices as well. 4.2. Critical incidents not associated with intervention content in intervention arm participants’talk The previous section demonstrated various interpretations about the intervention as triggering PA increase, sitting reduction, and thoughts about PA and sitting. However, participants may have been influenced or inspired by other sources as well. Some intervention arm participants framed LMI as a critical incident, triggering change in behavior or thoughts. Others highlighted other critical incidents, such as important others’support (school mate, boyfriend), change of season (spending more time outdoors), school change (climbing stairs, walking to school K. Kostamo, et al. Social Science & Medicine 232 (2019) 389–397 394 instead of using public transport), vocational studies (wanting to maintain the ability to work), and positive experiences during school PE lessons. The following extract illustrates the last point on the preceding list, how gaining positive PA experiences has contributed to more positive thoughts about PA. Here, being introduced to different types of PA is presented as a new understanding of PA, more specifically, that exercise can be fun and lead to positive experiences. I: …Since this period started, has something happened that has affected your thoughts about PA, and if yes then what? P31: The Let's Move It that it has like that you should be more physically active, and the school PE too when we have done all sorts of things it has become like hey this really makes one feel good, that I could do more of this and. You've gotten to try more stufflike that, well I have tried almost everything before but it has brought a new perception on all the sports, and that has been quite fun. Furthermore, one participant reported that the trial body composition measurement had helped her understand that muscle and fat in one's body may be affected. Another noted that accelerometer wearing had inspired her to be on the move more than usual, “that it would show something else beside the zeros”. Critical incidents related to changes in sitting reduction behavior or thoughts were mostly associated with the intervention. In addition, one participant said he had acquired information about sitting from school and the internet, but did not specify whether the sources were LMIrelated. 4.3. Critical incidents in control arm participants’talk Interventions do not happen in a vacuum. On the one hand, gaining new understanding of PA importance was illustrated as a critical incident among participants in both intervention and control arms. On the other hand, school change, change of season, vocational studies, positive PA experiences, and accelerometer wearing were also reported as inspiring to PA increase by participants in both arms. Control participants’critical incidents were mainly associated with transitions and circumstance changes such as weight gain,new equipment for muscular exercise at home,school change, and change of season (regular ice hockey rehearsals). Critical incidents related to new thoughts about PA were vocational studies (understanding PA as critical in maintaining work ability) and positive PA experiences (which in turn contributed to more positive thoughts about PA). In the following extract, one participant describes how she and her friends had begun gym exercising to keep fit and be able to spend some time together. Moreover, positive experiences from gym exercise have changed her previous prejudices about the gym into enthusiasm for it. I: …Yea. How did the change that you described happen, where did it start from? P26: Well, it just, my friends and I have had this thing for many summers now that now we like get into summer shape like go to the gym more, and we have never really had it in us to do it properly, but now we like thought that we like are all in different schools and see each other a lot more rarely so this is like a thing where we could all meet up, and get thelike summer shape and then that motivates more to go to the gym when you see those friends that you don't see as often anymore, so it is nicer then. …I: How has this felt? P26: Really nice, now like I used to think that the gym is kind of heavy kind of annoying, like, something that one would not like to go and do but then, now that you start and see that it is not that difficult and it is not that bad and it is healthy so, it is like nice to go there then. Some incidents in control participants' accounts were, by nature, similar to critical incidents associated with the LMI intervention content. Two of the control group students reported that gaining new knowledge (specifically, understanding that regular PA is important for one's future vocation) had increased their motivation for PA. I: Can you elaborate on what got you to think that it would be good to increase physical activity or those, make those choices of for example taking the stairs instead of the elevator or escalator? P29: Well I should improve my physical fitness because of this profession, since this is a physically demanding job after all so you should be able to endure it. I: Yeah. And how were you physically active before this change you described? P29: Almost not at all. After school I always just stayed home and didn't do anything. So nowadays I am much more active compared to what it was before. Moreover, one control participant described that survey answering and accelerometer wearing had made her think about reasons for PA. This participant also reported that she had begun to think more about sitting less. 5. Discussion To our knowledge, this study is the first to use the CIT in a qualitative process evaluation of a behavior change intervention. The CIT seems a promising approach for directing analysis towards potentially crucial intervention elements as described by the participants themselves. As illustrated above, some critical incidents were presented as natural changes related to circumstances and transitions while others formed developmental pathways and learning processes. Stewart and Smith (2014) suggest that critical incidents differ from other events by nature because they mark the beginning of a new trajectory. Examining critical incidents as markers of new trajectories may help in understanding why some adolescents related sitting on gym balls and twisters to change while others did not. Introduction of gym balls might be associated with change only when it is interpreted in connection with recently gained motivation or new insight. This study set out to examine what participants of the LMI trial in both arms, reporting insufficient PA at baseline, perceived to be critical incidents to changes in their PA and meaning-making during the twomonth intervention period. In intervention participants talk, 23 out of the identified 31 critical incidents were related to the intervention. These were categorized into three critical incident categories. In control participants, eight critical incidents were extracted. These did not substantially differ from the eight non-LMI-incidents of the intervention arm participants, indicating that both arms were subjected to similar influences from the natural, non-intervention environment. The LMI intervention was present at critical incidents in various ways: in providing new understanding of PA and a perception of freedom to make one's own choices; providing knowledge of PA benefits, practical tips for PA increase and maintenance in one's daily life, the disadvantages of sitting, and techniques to reduce sitting in one's daily life; helping in becoming aware of one's own sitting behavior; and helping in adopting self-regulation skills. The critical incident, “gaining new understanding of PA and its benefits”, was identified among participants in both arms, whereas freedom of choice, practical tips, selfregulation techniques, awareness of own sitting behavior, and knowledge of the disadvantages of sitting were unique to the LMI intervention arm participants' talk. It is possible to associate these with self-efficacy, outcome expectations, autonomous motivation, and self-regulation (see Table 1). Many changes were not related to either behavior or thinking, but both. Whereas critical incidents of intervention participants were associated with PA or sitting, those of control participants’accounts K. Kostamo, et al. Social Science & Medicine 232 (2019) 389–397 395 related to PA, with one exception (mere measurement effect; Rodrigues et al., 2015). All intervention and most control arm participants described only PA increase or maintenance. With the exception of one control participant, changes in thinking were presented as more positive or inclusive (or no change in thinking). Despite the short twomonth timeline, there were several types of critical incidents that were associated with change. The findings are partly aligned with the LMI intervention theory. Perceived freedom for own choice and benefits of light PA were highlighted in line with autonomy support principles of the LMI theory model (Table 1) and previous research. Previous literature has associated perceived benefits with PA participation (e.g., Brooks and Magnusson, 2007;Bélanger et al., 2011) as well as adopting and increasing PA (Sallis et al., 1999;Kostamo et al., 2017). Liimakka et al. (2013) categorized vocational student interviewees as target-oriented sports club adolescents, independently exercising adolescents, and adolescents who preferred hanging around with friends. Typically, independent exercisers valued the possibility to choose themselves when and how to exercise, while spending time with friends was more important in the “hanging out”category. We did not classify interviewees, but both freedom and friend considerations were present in our findings. Instead of self-regulation as a form of planning one's regular leisuretime exercise, most participants describing PA increase illustrated concrete and easy ways to adopt PA across the day. These new practices were frequently related to light PA and sitting reduction, which they conceptualized not as two separate processes, but as similar behaviors. These represent concrete examples that may be helpful when adopting new daily routines. The LMI intervention contained provision of tips, for example, on sessions 1 and 2. Further relating the findings to the intervention theory, the environmental changes were not brought up as critical incidents in participant talk. Participants seem to have construed PA as a personal responsibility, using current cultural expectations (Crawford, 2006) when describing their experiences. 5.1. Strengths and limitations There are both advantages and limitations to CIT as a method. CIT helped in focusing and limiting the text corpus to accounts relevant to change. For example, even if one student presented media criticism as an impressive intervention component, she did not explicitly associate it with change in her talk. Necessitating that the participant actively attributes the intervention to be the source of the change, the CIT uniquely sheds light on the active ingredients of this intervention as used by the participants themselves. CIT brings particular benefits that thematic analysis cannot render. Contrary to, for example, the question “how do participants believe they benefitted?”the question that the CIT is able to answer is “how do participants associate LMI with changes in their PA habits or thoughts about PA?”A good answer requires an analysis of the process (which may involve as many as four components: background, triggering factors, consequences, and experiences/ present state), which is not possible using a thematic analysis only. In addition to applying an innovative analytical methodology, a further strength of this study is the inclusion of not only intervention arm participants, but also control participants, thus allowing the comparison of the findings across trial arms, and avoiding the potential overestimation of intervention influences, which could occur in interviewing only intervention participants. Finally, the number of individually interviewed interviewees is relatively large, contributing to a likely saturated data. Change descriptions presented here are interpretative. For example, the timing of described changes (occurred before or during the intervention period) sometimes had to be concluded by the coder. Moreover, focusing on those incidents that students highlighted as meaningful for their PA change does not reveal why some intervention contents were not presented as critical. Our choice of examining PA change leaves out accounts of no change, which can also be illuminating in process evaluation. It is also possible that some students are more used to verbalizing events in their life using a narrative structure that includes critical incidents, whereas others are not as capable of linking various events. Finally, due to funding resources, it was not possible to conduct analyses immediately after the interviews, thus, we were, for example, unable to ensure data saturation or ask participants’views on our interpretations. As the interview guide included only one question about sitting reduction, the interviews produced more talk about changes in PA than changes in sitting. Also, the interview guide may have directed the analysis towards expected results, that is, the interview talk illustrating the intervention as a critical incident in theme 4. As in all interview studies, a tendency for socially desirable answers may have produced positive accounts. Nevertheless, we do not suggest that interview talk actually reveals participants’authentic experiences or changes in their cognitive structures. Instead, we stress that trial participants have been able to talk about identified critical incidents using those resources the intervention has provided. Students had only a two-month time to experience critical incidents or changes in behavior or thoughts. In previous studies using the CIT, critical incidents have often been conceptualized as crucial for the beginning of a new trajectory (Stewart and Smith, 2014), whereas our aim to identify important intervention components was more modest. However, as the participants were able to present 23 intervention-related critical incidents, it would be interesting to find out how they might describe PA changes later in life: how the presently gained resources might be realized across time (e.g., one year after the intervention). 6. Conclusions This study analyzed adolescents' accounts of critical events crucial for change in their PA behavior and related thinking at the six-week follow-up of a randomized trial. The LMI intervention was present in the intervention arm participants’critical incidents in multiple ways, indicating that various intervention contents have offered resources to them. The results illustrate that, in real-world trials, control arm participants are also exposed to other influences that may operate via similar mechanisms (e.g., offer resources for a new understanding of PA) as the intervention. The study also highlights that very detailed and intensive assessments (survey answering, body composition measurement, and accelerometer wearing) can result in increases in PA during trials, posing a threat to trial validity. Our results point to the applicability of the CIT in process evaluation of health interventions and to the added value of qualitative evaluation in shedding light on quantitative trial findings. Acknowledgements The authors would like to thank the LMI project stafffor their practical assistance, all schools and students for participation in the study, and all reviewers for their helpful comments. We would also like to thank professor Taru Lintunen for her comments on earlier versions of the manuscript. The study was supported by the Finnish Ministry of Education and Culture (grant number 81/626/2014). The first author was supported by Finnish Cultural Foundation and Faculty of Social Sciences in the University of Helsinki (joint funding for doctoral students, grant number 00180568). NH was supported by the Academy of Finland (grant number 285283). The funding bodies played no role in the writing of this manuscript or the decision to submit it for publication. Appendix A. Supplementary data Supplementary data to this article can be found online at https:// K. Kostamo, et al. Social Science & Medicine 232 (2019) 389–397 396 doi.org/10.1016/j.socscimed.2019.05.014. References Bélanger, M., Casey, M., Cormier, M., Filion, A.L., Martin, G., Aubut, S., et al., 2011. Maintenance and decline of physical activity during adolescence: insights from a qualitative study. Int. J. Behav. Nutr. Phys. Act. 8. ijbnpa.biomedcentral.com/ articles/10.1186/1479-5868-8-117. Brooks, F., Magnusson, J., 2007. Physical activity as leisure: the meaning of physical activity for the health and well-being of adolescent women. Health Care Women Int. 28, 69–87. Butterfield, L., Borgen, W.A., Amundson, N.E., Maglio, A.-S.T., 2005. Fifty years of the critical incident technique: 1954–2004 and beyond. Qual. Res. 5 (4), 475–497. https://doi.org/10.1177/1468794105056924. Chell, E., Pittaway, L., 1998. A study of entrepreneurship in the restaurant and cafe' industry: exploratory work using the critical incident technique as a methodology. Int. J. Hosp. Manag. 17 (1), 23–32. Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M., 2013. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int. J. Nurs. Stud. 50 (5), 585–592. https://doi.org/10.1016/j.ijnurstu. 2012.09.010. Crawford, R., 2006. Health as a meaningful social practice. Health: Interdiscipl. J. Soc. Stud. Health Illness Med. 10 (4), 401–420. Deaton, A., Cartwright, N., 2018. Understanding and misunderstanding randomized controlled trials. Soc. Sci. Med. 210, 2–21. Hammersley, M., 2003. Recent radical criticism of interview studies: any implications for the sociology of education? Br. J. Sociol. Educ. 24 (1), 119–126. https://doi.org/10. 1080/01425690301906. Hankonen, N., Heino, M.T.J., Araújo-Soares, V., Sniehotta, F.F., Sund, R., Vasankari, T.,., Haukkala, A., 2016. 'Let's Move It' - a school-based multilevel intervention to increase physical activity and reduce sedentary behaviour among older adolescents in vocational secondary schools: a study protocol for a cluster-randomised trial. BMC Public Health 16, 451–466. https://doi.org/10.1186/s12889-016-3094-x. Hankonen, N., Absetz, P., Araújo-Soares, V., 2019. Changing activity behaviors in vocational school students: the stepwise development and optimized content of the ’Let's Move it’intervention. https://psyarxiv.com/ak68f, Accessed date: 4 April 2019. Kostamo, K., Vesala, K.M., Hankonen, N., 2017. Leikistä harrastukseen, hyötyyn ja pelailuun: liikunnan merkitysten muutokset nuorten kilpakirjoituksissa. [From game to hobby, benefit and play: meanings of physical activity and its change in competitive writings of adolescents]. Psykologia 52 (1), 23–37. Liimakka, S., Jallinoja, P., Hankonen, N., 2013. Liikutaan ja/vai hengaillaan? Liikunta ja kaverit ammatillisessa oppilaitoksessa opiskelevien nuorten elämänkokonaisuudessa. [Exercising and/or hanging out? Physical activity and friends in the lives of adolescents in vocational schools]. Liikunta Tiede 50 (6), 32–39. Moore, G.F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W.,., Baird, J., 2015. Process evaluation of complex interventions: medical Research Council guidance. BMJ Br. Med. J. (Clin. Res. Ed.) 350 doi:ARTN h125810.1136/bmj.h1258. Niska, M., Hynynen, S.-T., Vesala, K.M., 2011. Yrittäjien menestystarinat näkökulmana yrittäjyyteen ja sitä edistäviin tekijöihin (Entrepreneurs' success stories as perspective to entrepreneurship and its promotion). In: Vesala, K.M., Vihinen, H. (Eds.), Yristysten menestystarinat ja yrittäjyyden edistäminen. Tapaustutkimus yrittäjistä, kehittäjistä ja politiikkadiskursseista Suupohjan seutukunnassa, pp. 49–93 MTT raportti 16. Rodrigues, A.M., O'Brien, N., French, D.P., Glidewell, L., Sniehotta, F.F., 2015. The question–behavior effect: genuine effect or spurious phenomenon? A systematic review of randomized controlled trials with meta-analyses. Health Psychol. 34 (1), 61–78. https://doi.org/10.1037/hea0000104. Rothman, A.J., Bartels, R.D., Wlaschin, J., Salovey, Peter, 2006. The strategic use of gain‐ and loss‐framed messages to promote healthy behavior: how theory can inform practice. J. Commun. 56, S202–S220. https://doi.org/10.1111/j.1460-2466.2006. 00290.x. Sallis, J.F., Calfas, K.J., Alcaraz, J.E., Gehrman, C., Johnsons, M.F., 1999. Potential mediators of change in a physical activity promotion course for university students: project GRAD. Ann. Behav. Med. 21 (2), 149–158. Stewart, B.S., Smith, A.C.T., 2014. The significance of critical incidents in explaining gym use amongst adult populations. Qual. Res. Sport Exerc. Health 6 (1), 45–61. K. Kostamo, et al. Social Science & Medicine 232 (2019) 389–397 397