Counteracting subliminal cues that threaten national identity

British Journal of Social Psychology (2021) ©2021 The Authors. British Journal of Social Psychology published by John Wiley & Sons Ltd on behalf of British Psychological Society www.wileyonlinelibrary.com Counteracting subliminal cues that threaten national identity Jolien A. vanBreen 1 *, Soledad deLemus 2 , Russell Spears 3 and Toon Kuppens 3 1 Universiteit Leiden, the Netherlands 2 Universidad de Granada, Spain 3 Rijksuniversiteit Groningen, the Netherlands In spite of their subtle nature, subliminal cues of group devaluation can have profound effects on members of targeted groups. Across three studies, we examine factors that allow people to counteract subliminal cues of group devaluation. We do this in the context of Spanish–German intergroup relations following the 2008 financial crisis. Throughout the crisis, narratives in politics and the media have drawn on national stereotypes to legitimize the economic situation in Spain. We argue that this represents a threat to our Spanish participants and that exposure to subliminal cues that reflect this threat will trigger responses that counteractthis threat. Indeed, results showed that when subliminal associations legitimize the disadvantage faced by the group, our Spanish participants reversed the subliminal associations to which they were exposed. These findings show that Spanish participants are able to counteract the devaluation of their national in-group, even when that devaluation occurs outside of conscious awareness. There is evidence that, in many Western societies, social devaluation of disadvantaged groups is becoming increasingly subtle (Pearson, Dovidio, & Gaertner, 2009; Swim, Aikin, Hall, & Hunter, 1995). Changes in societal norms mean that it is increasingly unacceptable to express prejudicial attitudes explicitly (Crandall & Eshleman, 2003; but see Betz & Johnson, 2004), but the fact that these attitudes are not expressed explicitly does not mean they have disappeared. In fact, processes that contribute to social devaluation, such as stereotyping, can occur subliminally (Blair, 2002; Ca~ nadas, Rodr ıguez-Bail on, Milliken, & Lupi a~ nez, 2013; Devine, 1989). That is, behaviour and cognition can be significantly biased without either the perpetrator or the victim being consciously aware of it. In this paper, we examine the coping mechanisms people employ to deal with subliminal cues of devaluation facing their group. Even when it is very subtle, social devaluation has far-reaching effects on interactions. On the side of the perpetrator, seminal work by Devine (1989) showed that subliminal exposure to race-related words activates stereotype constructs, which then biased participants’ impressions of novel individuals in stereotype-consistent ways. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. *Correspondence should be addressed to Jolien A. van Breen, Universiteit Leiden, Leiden, the Netherlands (email: [email protected]). DOI:10.1111/bjso.12474 1 These processes also affect members of target groups: Black participants showed poorer performance after having interacted with White partners who hold implicit anti-Black biases (Holoien & Shelton, 2012). These findings suggest that the subtlety of subliminal devaluation cues belies the harmful effect they have on those who are targeted by them. In light of this, considerable research has examined how people cope when they are exposed to subliminal cues that devalue the groups to which they belong. On the one hand, there is evidence that –as cues of group devaluation become subtler –they are more difficult to recognize and difficult to confront directly (Kray, Thompson, & Galinsky, 2001; Major, Quinton, & Schmader, 2003). That is, when cues are very subtle, or even subliminal, members of the target group seem to have little choice but to assimilate. More recently, however, there is growing evidence that people can counteract subliminal cues of devaluation. This is in part due to increasing recognition that there are many different strategies by which members of the target group can counteract group devaluation, including a host of indirect and subtle strategies in addition to the more well-known such as protest and interpersonal confrontation. For instance, it has been shown that members of disadvantaged groups can counteract that disadvantage through their behaviour. Crisp, Bache, and Maitner (2009) asked engineering students to complete a math task which ostensibly reflected gender differences and found that the female students performed better than their male peers. We might interpret this as counteracting the stereotype – women demonstrate that the stereotype is untrue through their own counterstereotypical behaviour. Further, tendencies to counteract group-based devaluation can be expressed on implicit measures, without the need for conscious awareness. Ramos et al., (2016) exposed women and men to sexism and showed that following this exposure, female participants in particular showed evidence for a reduction in implicit gender bias. Rather than going along with the sexism to which they were exposed, they instead opposed it (see also de Lemus, Spears, Bukowski, Moya, & Lupi a~ nez, 2013; de Lemus, Spears, Lupi a~ nez, Bukowski, & Moya, 2018). Likewise, following social threats, implicit mechanisms can be drawn on to buffer self-esteem (Leitner & Forbes, 2015). The notion that tendencies to counteract devaluation can be expressed in implicit measures is supported by developments in the literature on implicit cognition, which has shown that motivational processes can operate outside of conscious awareness (Custers & Aarts, 2010; Moskowitz, Li, Ignarri, & Stone, 2011). In sum, then, there is increasing evidence that the desire to counteract and oppose the devaluation of one’s in-group can be expressed in subliminal or implicit processes. However, the work discussed above does not speak directly to cases where the threat itself is conveyed subliminally. Recent research from our own laboratory directly examined the question whether tendencies to counteract in-group devaluation can take place when exposure to devaluation occurs subliminally (van Breen, Spears, Kuppens, & de Lemus, 2018). This work showed that exposure to subliminal cues of devaluation triggers responses designed to counteract that devaluation. Following subliminal exposure to gender stereotypes, feminist women showed counter-stereotypical behaviour, and the subliminal prime ‘woman’ facilitated the recognition of positive (rather than negative) words. That is, instead of ‘going along’ with the negative and stereotypical representation of the group, feminist women reversed the associations to which they were exposed (van Breen et al., 2018). In sum, it seems that it is possible to counteract subliminal cues of devaluation. The current work examines some of the conditions necessary to allow people to counteract subliminal cues of devaluation. We argue that people are able to counteract subliminal forms of devaluation if such cues tap into a salient threat to the group. Research has shown that subliminal social cues 2Jolien A. van Breen et al. become much more consequential when they do not stand alone, but rather tap into an issue that is meaningful to the individual (Amir, Foa, & Coles, 2000; Strahan, Spencer, & Zanna, 2002). Threat is a powerful motivational force in both social and non-social contexts and can be detected even when conveyed through subliminal cues (Amir et al., 2000; Kaiser, Vick, & Major, 2006; McNally, Amir, & Lipke, 1996). For instance, it has been shown that subliminal prejudice cues have a greater effect on those who expect to be the target of prejudice (Kaiser et al., 2006). As such, exposure to subliminal cues might still produce relatively sophisticated responses, by triggering strong motivating factors such as threat. We argue that people are more able to counteract subliminal devaluation cues when those cues serve as reminders of an existing threat to the group. We examine this idea in the context of Spanish national identity following the 2008 financial crisis. The 2008 financial crisis in Spain The financial crisis has had clear effects across the European Union, but particularly so in Southern European countries, including Spain, but also Greece, Italy, and Portugal. Germany (alongside the Netherlands and the UK) is perceived to have played an important role in enforcing far-reaching austerity in Southern Europe (see van Hecke, 2017; Sierp & Karner, 2017). In the context of the 2008 economic crisis, then, Germany is a salient outgroup to our Spanish participants. The context of Spanish–German intergroup relations is particularly suited to examine the central hypothesis of this study –that even subliminal cues of devaluation can be counteracted if they tap into particularly threatening elements of the devaluation context. Our design requires different kinds of negative group representations which differ in the level of threat they represent. Normally, this would require different contents –so that some content is more threatening than other content. However, in the Spanish–German intergroup context, we are able to vary the threatening nature of the associations with only minimal variations in content, as explained below. Since the onset of the financial crisis, Spanish–German intergroup relations have been characterized, first, by considerable economic disadvantage for Spain. In addition, the intergroup context is characterized by national stereotypes (Linssen & Hagendoorn, 1994; Pennebaker, Rim e, & Blankenship, 1996; Willis & Rodr ıguez-Bail on, 2008). When stereotypes are used in a purely descriptive sense, they are not necessarily problematic, because they provide a sense of ‘what it means’ to be a group member (G omez, Seyle, Huici, & Swann, 2009). Indeed, previous research has shown that Spanish participants are quite willing to endorse national stereotypes of their own group and do not necessarily perceive this negatively (Morales, Garc ıa, Rodr ıguez-Bail on, & Moya, 2004). Crucially for the current study, however, stereotypes can also serve a legitimizing function –that is, stereotypes can be used to provide a justification for the status hierarchies within society and the inequalities these produce (Reyna, Henry, Korfmacher, & Tucker, 2006; Weiner, 1995). Indeed, this seems to be what has occurred in the context of the financial crisis: Narratives in politics and the media have attempted to explain Spain’s disadvantage with reference to national stereotypes (see Sierp & Kanner, 2017). For instance, it has been suggested by newspaper commentators that the economic situation in Spain was due to ‘poor work ethic’ (Bloom, 2015; Brooks, 2011; Friedman, 2011). This is a powerful way to legitimize inequality, because the traits invoked by stereotypes are perceived as causal factors in producing a group’s outcomes (Kressel & Uleman, 2015; Reyna et al., 2006). In other words, stereotypes can be used to assign blame and responsibility for disadvantage to the disadvantaged group (Capucha, Est^ ev~ ao, Calado, & Capucha, 2014; Weiner, 1995). Counteracting subliminal threat cues 3 At the time when these studies were conducted, this supposed legitimacy associated with the economic crisis formed a very salient threat to national identity within Spain (Becker, Wagner, & Christ, 2011; Bukowski, de Lemus, Rodriguez-Bail on, & Willis, 2017). The salience of this threat can be seen for instance in the fact that people began to retaliate against such narratives in the media (as can be seen from this Twitter thread). In sum, the context of the financial crisis in Spain is associated with a form of identity threat that arises not from references to the financial crisis per se or from references to stereotypes per se, but rather their combined effects, whereby stereotypes are used to imply that the disadvantage faced by the in-group is legitimate. The current studies Across three studies, we expose Spanish participants to subliminal cues that reflect negatively on the national in-group. We expect that participants will counteract those cues that make the in-group responsible for the disadvantage they face. Participants are expected to be especially sensitive to such cues, because –even though they are subliminal –they tap into a salient dimension of threat in people’s daily lives. To manipulate subliminal exposure to cues of group devaluation, we use a priming procedure. That is, participants are exposed to subliminal in-group or out-group primes (Spanish/German) that are paired with supraliminal target words 1 . A brief note is required here to clarify the terminology used. Throughout this paper, we use ‘subliminal’ to describe information that is presented with very brief presentation times, to be processed outside of conscious awareness. When describing responses and outcome measures, we use ‘implicit’ rather than ‘subliminal’ because the latter is a perceptual term and is not appropriate when discussing responses. All studies reported here were conducted in line with APA ethical guidelines, as well as institutional ethics requirements. STUDY 1 This study examines the notion that people can counteract subliminal forms of group devaluation, when those cues draw on (threatening) notions of blame and legitimacy. We use a priming procedure to expose Spanish participants to subliminal primes paired with supraliminal target words. Across conditions, these prime-target pairings reflect negatively on the in-group. In the stereotype condition, participants are subliminally exposed to low-competence stereotypes of the in-group (Spanish-lazy). In the disadvantage condition, participants are subliminally exposed to reminders of the financial crisis (Spanish-poor). In the implied legitimacy condition, these subliminal associations are combined, so that the in-group disadvantage is legitimized through the use of stereotypes (Spanish-lazy; Spanish-poor). Finally, we include a control condition that exposes participants to positive in-group associations (Spanish-efficient; Spanish-rich). We expect that participants in the implied legitimacy condition will counteract the negative representation of the group. To measure these tendencies, we include, first, an evaluative decision task. Participants are asked to categorize words as positive or negative. 1 Recently, there has been some controversy over the notion of priming (Doyen et al., 2012; Shanks et al., 2013). In light of this debate, it is worth emphasizing that in our case, the subliminal priming manipulation functions somewhat differently than traditional manipulations of this sort. Most importantly, our reasoning and predictions do not require any new associations or preferences to be established. Rather, our manipulation can function as a subliminal reminder of the stigmatizing messages to which people are exposed in their daily lives (except in the control condition, as outlined below). 4Jolien A. van Breen et al. Each word is preceded by a subliminal prime, in this case ‘Spanish’ or ‘German’. The ingroup and out-group primes are presented subliminally, as they are during the manipulation. The outcome of interest is the influence of the subliminal prime (Spanish/German) on the speed with which people recognize positive and negative words. As such, this task enables participants to counteract the manipulation (which represents the group negatively) by more readily associating the in-group with positive words. We expect that (H1) after being exposed to the implied legitimacy condition, ingroup primes (vs out-group primes) facilitate the recognition of positive targets. That is, we expect to observe a 3-way interaction between the manipulation, the target type (positive vs negative), and the prime type (in-group vs out-group). Second, we include a math task, which captures tendencies to counteract the manipulation through behaviour. Given that the manipulation represents the in-group as low in competence, we expect that people will counteract this by reasserting their competence. Specifically, we expect that following the implied legitimacy condition, people will (H2) show increased persistence in the math task (Nussbaum & Steele, 2007; van Breen et al., 2018), relative to the other conditions. The subtle nature of the outcome measures we use provides a degree of ‘structural fit’ with the subliminal manipulation (Payne, Burkley, & Stokes, 2008). Put differently, a subtle manipulation is likely to produce subtle effects. Following this reasoning, we do not expect effects of our subliminal manipulation on more explicit coping responses. To generate some exploratory insight into this issue, we include three measures of more explicit coping responses, as a contrast to the more subtle outcome measures. We include a measure of in-group bias, a mood scale, and a measure of collective action –these are described in the Appendix S1. Finally, based on previous research (Ellemers, Spears, & Doosje, 1997; van Breen et al., 2018), we examine the role of in-group identification in our findings. Method Design & procedure Upon arrival at the laboratory, participants read the study information and provided informed consent. Participants were randomly assigned to one of the four experimental conditions and completed the manipulation, followed by the central dependent variables: the evaluative priming measure and the math task. Participants then indicated their nationality and their identification with the national group. Participants then completed the exploratory measures (see Appendix S1). To end the study, participants completed a funnelled debriefing and were thanked for their participation. Power Previous research using a similar manipulation and dependent variables (van Breen et al., 2018) found effects of d=0.35, with a lower bound of d=0.17. As such, we wanted to ensure that our sample can detect effect sizes of around d=0.15. The evaluative priming measure (described below) has a multilevel data structure, with multiple observations per participant. Power analysis using G*Power (Faul, Erdfelder, Lang, & Buchner, 2007) does not allow estimation for multilevel models, but does offer estimation for a hierarchical RM ANOVA, which we use here. Given that this analysis does not fully incorporate the multilevel structure, it is likely to yield a relatively conservative estimate. Given a=0.05, 132 participants are required to detect simple effects of a small size (d0.15) with a Counteracting subliminal threat cues 5 power of 1-b=0.80. Based on this, we decided to collect a minimum of 132 participants. Maximum participant numbers were determined by the number of participants that could be recruited within a 2-week period. The additional participants provided a buffer in case not all participants met the inclusion criteria (i.e., Spanish nationality). We then examined what a sample size of N=132 would mean for the math task, which did not have a multilevel structure. A sensitivity analysis for these measures showed that, given a=.05 and a power of 1-b=.80, a sample of 132 participants can detect effects in the range of d0.48. Participants Undergraduates from the University of Granada (N=163) completed the study. Those who did not have the Spanish nationality (N=13) were excluded. We further excluded those who failed to comply with instructions or who had high error rates (>20%) during the manipulation (N=9). This left a total of 141 participants (23 men; 16%), roughly equally distributed over the conditions (N control =35; N disadvantage =34; N stereotype =38; N implied_legit =34). The average age was 20.50 years old, ranging from 18 to 45 years old. Manipulation The subliminal associations to which participants were exposed were manipulated by the repeated pairing of subliminal in-group and out-group primes with supraliminal target words. The subliminal primes were ‘Spanish’ (in-group) and ‘German’ (out-group). The supraliminal target words were selected based on a pre-test (see Appendix S1). We selected 10 low-competence traits rated as stereotypical for the Spanish in-group, and 10 high-competence traits that were rated as stereotypical for the German out-group, as well as 10 nouns reflecting economic disadvantage, and 10 nouns reflecting economic advantage. These primes and targets were combined to create four different conditions. In the stereotype condition, participants were exposed to associations reflecting lowcompetence stereotypes of the in-group and high-competence stereotypes of the outgroup, such as ‘Spanish-lazy’ and ‘German-efficient’ (see Linssen & Hagendoorn, 1994; Pennebaker et al., 1996). The disadvantage condition subliminally associated the ingroup with targets relating to the economic disadvantage, such as ‘Spanish-debt’, while subliminally associating ‘German’ with ‘credit’ and ‘wealth’. In the implied legitimacy condition, ‘Spanish’ was associated with both economic disadvantage and lowcompetence stereotypes (i.e., ‘Spanish-lazy’; ‘Spanish-debt’) and ‘German’ was associated with both economic advantage and high-competence stereotypes (‘German-efficient’; ‘German-wealth’). This condition combines the associations used in the stereotype and disadvantage conditions. The fourth condition was a control condition, which was the same as the implied legitimacy condition except that Spanish and German primes were switched, such that ‘German’ was now associated with low competence and economic disadvantage, and ‘Spanish’ was associated with high competence and economic advantage. The associations participants saw in this condition then do not map onto any specific threat in participants’ daily lives, as they represent the in-group positively. The purpose of this control condition was to match the implied legitimacy condition in terms of complexity (combining 2 types of associations). The manipulation consisted of 120 trials, in which the subliminal prime (‘Spanish’ or ‘German’) was presented for 42 ms, with a supraliminal forward and backward mask 6Jolien A. van Breen et al. presented for 100 ms. Following the masked prime, the target appeared. The prime and masks appeared as a string of letters flashing on the screen, and the task instructions informed them: ‘You will see some letters flash on the screen as the computer selects the target word from a list. Once the word appears, please answer the question below.’ Participants were then asked to answer a question about the target word, to encourage deeper processing. Responses to the question were not analysed. In-group identification Based on previous research (de Lemus et al., 2013; van Breen et al., 2018), we included in-group identification as a covariate. Participants completed the multidimensional identification measure (Leach et al., 2008, N=14, a=.93) using a 9-point Likert scale. Dependent measures Evaluative priming measure. We included an evaluative priming measure (Fazio, Jackson, Dunton, & Williams, 1995) to examine the positive and negative associations people make with the in-group and out-group. Participants saw positive or negative target words (supraliminal), preceded by subliminal in-group or out-group primes. The subliminal primes were ‘Spanish’ or ‘German’, as in the manipulation. The targets were positive or negative words without stereotypical connotations –such as ‘love’ or ‘peace’ – taken from the Spanish translation of the IAT measure (see Rodr ıguez-Bail on, Ruiz, & Moya, 2009). Participants were asked to classify these targets as positive or negative, as quickly as they could. There were 4 different trial types depending on the prime-target pairing: in-group positive pairs, in-group negative pairs, out-group negative pairs, and out-group positive pairs. The task included 120 trials in total. The outcome of central interest is the speed with which people can recognize positive and negative words, and the influence of the primes on response times. Table 1 summarizes the subliminal and supraliminal components of Study 1. Table 1. Overview of the priming procedures Example of associations Subliminal prime (IG/OG) Supraliminal target Manipulation ‘Spanish’ Lazy Stereo condition; Legit condition ‘German’ Productive Stereo condition; Legit condition ‘Spanish’ Poor Disadv condition; Legit condition ‘German’ Rich Disadv condition; Legit condition ‘Spanish’ Rich Control condition ‘German’ Lazy Control condition Eval priming measure ‘Spanish’ Applause ‘German’ Applause ‘Spanish’ Tomb ‘German’ Tomb Counteracting subliminal threat cues 7 Persistence and performance. We included a math task, as a measure of persistence and performance in a competence domain to assess the desire to counteract devaluation through behaviour (Crisp et al., 2009; van Breen et al., 2018). The task consisted of 8 math problems in increasing order of difficulty. Participants were asked to choose the correct answer from 4 options. If they did not know the answer, participants could choose ‘skip this question’. For the final item, the correct answer was not amongst the options, so that the item was unsolvable, yielding a measure of persistence –that is, the amount of time participants spent on the item before finally pressing ‘skip’. Participants might counteract in-group stereotypes of low competence by persisting on the unsolvable item. Analytical procedure For the evaluative decision task, the simple comparison of central interest is the effect of the in-group prime versus the out-group prime on reaction times. We argue that after exposure to the implied legitimacy condition, subliminal in-group primes facilitate positive responses, relative to out-group primes. To evaluate evidence for this prediction, we fit a multilevel model including a random intercept for each participant, which reflects the multilevel structure whereby trials are nested within participants. For the math task, we examine the effect of the manipulation on persistence and performance in the task. We argue that after exposure to implied legitimacy condition, participants will persist for longer and perform better, as a way of ‘disproving’ the negative image of the group to which they were exposed. We evaluate evidence for this prediction by fitting an ANOVA model, including the manipulation, group identification, and their interaction. Preliminary analyses It is common for reaction time (RT) data to be significantly skewed, as reaction times are constrained at the lower end of the scale, but not at the higher end of the scale. Indeed, the RT data for the evaluative decision task were negatively skewed (skewness =15.96, kurtosis =506.90). We took the following steps to address this issue. First, based on previous research, we considered that any response below 300 ms is an anticipation (Ratcliff, 1993), and these were excluded. At the higher end, we excluded the most extreme 10% of observations. That is, we chose an ‘objective’ cut-off rather than a databased cut-off (Whelan, 2008). This approach was preferred because the same cut-off can be applied to any subsequent studies. Applying this cut-off to the data meant that any RTs above 900 ms were excluded, resulting in a distribution that was close to normal (skewness =0.48; kurtosis =0.016), with a median RT of 593 ms. Preliminary analysis established that group identification could be used as a covariate, as it was not affected by the manipulation, F<1, p=.786. Further, preliminary analysis of the evaluative priming measure confirmed the existence of a random Participant factor in the evaluative priming task, Wald’s Z=7.64, p<.001, reflecting the multilevel structure whereby trials are nested within participants. Therefore, the multilevel model described below includes a random Subject factor. Pilot study We argue thatthe tendency toassociate in-group primes (more than out-group primes)with positive words represents a coping response. However, previous work has demonstrated 8Jolien A. van Breen et al. that under some circumstances, positive group associations are the default response (e.g., Otten & Moskowitz,2000). Toobtain some insight into this possibility, we conducted a pilot study with N=24 participants, who were asked to complete the evaluative priming task in the absence of any manipulation. Analysis of this ‘baseline’ demonstrated that –in the absence of a manipulation, there was no evidence that the subliminal primes affected reaction times to positive and negative words, F(1,2574) =1.00, p=.317. However, there was a main effect of target category, so that positive words elicited faster responses (M=599) than negative words (M=610), F(1,2574) =9.28, p=.002. Results Evaluative priming task Our hypothesis is that after exposure to the implied legitimacy condition, subliminal ingroup primes (relative to out-group primes) facilitate the recognition of positive target words. The full model is shown in Table 2. The omnibus analysis showed a main effect of target valence: Positive targets (M=605 ms) elicited faster responses than negative targets (M=613 ms). The hypothesized 3-way interaction between the manipulation, target valence, and the group prime also reached significance, as did several of the lower order terms (see Table 2). There was no evidence for a further interaction with in-group identification, F<1, P=.316. Table 3 shows the breakdown of the 3-way interaction into simple effects –yielding one significant simple effect. After exposure to the implied legitimacy condition, positive targets were recognized more quickly when preceded by in-group primes (M=611 ms) relative to out-group primes (M=621 ms), supporting hypothesis 1. Figure 1 represents the interaction graphically. Persistence and performance We assessed whether people similarly counteracted the implied legitimacy condition through their behaviour, specifically through persistence and performance on a Table 2. Full results for the evaluative priming measure in Study 1 Fixed Effect F-value df (denominator) p-value Intercept 18544.51 144 0.000 Manipulation [0 =control; 1 =stereo; 2=disadv; 3 =implied legit] 0.49 141 0.687 Prime [0 =in-group; 1 =out-group] 0.31 14801 0.577 Target [0 =negative; 1 =positive] 15.96 144 0.000 Prime*Target 0.09 14802 0.763 Manipulation *Prime 3.19 14800 0.023 Manipulation *Target 4.42 141 0.005 Manipulation *Prime*Target 2.89 14802 0.034 Random effects Wald Z p-value Residual 85.99 0.000 Intercept [subject =Subject] 7.59 0.000 Note. The model also included, but does not display, the term reflecting identification with the national ingroup and its interactions with the other terms. Counteracting subliminal threat cues 9 effects of a small size (d0.15). Therefore, we decided to collect a minimum of 111 participants. As before, maximum participant numbers were determined by the number of participants that could be recruited within a 2-week period. The additional participants provided a buffer in case not all participants met the inclusion criteria (i.e., Spanish nationality). Participants Undergraduates from the University of Granada (N=159) participated in this study. Those who did not have the Spanish nationality (N=7) were excluded from the sample. Five participants who had high error rates (>20%) during the manipulation were also excluded (N=5). This left a total of 147 participants (46 men; 31%) in the final sample, equally divided over the experimental conditions (N disadvantage =50; N implied_legit =48; N direct_legit =49). The average age was 20 years old, ranging from 18 to 42 years old. Manipulation The implied legitimacy condition and the disadvantage condition were the same as in Studies 1 and 2. The ‘direct legitimacy condition’ was new. This condition had the same structure as the implied legitimacy condition and combined two types of associations: ‘Spanish’ was subliminally associated with economic disadvantage and legitimacy targets (i.e., ‘Spanish-deserves’; ‘Spanish-debt’). The out-group prime ‘German’ was subliminally associated with both economic advantage and non-legitimacy (‘German-receives’; ‘German-wealth’). In sum, all three conditions exposed participants to subliminal reminders of in-group disadvantage (Spanish-Poor), but two of the conditions additionally suggested that the in-group was responsible for this disadvantage, either by reference to stereotypes (implied legitimacy condition) or by direct reference to legitimacy (direct legitimacy condition). Preliminary analyses For the RT scores in the evaluative priming measure, we used the same cut-off as in Studies 1 and 2. The resulting distribution was approximately normal (skewness =0.47; kurtosis =0.06), with a median RT of 598 ms. As before, there was a random Subject factor (Wald’s Z=8.21, p<0.001), which is included in the multilevel model described below. We established that group identification (a=0.94) could be used as a covariate, as it was not affected by the manipulation, F<1, p=.662. Results The model applied to the evaluative priming tasks is shown in Table 6. There was a main effect of target valence: Positive targets (M=608 ms) elicited faster responses than negative targets (M=618 ms). Moreover, there were main effects of the prime category and the manipulation, although the interaction between these terms did not reach significance. Based on the presence of additive main effects, we examined the hypothesized simple effects, which are shown in Table 7. The simple effects were replicated in line with hypothesis 1: After exposure to the implied legitimacy condition, the categorization of positive targets was facilitated by in-group primes (M=607 ms) versus out-group primes (M=615 ms). Additionally, in line with hypothesis 2, the direct 16 Jolien A. van Breen et al. legitimacy condition showed a similar effect; the categorization of positive targets was facilitated by in-group primes (M=591 ms) versus out-group primes (M=600 ms). These effects arise from additive main effects, rather than a 3-way interaction (Figure 3). Discussion The findings of Study 3 replicated findings of Studies 1 and 2: The implied legitimacy condition triggered responses that serve to counteract group devaluation. Specifically, after exposure to a negative representation of the group, participants more readily associated positive target words with the in-group than the out-group. Further, Study 3 contributes to our understanding of how the implied legitimacy condition was interpreted. The condition where subliminal associations legitimized disadvantage through stereotypes produced similar effects as the condition in which legitimizing information was directly manipulated. Additionally, as in Studies 1 and 2, the effect was absent when no legitimizing information is given (i.e., the disadvantage condition). Table 6. Full results for the evaluative priming measure in Study 3 Fixed effect F-value df (denominator) p-value Intercept 25333.26 146 0.000 Manipulation [0 =disadv; 1 =direct legit; 2 =implied legit] 3.69 144 0.027 Prime [0 =in-group; 1 =out-group] 11.23 15884 0.001 Target [0 =negative; 1 =positive] 44.21 15876 0.000 Prime *Target 1.21 15876 0.271 Manipulation *Prime 0.11 15884 0.895 Manipulation *Target 1.69 15876 0.185 Manipulation *Prime *Target 0.52 15876 0.594 Random Effects Wald Z p-value Residual 89.09 0.000 Intercept [subject =Subject] 8.06 0.000 Note. The model also included, but does not display, the term reflecting identification with the national ingroup and its interactions with the other terms. Table 7. Simple effects in the evaluative priming measure in Study 3 Condition Target valence Group prime M diff SE p-value 95% CI Spanish German Lower Upper Disadvantage Negative 627.72 632.37 4.65 3.67 .206 11.87 2.56 Positive 615.44 618.82 3.38 3.66 .356 10.57 3.80 Implied legitimacy Negative 620.28 623.10 2.82 3.73 .450 10.14 4.50 Positive 606.70 615.04 8.35 3.71 .024*15.61 1.08 Direct legitimacy (new) Negative 600.18 603.13 2.95 3.66 .421 10.13 4.23 Positive 591.45 599.56 8.11 3.61 .025*15.19 1.03 Across conditions Across valences 610.30 615.34 5.05 1.45 .001*** 7.99 2.10 Note. Significant effects are highlighted in bold. Simple effects with an asterisk are described in the text. Counteracting subliminal threat cues 17 Meta-analysis Across studies, there was evidence that participants counteract subliminal cues of group devaluation, when those cues touch on particularly threatening issues. To provide insight into the robustness of the central effect, we meta-analysed the central effect across the 3 studies, following the procedures outlined in Goh, Hall, and Rosenthal (2016) and Morris and DeShon (2002). The effect size from each individual study (i.e., the mean difference) was weighted by sample size and combined to yield the meta-analytic effect size. Results showed that the implied legitimacy condition produced a robust effect on the evaluative priming measure, whereby in-group primes (vs out-group primes) facilitated responses to positive targets. The effect was small-to-medium in size and significant, d=0.34, Z=3.14, p=.002. GENERAL DISCUSSION Within the literature on group-based stigma and devaluation, it has become increasingly clear that group-based devaluation can take very subtle forms (Devine, 1989; Holoien & Shelton, 2012). The current work examines the conditions necessary to allow people to counteract subliminal cues of devaluation. We show that this need not trigger assimilation; instead, members of devalued groups have coping strategies available to address such experiences. We exposed Spanish participants to different types of subliminal associations that presented the in-group negatively. Results showed that, instead of assimilating, participants counteracted group devaluation. With regard to the circumstances that allow these responses to arise, people counteracted subliminal cues of devaluation specifically when those cues draw on national stereotypes to legitimize the effects of the economic crisis. We argue that this represents a particularly salient threat to our Spanish participants (Bukowski et al., 2017; Fritsche et al., 2017). This is in line with previous work we have conducted in the context of gender, which showed that the tendency to counteract subliminal stereotypes of women is particularly pronounced amongst feminists (van Breen et al., 2018), who are particularly aware of the problematic implications of gender stereotypes (van Breen, Spears, Kuppens, & de Lemus, 2017). As such, it seems that exposure to subliminal cues Figure 3. Reaction times in the evaluative priming measure in Study 3. Error bars represent 1 standard error. 18 Jolien A. van Breen et al. can produce relatively sophisticated responses, by triggering strong motivating factors like threat. Coping responses such as these, that aim to counteract devaluation, have sometimes been termed ‘resistance’ (de Lemus et al., 2013; van Breen et al., 2018) a term which we have also used here. Even though the term resistance may bring to mind quite explicit and direct responses, such as collective action (e.g., van Zomeren & Iyer, 2009), we believe that subtler responses –such as those demonstrated here –should also be considered part of the resistance ‘repertoire’ available to disadvantaged groups. Concretely, in these studies, tendencies to counteract devaluation are expressed through the evaluative associations people made –instead of going along with the negative representation of the in-group in the manipulation, participants make positive in-group associations. It is worth considering whether such subtle responses can have beneficial effects, either for the individual who resists or for others. Here, implicit forms of resistance are expressed as positive attitudes towards one’s in-group. This is in line with a growing body of research that has examined how resistance to devaluation may be expressed in intra-group processes (Leach & Livingstone, 2015; Scheepers, Spears, Doosje, & Manstead, 2003). For instance, devaluation might be resisted by re-emphasising one’s commitment to the group’s cultural practices (Droogendyk & Wright, 2017) or greater identification with the group (Branscombe, Schmitt, & Harvey, 1999). In other words, those who show implicit resistance may engage with their in-group in a more positive way. Further, subliminal responses can serve to buffer self-esteem (Rudman, Dohn, & Fairchild, 2007), and as such can benefit well-being of the individual. Throughout this work, we argue that the reason why people counteract the implied legitimacy condition in particular is that it produces a sense of threat. However, in these studies we could not measure the experience of threat directly. As such, we must consider whether there are processes other than threat that could explain our findings in the implied legitimacy condition. One possibility would be that the implied legitimacy condition is easier to counteract than the other conditions, for instance because it is more obviously untrue. It certainly seems that national stereotypes can provide only very limited insight into macro-level economic phenomena like financial crises. Thus, participants might find the stereotype element of the implied legitimacy condition particularly easy to oppose. However, in Study 3, similar effects were obtained in the implied legitimacy and the direct legitimacy condition, which did not rely on stereotypes to make its legitimizing arguments. As such, we do not believe this line of reasoning can provide a straightforward explanation of our findings. Conclusion Given increasing evidence that group-based devaluation can take very subtle forms, we examine how members of a disadvantaged group cope with these experiences. In the context of the financial crisis in Spain, we show that members of disadvantaged groups are able to counteract subliminal cues of group devaluation. Importantly, the current findings go beyond demonstrating the occurrence of resistance, by examining the underlying concerns that trigger it: Spanish participants counteract subliminal associations that legitimize in-group disadvantage. In sum, these studies show that members of disadvantaged groups can draw on subtle but sophisticated strategies to address subliminal cues of group-based devaluation. Counteracting subliminal threat cues 19 Acknowledgements The authors would like to thank Maϊka Telga and  Alvaro Rodr ıguez-L opez for help with data collection. The work reported here was supported by Grant no. PSI2016-79971-P from the Spanish Ministry of Science and Technology (AEI/FEDER, UE; awarded to Soledad de Lemus), as well as by the Endowed Chair of Russell Spears at the University of Groningen. Conflict of interest All authors declare no conflict of interest. Author contributions Jolien van Breen (Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Project administration; Resources; Validation; Visualization; Writing – original draft; Writing –review & editing) Soledad de Lemus (Conceptualization; Funding acquisition; Methodology; Resources; Supervision; Validation; Writing –review & editing) Russell Spears (Conceptualization; Funding acquisition; Supervision; Writing –review & editing) Toon Kuppens (Conceptualization; Methodology; Resources; Supervision; Validation; Writing –review & editing). Data availability statement Data and Syntax associated with this project are available on the Open Science Framework, at https://osf.io/9dpk8 References Algom, D., Chajut, E., & Lev, S. (2004). A rational look at the emotional Stroop phenomenon: a generic slowdown, not a Stroop effect. Journal of Experimental Psychology: General,133, 323–338. https://doi.org/10.1037/0096-3445.133.3.323 Amir, N., Foa, E. B., & Coles, M. E. (2000). Implicit memory bias for threat-relevant information in individuals with generalized social phobia. Journal of Abnormal Psychology,109, 713. https:// doi.org/10.1037/0021-843X.107.2.285 Becker, J. C., Wagner, U., & Christ, O. (2011). Consequences of the 2008 financial crisis for intergroup relations: The role of perceived threat and causal attributions. Group Processes & Intergroup Relations,14, 871–885. https://doi.org/10.1177/1368430211407643 Betz, H. G., & Johnson, C. (2004). Against the current—stemming the tide: the nostalgic ideology of the contemporary radical populist right. Journal of Political Ideologies,9, 311–327. https://doi. org/10.1080/1356931042000263546 Blair, I. V. (2002). The malleability of automatic stereotypes and prejudice. Personality and Social Psychology Review,6, 242–261. https://doi.org/10.1207/S15327957PSPR0603_8 Bloom, D. (2015). Angry Greeks compare Germans to Nazis after they are forced into austerityladen £60bn bailout. the Mirror. Retrieved from http://www.mirror.co.uk/news/uk-news/ angry-greeks-compare-germans-nazis-6059006 Branscombe, N. R., Schmitt, M. T., & Harvey, R. D. (1999). Perceiving pervasive discrimination among African Americans: Implications for group identification and well-being. Journal of Personality and Social Psychology,77(1), 135. https://doi.org/10.1037/0022-3514.77.1.135 20 Jolien A. van Breen et al. Brooks, D. (2011). The Spirit of Enterprise. the New York Times. Retrieved from http://www. nytimes.com/2011/12/02/opinion/brooks-the-spirit-of-enterprise.html Bukowski, M., de Lemus, S., Rodriguez-Bail on, R., & Willis, G. B. (2017). Who’s to blame? Causal attributions of the economic crisis and personal control. Group Processes & Intergroup Relations,20, 909–923. https://doi.org/10.1177/1368430216638529 Ca~ nadas, E., Rodr ıguez-Bail on, R., Milliken, B., & Lupi a~ nez, J. (2013). Social categories as a context for the allocation of attentional control. Journal of Experimental Psychology: General,142, 934. https://doi.org/10.1037/a0029794 Capucha, L., Est^ ev~ ao, P., Calado, A., & Capucha, A. R. (2014). The role of stereotyping in public policy legitimation: the case of the pigs label. Comparative Sociology,13, 482–502. https://doi. org/10.1163/15691330-12341316 Crandall, C. S., & Eshleman, A. (2003). A justification-suppression model of the expression and experience of prejudice. Psychological Bulletin,129, 414. https://doi.org/10.1037/0033-2909. 129.3.414 Crisp, R. J., Bache, L. M., & Maitner, A. T. (2009). Dynamics of social comparison in counterstereotypic domains: Stereotype boost, not stereotype threat, for women engineering majors. Social Influence,4, 171–184. https://doi.org/10.1080/15534510802607953 Custers, R., & Aarts, H. (2010). The unconscious will: How the pursuit of goals operates outside of conscious awareness. Science,329(5987), 47–50. https://doi.org/10.1126/science.1188595 de Lemus, S., Spears, R., Bukowski, M., Moya, M., & Lupi a~ nez, J. (2013). Reversing implicit gender stereotype activation as a function of exposure to traditional gender roles. Social Psychology,44 (2), 109. https://doi.org/10.1027/1864-9335/a000140 de Lemus, S., Spears, R., Lupi a~ nez, J., Bukowski, M., & Moya, M. (2018). Automatic ingroup bias as resistance to traditional gender roles?. Social Psychological Bulletin,13(4), 1–24. https://doi. org/10.32872/spb.v13i4.29080 Devine, P. G. (1989). Stereotypes and prejudice: their automatic and controlled components. Journal of Personality and Social Psychology,56(1), 5–18. https://doi.org/10.1037/00223514.56.1.5 Doyen, S., Klein, O., Pichon, C. L., & Cleeremans, A. (2012). Behavioral priming: it’s all in the mind, but whose mind? PLoS One,7(1), e29081. https://doi.org/10.1371/journal.pone.0029081 Droogendyk, L., & Wright, S. C. (2017). A social psychological examination of the empowering role of language in Indigenous resistance. Group Processes & Intergroup Relations,20, 303–316. https://doi.org/10.1177/1368430216683532 Ellemers, N., Spears, R., & Doosje, B. (1997). Sticking together or falling apart: In-group identification as a psychological determinant of group commitment versus individual mobility. Journal of Personality and Social Psychology,72, 617. https://doi.org/10.1037/0022-3514.72.3.617 Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods,39, 175–191. https://doi.org/10.3758/BF03193146 Fazio, R. H., Jackson, J. R., Dunton, B. C., & Williams, C. J. (1995). Variability in automatic activation as an unobtrusive measure of racial attitudes: a bona fide pipeline? Journal of Personality and Social Psychology,69, 1013–1027. https://doi.org/10.1037/0022-3514.69.6.1013 Friedman, T. L. (2011). Can Greeks become Germans? the New York Times. Retrieved from http:// www.nytimes.com/2011/07/20/opinion/20friedman.html?hp%20http:/www.nytimes.com/ 2011/12/02/opinion/brooks-the-spirit-of-enterprise.html?_r=1 Fritsche, I., Moya, M., Bukowski, M., Jugert, P., de Lemus, S., Decker, O., ... Navarro-Carrillo, G. (2017). The Great Recession and Group-Based Control: Converting Personal Helplessness into Social Class Ingroup Trust and Collective Action. Journal of Social Issues,73(1), 117–137. https://doi.org/10.1111/josi.12207 Goh, J. X., Hall, J. A., & Rosenthal, R. (2016). Mini meta-analysis of your own studies: Some arguments on why and a primer on how. Social and Personality Psychology Compass,10(10), 535–549. https://doi.org/10.1111/spc3.12267 Counteracting subliminal threat cues 21 G omez, A., Seyle, D. C., Huici, C., & Swann, Jr, W. B. (2009). Can self-verification strivings fully transcend the self–other barrier? Seeking verification of ingroup identities. Journal of Personality and Social Psychology,97, 1021–1039. https://doi.org/10.1037/a0016358 Holoien, D. S., & Shelton, J. N. (2012). You deplete me: The cognitive costs of colorblindness on ethnic minorities. Journal of Experimental Social Psychology,48, 562–565. https://doi.org/10. 1016/j.jesp.2011.09.010 Kaiser, C. R., Vick, S. B., & Major, B. (2006). Prejudice Expectations Moderate Preconscious Attention to Cues That Are Threatening to Social Identity. Psychological Science,17, 332–338. https://doi.org/10.1111/j.1467-9280.2006.01707.x Kawakami, K., Dovidio, J. F., Moll, J., Hermsen, S., & Russin, A. (2000). Just say no (to stereotyping): effects of training in the negation of stereotypic associations on stereotype activation. Journal of Personality and Social Psychology,78, 871–888. https://doi.org/10.1037/0022-3514.78.5.871 Kray, L. J., Thompson, L., & Galinsky, A. (2001). Battle of the sexes: Gender stereotype confirmation and reactance in negotiations. Journal of Personality and Social Psychology,80, 942–958. https://doi.org/10.1037/0022-3514.80.6.942 Kressel, L. M., & Uleman, J. S. (2015). The causality implicit in traits. Journal of Experimental Social Psychology,57,51–54. https://doi.org/10.1016/j.jesp.2009.08.018 Leach, C. W., & Livingstone, A. G. (2015). Contesting the meaning of intergroup disadvantage: Towards a psychology of resistance. Journal of Social Issues,71, 614–632. https://doi.org/10. 1111/josi.12131 Leach, C. W., van Zomeren, M., Zebel, S., Vliek, M. L. W., Pennekamp, S. F., Doosje, B., ...Spears, R. (2008). Group-level self-definition and self-investment: a hierarchical (multicomponent) model of in-group identification. Journal of Personality and Social Psychology,95(1), 144–165. https://doi.org/10.1037/0022-3514.95.1.144 Leitner, J. B., & Forbes, C. E. (2015). The role of implicit mechanisms in buffering self-esteem from social threats. In Z. Jin (Ed.), Exploring implicit cognition: learning, memory, and social cognitive processes (pp. 183–204). Hershey, Pennsylvania: IGI Global. https://doi.org/10.4018/ 978-1-4666-6599-6 Linssen, H., & Hagendoorn, L. (1994). Social and geographical factors in the explanation of the content of European nationality stereotypes. British Journal of Social Psychology,33, 165–182. https://doi.org/10.1111/j.2044-8309.1994.tb01016.x Major, B., Quinton, W. J., & Schmader, T. (2003). Attributions to discrimination and self-esteem: Impact of group identification and situational ambiguity. Journal of Experimental Social Psychology,39, 220–231. https://doi.org/10.1016/S0022-1031(02)00547-4 McNally, R. J., Amir, N., & Lipke, H. J. (1996). Subliminal processing of threat cues in posttraumatic stress disorder? Journal of Anxiety Disorders,10, 115–128. https://doi.org/10.1016/0887-6185 (95)00040-2 McNamara, T. P., & Healy, A. F. (1988). Semantic, phonological, and mediated priming in reading and lexical decisions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 398–409. https://doi.org/10.1037/0278-7393.14.3.398 Morales, E., Garc ıa, L., Rodr ıguez-Bail on, R., & Moya, M. (2004). Predicci on de estereotipos acerca de diferentes regiones de Espa~ na. Encuentros En Psicolog ıa Social,2(1), 293–297. Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods,7(1), 105. https://doi.org/ 10.1037/1082-989X.7.1.105 Moskowitz, G. B., Li, P., Ignarri, C., & Stone, J. (2011). Compensatory cognition associated with egalitarian goals. Journal of Experimental Social Psychology,47, 365–370. https://doi.org/10. 1016/j.jesp.2010.08.010 Nussbaum, A. D., & Steele, C. M. (2007). Situational disengagement and persistence in the face of adversity. Journal of Experimental Social Psychology,43(1), 127–134. https://doi.org/10. 1016/j.jesp.2005.12.007 22 Jolien A. van Breen et al. Otten, S., & Moskowitz, G. B. (2000). Evidence for implicit evaluative in-group bias: Affect-biased spontaneous trait inference in a minimal group paradigm. Journal of Experimental Social Psychology,36(1), 77–89. https://doi.org/10.1006/jesp.1999.1399 Payne, B. K., Burkley, M. A., & Stokes, M. B. (2008). Why do implicit and explicit attitude tests diverge? The role of structural fit. Journal of Personality and Social Psychology,94(1), 16–31. https://doi.org/10.1037/0022-3514.94.1.16 Pearson, A. R., Dovidio, J. F., & Gaertner, S. L. (2009). The nature of contemporary prejudice: Insights from aversive racism. Social and Personality Psychology Compass,3, 314–338. https:// doi.org/10.1111/j.1751-9004.2009.00183.x Pennebaker, J. W., Rim e, B., & Blankenship, V. E. (1996). Stereotypes of emotional expressiveness of northerners and southerners: a cross-cultural test of Montesquieu’s hypotheses. Journal of Personality and Social Psychology,70, 372–380. https://doi.org/10.1037/0022-3514.70.2.372 Ramos, M. R., Barreto, M., Ellemers, N., Moya, M., Ferreira, L., & Calanchini, J. (2016). Exposure to sexism can decrease implicit gender stereotype bias. European Journal of Social Psychology, 46, 455–466. https://doi.org/10.1002/ejsp.2165 Reyna, C., Henry, P. J., Korfmacher, W., & Tucker, A. (2006). Examining the principles in principled conservatism: the role of responsibility stereotypes as cues for deservingness in racial policy decisions. Journal of Personality and Social Psychology,90(1), 109–128. https://doi.org/10. 1037/0022-3514.90.1.109 Rodr ıguez-Bail on, R., Ruiz, J., & Moya, M. (2009). The impact of music on automatically activated attitudes: Flamenco and gypsy people. Group Processes & Intergroup Relations,12, 381–396. https://doi.org/10.1177/1368430209102849 Rudman, L. A., Dohn, M. C., & Fairchild, K. (2007). Implicit self-esteem compensation: automatic threat defense. Journal of Personality and Social Psychology,93, 798. https://doi.org/10.1037/ 0022-3514.93.5.798 Scheepers, D., Spears, R., Doosje, B., & Manstead, A. S. R. (2003). Two Functions of verbal intergroup discrimination: Identity and instrumental motives as a result of identification and group threat. Personality and Social Psychology Bulletin,29, 568–577. https://doi.org/10.1177/ 0146167203029005002 Shanks, D. R., Newell, B. R., Lee, E. H., Balakrishnan, D., Ekelund, L., Cenac, Z., ...Moore, C. (2013). Priming intelligent behavior: An elusive phenomenon. PLoS One,8, e56515. https://doi.org/10. 1371/journal.pone.0056515 Sierp, A., & Karner, C. (2017). National stereotypes in the context of the European crisis. National Identities,19(1), 1–9. https://doi.org/10.1080/14608944.2016.1209646 Spears, R., Gordijn, E., Dijksterhuis, A., & Stapel, D. A. (2004). Reaction in action: intergroup contrast in automatic behavior. Personality & Social Psychology Bulletin,30, 605–616. https://doi.org/ 10.1177/0146167203262087 Strahan, E. J., Spencer, S. J., & Zanna, M. P. (2002). Subliminal priming and persuasion: Striking while the iron is hot. Journal of Experimental Social Psychology,38, 556–568. https://doi.org/10. 1016/S0022-1031(02)00502-4 Swim, J. K., Aikin, K. J., Hall, W. S., & Hunter, B. A. (1995). Sexism and racism: Old-fashioned and modern prejudices. Journal of Personality and Social Psychology,68, 199–214. https://doi. org/10.1037/0022-3514.68.2.199 van Breen, J. A., Spears, R., Kuppens, T., & de Lemus, S. (2017). A multiple identity approach to gender: Identification with women, identification with feminists, and their interaction. Frontiers in Psychology,8, 1019. https://doi.org/10.3389/fpsyg.2017.01019 van Breen, J. A., Spears, R., Kuppens, T., & de Lemus, S. (2018). Subliminal gender stereotypes: Who can resist?. Personality and Social Psychology Bulletin,44, 1648–1663. https://doi.org/10. 1177/0146167218771895 van Hecke, M. (2017). Imag (in) ing the eurocrisis: a comparative analysis of political cartoons. National Identities,19(1), 129–147. https://doi.org/10.1080/14608944.2016.1199541 Counteracting subliminal threat cues 23 Van Zomeren, M., & Iyer, A. (2009). Introduction to the social and psychological dynamics of collective action. Journal of Social Issues,65, 645–660. https://doi.org/10.1111/j.1540-4560. 2009.01618.x Weiner, B. (1995). Judgments of responsibility: A foundation for a theory of social conduct. Guilford Press. Willis, G. B., & Rodr ıguez-Bail on, R. (2008). Prediciendo el estereotipo del endogrupo: factores estructurales y comparaci on intergrupal. Revista De Psicolog ıa Social,23, 193–201. https://doi. org/10.1174/021347408784135823 Received 29 June 2020; revised version received 17 May 2021 Supporting Information The following supporting information may be found in the online edition of the article: Appendix S1. Supplementary Information. 24 Jolien A. van Breen et al.