Data for: How do we measure and increase systems thinking? Comparing self-reported and performative metrics in response to building causal loop models

Page 1 of 27 Fuzzy cognitive mapping as a tool for measuring and increasing systems thinking: A test using Mental Modeler software Survey Start of Block: Consent Consent Welcome! Thank you for your interest in participating in our study! You must be 18 years or older to participate. YOU NEED TO TAKE THIS SURVEY ON A COMPUTER, NOT A PHONE. If you are not currently using a computer, please close out of this survey, and access it from a computer. You will be given brief instructions in how to use an online software tool. We will provide you with a scenario, and ask you to make a diagram of that scenario to the best of your ability. We will also ask you to answer a few brief questions about yourself. The study will take about 30 minutes. You will receive $5.00 for your participation. There are no known risks associated with your participation in this research study. The answers and information you provide in this study will remain confidential. Should the data be published, no individual information will be disclosed. Your participation in this study is completely voluntary. You are free to decline to answer any particular question you do not wish to answer for any reason, and you are free to discontinue participation completely at any point for any reason and still receive payment. We encourage your to print this consent form for your records. If you have any additional questions about the study, please contact Dr. Cindy Frantz, at [email protected] or 440-775-8499. Should you have any questions about your rights as a research participant, please contact the Oberlin College Institutional Review Board, (440) 775 8410 or oc[email protected]. Thank you! If you are willing to participate, click below! o Yes, I am 18 years or older and willing to participate. (1) o No thanks. (2) Skip To: End of Survey If Welcome! Thank you for your interest in participating in our study! You must be 18 years or ol... = No thanks. End of Block: Consent Start of Block: Control for Ecological Knowledge Page 2 of 27 ControlEcoknowcourse I have taken a biology or environmental science course in college or high school that provided me with an understanding of basic ecological concepts. o Yes (5) o No (6) ControlEcoknowTerms People differ considerably in their familiarity with ecological terms. How many of the following ecological terms do you think you could accurately define: ▢ Predator (1) ▢ Prey (2) ▢ Autotroph (3) ▢ Heterotroph (4) ▢ Food web (5) ▢ Food chain (6) ▢ Eutrophication (7) ▢ Trophic (8) ▢ Biomagnification (9) ▢ Carrying capacity (10) End of Block: Control for Ecological Knowledge Start of Block: Systems Thinking Full Scale Page 3 of 27 ST Please indicate your agreement with each of the following statements, using the scale provided. There are no right or wrong answers. Page 4 of 27 Strongly Disagre e (1) Moderatel y Disagree (2) Slightly Disagre e (3) Neutra l (4) Slightl y Agree (5) Moderatel y Agree (6) Strongl y Agree (7) The Earth, including all its inhabitants, is a living system. (1) o o o o o o o All the Earth's systems, from the climate to the economy, are interconnected . (2) o o o o o o o Seemingly small choices we make today can ultimately have major consequences . (3) o o o o o o o Individual people are not as separate from one another as they seem. (4) o o o o o o o Environmental problems, social problems, and economic problems are all separate issues. (5) o o o o o o o Page 5 of 27 When I have to make a decision in my life, I tend to see all kinds of possible consequences to each choice. (6) o o o o o o o I learn best when I can see how the different pieces of a subject relate to one another. (7) o o o o o o o I like to know how events or information fit into the big picture. (8) o o o o o o o Ultimately, we can break all problems down into what is simply right or wrong. (9) o o o o o o o Rules and laws should not change a lot over time. (10) o o o o o o o Everything is constantly changing. (11) o o o o o o o Only very large events can significantly change big systems like economies or ecosystems. (12) o o o o o o o Page 6 of 27 Adding just one more small farm upstream from a lake can permanently alter that lake. (13) o o o o o o o My health has nothing to do with what is happening in the world. (14) o o o o o o o It is possible for a community to organize into a new form that was not planned or designed by an authority or government. (15) o o o o o o o End of Block: Systems Thinking Full Scale Start of Block: Local ST_BEFORE Page 7 of 27 Local_ST_BEFORE Below we ask some questions about how you feel in relation to the world. How you feel can change over time. Please indicate your agreement with each of the following statements, based on how you feel RIGHT NOW. Strongly Disagree (1) Slightly Disagree (2) Neutral (3) Slightly Agree (4) Strongly Agree (5) I think of the place I live as a social system composed of interrelated parts. (1) o o o o o I think of the place I live as an economic system composed of interrelated parts. (2) o o o o o I think of the place I live as an environmental system composed of interrelated parts. (3) o o o o o I think of myself as part of a social community. (4) o o o o o I think of myself as part of a economic community. (5) o o o o o I think of myself as part of a ecological community. (6) o o o o o Page 8 of 27 End of Block: Local ST_BEFORE Start of Block: Scale of Impact ST_BEFORE Scl_Impact_ST_Before Below we ask some questions about how you feel in relation to the world. How you feel can change over time. Please indicate your agreement with each of the following statements, based on how you feel RIGHT NOW. Strongly disagree (14) Moderately disagree (15) Neutral (16) Moderately agree (17) Strongly agree (18) My local actions affect things that happen at a global scale (1) o o o o o Things that happen on a global scale affect my local community (2) o o o o o Things that I do now affect things that happen later (3) o o o o o Things that have happened in the past continue to affect the present (4) o o o o o It is important to consider how our current actions now affect future generations (5) o o o o o End of Block: Scale of Impact ST_BEFORE Page 9 of 27 Start of Block: Cat RippleEcological Trophic RippleCatDesc A neighborhood borders a nature preserve managed by the Nature Conservancy. There are no fences or gates to block access to this preserve or movement of animals into and out of the preserve. The preserve is particularly popular with two groups of local residents -- bicyclists (who appreciate the well-managed bike paths and don’t pay much attention to the wildlife) and birders (who particularly appreciate the hawk population for which the reserve is renowned). The preserve contains a food web composed of sedge plants, insects, mice, lizards and hawks. The feeding relationships (food web) among these organisms are as depicted in the diagram below: the mice rely on the sedge plants as their primary food source; the lizards rely on insects for food; the hawks, in turn, rely on both the mice and lizards as their principal food source and can easily shift their diet from one to another. The mice are important for controlling the abundance of sedge and the lizards are important for controlling the abundance of insects. The feeding relationships and therefore the abundance of each of these different organisms has remained fairly stable for some time. Bob was recently hired by a local technology firm and has purchased a house in the neighborhood. Felix, Bob’s cat, is a very successful and selective hunter -- he catches and kills mice but does not hunt either lizards or birds. This neighborhood (and the nature preserve) has not previously had a free-roaming cat. RippleCatImpact Consider the relationships described in the text above and then use your best judgement to assess how much each of the following is likely to change in abundance after Bob’s family (including Felix the cat!) moves into the neighborhood. Assume that Felix is likely to live for at least another seven years and will continue to enjoy hunting. Use the choices below to Page 16 of 27 RippleGoldfishRespon Multiple factors may be responsible for an impact. To what extent are the following responsible for the impacts of the goldfish on the reservoir? Not at all responsible (1) A little bit responsible (3) Very responsible (4) I don't know (5) Ted's parents (1) o o o o Ted's sister (2) o o o o Ted (3) o o o o Local fish store (4) o o o o The goldfish (5) o o o o Great blue herons (6) o o o o Widemouth bass (7) o o o o Rollerbladers (8) o o o o GoldRipFdbkInterest This scenario is one we've never used before. We would love to get your feedback on it. Page 17 of 27 How interesting was the scenario? o Extremely boring (1) o Slightly boring (2) o Neither boring nor interesting (3) o Slightly interesting (4) o Extremely interesting (5) GoldRipFdbkDifficult How difficult were the questions? o Extremely easy (1) o Slightly easy (2) o Neither easy nor difficult (3) o Slightly difficult (4) o Extremely difficult (5) End of Block: Goldfish RippleEcological Trophic Start of Block: Bookstore Ripple RippleBookstore Small independently-owned bookstores were prominent and successful businesses in most communities throughout much of the 20th century. During the 1980s a number of bookstore chains, such as Barnes and Nobles, Borders and Waldenbooks became prominent. These brick-and-mortar chains were often able to outcompete the independent bookstores. In response, independent bookstores across the country went out of business. Founded in 1994, Amazon emerged as online competition for these chain bookstores. As a result, by the early 2000s, many of the brick-and-mortar chains had themselves gone out of business. With reduced competition from the large brick-and-mortar chains, we are now beginning to see the re-emergence of small independent bookstores which often provide a social gathering space for book lovers. Locally-owned businesses (as opposed to chains) have an added benefit for communities because more of the profits tend to be spent within the community. Consider a downtown that provides a specific example of the situation described Page 18 of 27 above. Karl’s Books, an independent bookstore was beloved within the community, but was ultimately driven out of business by a Borders Bookstore that opened up shop in a nearby mall in the 1980s (long before Amazon was on the scene). Borders then went out of business as a result of competition from online sales from Amazon. Henrietta’s Books and Coffee Shop, an independent bookseller then opened shop in an empty storefront in the downtown. Community members who missed Karl’s bookstore were particularly pleased to have a bookstore back in the downtown. The wild success of Henrietta’s increased foot traffic such that surrounding stores increased sales as well. RippleBookstoreImpac Assess the degree of impact that a successful Amazon had on each of the businesses entities. “Positive impact” means that the success of Amazon increased the success of the business described. Very negative impact (1) Negative impact (2) No impact (3) Positive impact (4) Very positive impact (5) I don't know (6) Karl's bookstore (1) o o o o o o Borders bookstore (2) o o o o o o Henrietta's Books and Coffee Shop (3) o o o o o o Other small businesses in the downtown after Henrietta's opened (4) o o o o o o Page 19 of 27 RipplBookstoreRespon Multiple factors may be responsible for an impact. To what extent are the following responsible for the success of Henrietta’s? Not at all responsible (1) A little bit responsible (2) Very responsible (3) I don't know (4) Karl's bookstore (1) o o o o Borders bookstore (2) o o o o Local community members (3) o o o o Amazon (4) o o o o The emergence of the internet as an important market for goods (5) o o o o Increasing recognition of the importance of local business (6) o o o o BookRipFdbkInterest This scenario is one we've never used before. We would love to get your feedback on it. How interesting was the scenario? o Extremely boring (1) o Slightly boring (2) o Neither boring nor interesting (3) o Slightly interesting (4) o Extremely interesting (5) Page 20 of 27 BookRipFdbkDifficult How difficult were the questions? o Extremely easy (1) o Slightly easy (2) o Neither easy nor difficult (3) o Slightly difficult (4) o Extremely difficult (5) End of Block: Bookstore Ripple Start of Block: Placeholder for the bookstore ripple Placeholderbookstore Please continue to the next page. End of Block: Placeholder for the bookstore ripple Start of Block: Mental Modeler Instructions MMInstructVideo Please watch the following video. A small arrow will appear in the lower right corner to advance to the next part of the study when you have finished watching the video. Please watch the video in full screen mode so you can see everything. To enter full screen mode, click on the four arrows just to the left of the word vimeo in the bottom right corner. You will not be able to review this video after you advance; if you wish to review the contents do so before you advance. Page 21 of 27 MMInstructVideoTimer Timing First Click (1) Last Click (2) Page Submit (3) Click Count (4) End of Block: Mental Modeler Instructions Start of Block: MM scenario description and link to MM. MMScenarioScreenshot Imagine a situation in which a population of rabbits and a population of foxes live in an environment that contains many large boulders, wind turbines and ponds with large areas of grass between these. The grass is the principal source of food for these rabbits -- the more grass that is present, the faster they reproduce (the more their population grows). The grass grows more abundant when it is fertilized by poop from the rabbits and the foxes. Grass also responds with faster growth when it receives more sunlight. The foxes eat rabbits. When the foxes have ample rabbits as food, they mate which increases the size of the fox population. Your job is to construct a simple mental modeler diagram that includes the interactions that occur within the environment described above that affect fox and rabbit populations. Include important components: Your model should minimally include the rabbit population, the fox population , and sunlight, but it can include anything else described above (i.e. other components) that you think might be important to depict this situation and the dynamics that might occur. Do not include components that do not directly or indirectly affect the fox and rabbit population. Do not include components that are not mentioned in the scenario above. Draw connections that indicate impact: Some of the components in your model will directly affect other components. You should draw an arrow extending from any component that has a direct influence on another component to the component(s) it directly influences. Only add an arrow connection when one component directly affects another. For example if A directly affects B and B directly affects C (i.e. A → B → C), but A does not directly affect C, then no arrow should be drawn directly between A and C (A is indirectly affecting C through B). Draw double connections if appropriate: If two components affect each other you should draw two connecting arrows, one from the first to the second and the other from the second going back to the first. It is acceptable for one component to have an arrow going out of it and not have an arrow going into it. Indicate the nature of the relationships involved in these connections: Click on each arrow connecting component. Indicate how the component at the start of an arrow affects the component at the point of the arrow. Choose a (+) to indicate that the component at the end of the arrow changes in the same direction as the component at the start (i.e. if the component at the start goes up, the component at the point will respond by going up; if the component at the start goes down, the component at the point will go down) Choose a (-) to indicate that the component at the end of the arrow changes in the opposite direction as the component at the start (i.e. if the component at the start goes up, the component at the point will respond by going down; if the component at the start goes down, the component at the point will respond by going up) Page 22 of 27 Here is the link to mental modeler: http://www.mentalmodeler.org/scenario/ The username and password are both: mentalmodeler Please upload a screenshot of your model below. MMScenarioFileUpload Please upload the file you saved from your model below. MMScenarioTimer Timing First Click (1) Last Click (2) Page Submit (3) Click Count (4) End of Block: MM scenario description and link to MM. Start of Block: Local ST_AFTER Page 23 of 27 Local_ST_AFTER Below we ask some questions about how you feel in relation to the world. How you feel can change over time. Please indicate your agreement with each of the following statements, based on how you feel RIGHT NOW. Strongly Disagree (1) Slightly Disagree (2) Neutral (3) Slightly Agree (4) Strongly Agree (5) I think of the place I live as a social system composed of interrelated parts. (1) o o o o o I think of the place I live as an economic system composed of interrelated parts. (2) o o o o o I think of the place I live as an environmental system composed of interrelated parts. (3) o o o o o I think of myself as part of a social community. (4) o o o o o I think of myself as part of a economic community. (5) o o o o o I think of myself as part of a ecological community. (6) o o o o o Page 24 of 27 End of Block: Local ST_AFTER Start of Block: Scale of Impact ST_AFTER Scl_Impact_ST_AFTER Below we ask some questions about how you feel in relation to the world. How you feel can change over time. Please indicate your agreement with each of the following statements, based on how you feel RIGHT NOW. Strongly disagree (15) Moderately Disagree (16) Neutral (17) Moderately agree (18) Strongly agree (19) My local actions affect things that happen at a global scale (1) o o o o o Things that happen on a global scale affect my local community (2) o o o o o Things that I do now affect things that happen later (3) o o o o o Things that have happened in the past continue to affect the present (4) o o o o o It is important to consider how our current actions now affect future generations (5) o o o o o End of Block: Scale of Impact ST_AFTER Page 25 of 27 Start of Block: Feedback