Meter for the Quantitative Analysis of Newspaper Sport Material

This is an electronic reprint of the original article. This reprint may differ from the original in pagination and typographic detail. Author(s): Title: Year: Version: Please cite the original version: All material supplied via JYX is protected by copyright and other intellectual property rights, and duplication or sale of all or part of any of the repository collections is not permitted, except that material may be duplicated by you for your research use or educational purposes in electronic or print form. You must obtain permission for any other use. Electronic or print copies may not be offered, whether for sale or otherwise to anyone who is not an authorised user. Meter for the Quantitative Analysis of Newspaper Sport Material Laine, Antti Laine, A. (2016). Meter for the Quantitative Analysis of Newspaper Sport Material. Studies in Media and Communication, 4(2), 44-53. https://doi.org/10.11114/smc.v4i2.1703 2016 Studies in Media and Communication Vol. 4, No. 2; December 2016 ISSN 2325-8071 E-ISSN 2325-808X Published by Redfame Publishing URL: http://smc.redfame.com 44  Meter for the Quantitative Analysis of Newspaper Sport Material Antti Laine Correspondence: Antti Laine, University of Jyväskylä, Finland. Received: June 30, 2016 Accepted: July 13, 2016 Online Published: August 11, 2016 doi:10.11114/smc.v4i2.1703 URL: http://dx.doi.org/10.11114/smc.v4i2.1703 Abstract This article presents a meter for the quantitative analysis of newspaper sport material. The meter makes it possible to measure and classify newspaper sport material in detail. The meter has three levels. The selected level depends on the research purpose and desired measurement accuracy. Measurement can focus on a certain level, or all levels can be used together. Individual variables can also be utilized at a certain level. The three levels with respective level units of observation are: 1) articles, photos, and graphics; 2) sets of articles; and 3) sets of data materials. The use of each level is presented in the article. The article also contains a summary of the newspaper sport material analysis form and an online link to the whole newspaper sport material analysis form, which is published as a supplementary file in SMC webpages. Reliability of the meter has been tested using test subjects. The test indicates that the meter is well-suited for measuring, classifying, and analyzing newspaper sport material; however, due to particularity and comprehensiveness, use of the meter requires meticulous familiarization and focus on the measurement and coding work. Keywords: newspaper, sport material, quantitative content analysis, meter, measurement 1. Introduction Newspaper sport pages have been studied widely quantitatively. In particular, the distribution of different sports, gender, and coverage of race have been typical research subjects. Additionally, for example, article types, article positioning, and photographic coverage have been researched. Considering the amount of studies conducted on the subject, it is somewhat surprising that a diverse and detailed meter for the quantitative analysis of newspaper sport material has not been developed so far. This study presents such a meter. The entire meter, or its individual variables, can be utilized depending on the research subject and intent. Originally, meter has been developed in order to be able to measure and classify in detail nearly 10,000 pages of newspaper data quantitatively (Laine 2011). 2. The Meter's Theoretical and Methodological Premises The measurement convention is situated in the field of quantitative content analysis. According to the classic definition in Berelson's (1952, 489) Content Analysis in Communication Research, content analysis is “a research technique for the objective, systematic, and quantitative description of manifest content of communication”. The manifest content of communications refers to the content of the research material as it is: what the text presents, how it describes the world, or what approaches it conveys (Väliverronen, 1998, 15). Perceptions of content analysis have changed in many ways since Berelson's work was published. Krippendorff (2004, 18) defines content analysis in broad terms as a research technique for making replicable and valid inferences from texts (or other meaningful matter) to the contexts of their use. This definition states that content analysis has become an umbrella concept for a wide range of analysis methods. Quantitative research designs have been joined by qualitative research designs (see e.g., Titscher, Meyer, Wodak, & Vetter 2000, 55), and the distinguishing terminological details of a research framework have been linked to the concept of content analysis. A good example is the second edition of Krippendorf's (2004) widely cited Content Analysis, in which quantitative and qualitative content analyses are more clearly distinguished from each other than in the first edition published about 25 years earlier (Krippendorf 1980). Creating a terminological division between quantitative and qualitative research methods is justifiable, though in practice, analysis methods often become intertwined. As Seppänen (2005) aptly remarks, it’s difficult to find an example of a study that is purely the quantitative reporting of content. Certain qualitative solutions are required for quantitative measurement of the material. If quantification is based, for example, on measuring the incidence of certain words, the subject words in the research must first be defined (Seppänen, 2005, 145–146). The intended use of the method is essential for defining content analysis. In a manner similar to this research, with a Studies in Media and Communication Vol. 4, No. 2; 2016 45 basis in quantitative media research, Riffe, Lacy, and Fico (2005) define content analysis as follows: Quantitative content analysis is the systematic and replicable examination of symbols of communication, which have been assigned numeric values according to valid measurement rules. The analysis is based on relationships involving those values, using statistical methods. The analysis is followed by a description of the results and presentation of conclusions. (Riffe, Lacy, & Fico, 2005, 24–36.) Five key concepts are distinguishable in this definition: 1) systematic; 2) replicable; 3) symbols of communication; 4) units of measure defined according to valid measurement rules and statistical analysis of relationship; and 5) describing and inferring. 3. Planning Work for the Meter The planning of the meter began by exploring numerous method guides that cover text analysis techniques. In addition to the aforementioned works by Berelson and Krippendorf, several publications were referenced, including those of Titscher et al. (2000), Methods of Text and Discourse Analysis, Neuendorf’s (2002) The Content Analysis Guidebook and Riffe et al. (2005), Analyzing Media Messages. Methods guides were utilized for defining central concepts and general measurement principles. In addition, sources that focused on analyzing newspapers were explored, especially from the Readership Institute at Northwestern University (e.g., Lynch, & Peer, 2000). With the methods and the research concepts clarified, numerous previous researches that analyze newspaper sport content quantitatively were explored (e.g., Bruce, Hovden, & Markula, 2010; Pedersen, Whisenant, & Schneider, 2003; Vincent, Imwold, Masemann, & Johnson, 2002). As meter planning progressed, a material-oriented emphasis proved unavoidable. Newspaper content has been researched quantitatively in many ways, but the utilized meters were not as specific and diverse as the set objectives. Wallin's (1998, 276–290) classification framework developed for the quantitative analysis of sport material in Swedish newspapers proved the most beneficial from a theoretical aspect. The other applied research was most beneficial in defining individual variables and classes, and in developing measuring practices. Developing the meter rarely progressed linearly, and was mainly an orbicular progression (see Seppänen, 2005, 148), alternating with literature reading, creating the meter (e.g., defining variables, classes, and measuring practices) and conducting test coding to different research materials. The purpose was to find theoretical tools from the literature and test their applicability to the materials. During the planning process, the “finished” meter had to be researcher tested multiple times until it was really finished, i.e., until it worked as planned in every way. Only after that was the meter tested with test subjects. 4. The Meter's Central Concepts and General Classification Principles The central concepts are unit of observation and coding unit. A unit of observation is a unit described by the data on researched phenomena that are entered into the memory (Pietilä, 1976, 102). The meter's units of observation are classified into five main categories: (1) articles; (2) photos; (3) graphics; (4) sets of articles; and (5) sets of data materials. A coding or classification unit is the part of a unit of observation that assigns it a certain variable value (Seppänen, 2005, 152). The primary general principle in the classification is that units of observation are designated to the content class of the variable to which they primarily belong. Primary refers to the coding unit that is dominant in the unit of observation. It is mandatory to define what is primary, as many units of observation contain coding units that designate them into multiple content classes. If a unit of observation can be designated to more than one content class, and a primary class cannot be defined, it is designated to the Mixed class. If a unit of observation is not designated to other content classes, or its content class cannot be defined, then it is placed in the Other class. The second general classification principle is so-called perceived final result (Pietilä, 2008, 40). In this context, this means that the units of observation are coded into variables based on the content that is visible in the newspapers. The third general principle concerns variables focused on classification of people (three article variables and one photo variable). The general principle for classifying these is that the people are coded to the class of the role by which they are primarily analyzed. 5. Three Levels of Measurement The meter has three levels. The selected level depends on the research purpose and desired measuring accuracy. Measurement can only be focused on a certain level, or all three levels can be used together. Individual variables can also be utilized at a certain level. On level I, the analysis focuses on articles, article photos, and graphics. All non-photo images are defined as graphics, except articles’ identifier graphics (e.g., newspaper logos). On level II, sets of articles are analyzed. On level III, analysis focuses on differentiating the complete set of data material. Next, each measuring level is discussed and concluded by presenting a summary of the quantitative analysis form on newspaper sport material (Figure 2). Studies in Media and Communication Vol. 4, No. 2; 2016 46 5.1 Articles, Photos and Graphics as Units of Observation (Meter, Level I) Level I classification objects are the meter's smallest units of observation. Text regions in the units of observation are not analyzed separately. Photos and their captions may form independent photo articles, but they are usually included in other articles. In defining article variables, photos and graphics and their captions are included in the article's unit of observation. Each photo and all graphics at least three cm² and their captions are defined as individual units of observation, and coded into the meter's photo and graphic variables. This is to clarify variables based on the class differentiation of these units of observation as well as variables based on the measured size in cm². It is not necessary to code graphics smaller than three cm². There are no restrictions related to size or type when coding photos. An article is a melding of content and form that is defined as being distinct. Size or type doesn't matter. Each article is coded into 22 article variables that contain one to 25 content classes, depending on the variable. Here, a content class is understood to be synonymous with the value of the variable (see Seppänen, 2005, 148–152). For variables based on classification, the interrelationship between variable values is analyzed on a nominal scale. A ratio scale is used for measure-based variables (See Seppänen, 2005, 152–153). This is achieved by calculating the size in cm², with page count being utilized as the ratio scale. Instead of calculating the size in cm², it is simpler to measure, in millimeters, the width of content placed on symmetrical columns (see e.g., Pulkkinen, 2008, 93–94), but not always possible: the number and size of columns varies in the newspapers. In comparative research, using page count as a ratio scale requires identical page size. All content smaller than a full page is measured with a ruler down to 0.1 cm. The width of the largest content on a page is examined by calculating the differential between other content and page size. Thus, the total size in cm² is constant. If a unit of observation extends from one page to another, a spread measurement principle is applied. This means that all but the largest unit of observation on a spread are measured, and the sum is subtracted from the spread's constant size to calculate the size of the largest article. Page/spread measurement principles are essential for comparison: as the page size remains constant, the measured portions can be compared with other portions. Non-rectangular content is partitioned for measuring into rectangles that follow the contour of the shape. Their sum is calculated. The measurement principles are demonstrated in Figure 1. It contains one newspaper spread segmented into articles (from the Swedish newspaper Aftonbladet, 17.2.2006) and detailed explanations related to measurement. In Aftonbladet, the page size is 933.72 cm² (width 25.1 cm x height 37.2 cm). The size of the spread is 1,867.44 cm² (933.72 cm² x 2). Studies in Media and Communication Vol. 4, No. 2; 2016 47 Figure 1. Measurement example of newspaper sport material Of the meter's 22 article variables, three are background variables (country, newspaper, and event) and three are identifier variables (date, article heading, and article code). If the research does not include newspaper or event-specific comparisons, background variables are not needed. The remaining 16 variables are used to categorize article content. The content-based article variables categorize the area the article deals with, the sport, article type, article positioning, and article size (cm²). There are three person variables (athlete, other person, and source person) and three gender variables related to the person variables. The journalist of the article is coded into five variables (journalist, journalist count, journalists’ gender, photos of journalists, and size of journalists’ photos). Not including published photos of the journalists and photographers (photos of photographers are not noted at all, as they are rare), all photos are classified into seven photo variables and graphics into six graphics variables. Photo captions are included as a part of photos and text over photos is subtracted from their size. Two of the photo variables are identifier variables (article code and photo code). The total number of photos in an article is entered into one of the Studies in Media and Communication Vol. 4, No. 2; 2016 48 photo variables, and the other four variables categorize photo content (color, size, subject, and subject’s gender). Analysis of photo types is not included in the meter as it is not economical to achieve. As with photos, there are two graphics identifier variables (an article code and graphics code). The total number of article graphics is entered into one of the graphics variables, and the other three variables categorize graphics content (color, size, and type). Analysis of graphics types, unlike photos, is included in the meter. 5.2 Sets of Articles as Units of Observation (Meter, Level II) At level II, sets of articles are analyzed. Newspaper journalism is so segmented that classification of individual articles, photos, graphics, and ads is not enough – especially when doing comparative analysis. For example, the number of articles alone does not reveal the full extent of sets of articles (Laine, 2011, 172–173). A set of articles refers to a grouping of at least two articles. If an article is not related to other articles, it is classified as its own set. Angle and layout are two essential factors in identifying sets of articles. The angle is defined as a content-based approach in analyzing the article’s subject. Layout refers to a newspaper's typographical and artistic composition. Layout styles are also used to rank and create a hierarchy between parts of a set of articles (Kuutti, 2006, 216–217). Sets of articles can be classified in many ways. To minimize the possibility of interpretation, the simplest possible classification rules have been adopted. A general classification rule for sets of articles is that – with the exception of teaser articles and articles that clearly continue on the following page – articles belonging to a set are situated on the same spread and laid out so that they are positioned in relation to each other. There are three detailed classification rules for sets of articles. The classification rule for articles on the same topic is that the angle must be similar. The classification rule for articles on differing topics is that the articles must be similar in form and laid out under a joint vignette or in the same page section. The classification rule for teaser articles is that they're situated in the set of articles to which they point. The articles on the spread in Figure 1 form three sets of articles. Articles 4 and 9 form two single sets, as there are no other related articles. Variable values for sets of articles are the same as the values for the corresponding article variables. The biggest set on the spread is made up of seven articles (1–3 and 5–8), the total size of which is 1,656.44 cm². There are 27 photos with a total size of 581.47 cm² and three graphics with a total size of 14.58 cm² related to the topic in the set of articles. Three journalists produced the set. There's one photo (6.27 cm²) of the journalists. The total size of the text in the set (1,054.12 cm²) was calculated by subtracting the total size of all photos and graphics (581.47 cm² + 6.27 cm² + 14.58 cm²) from the total size of the set of articles (1,656.44 cm²) (Figure 1). Each set of articles is coded into 15 variables. Background variables are identical to background variables at the other two levels. The date and coding for a set of articles are used as identifier variables. As article codes are used in the first classification level as identifier variables for photos and graphics, photos and graphics can also be analyzed on this level. Content variables for sets of articles are article count, set size (cm²), count, and total size of photos and graphics, test area size (text = article – [photos + graphics]), journalist count, and count and total size of photos of journalists (cm²). Articles without information on the journalist or produced by news agencies have a value of zero. With the exception of the journalist count, values for variables in the second classification level can be calculated based on first-level variable values. 5.3 Sets of Data Materials as Units of Observation (Meter, Level III) The research material analysis is condensed into a digest at level III, containing 13 variables. Background variables are identical to background variables at the other two levels. The other ten variables are the publication's date, page count, column width (cm²), non-sport news page count, non-sport news column width (cm²), sport news page count, column width of sport news (cm²), total coverage of the sporting event on sport pages (cm²), ad material on sport pages (cm²), and other sport coverage on sport pages (cm²). To consistently analyze the relationship between sport pages and other pages, front-page sport articles aren't considered sport pages on level III. So the amount of sport material on a publication's front page is subtracted from the total amount of coverage of the researched sport event/other sport coverage, because front-page sport articles are specified as sport material on level I. The relationship between sport pages and other pages can be analyzed by calculating the page count of a newspaper's non-sport news. As event coverage is measured on level I, by calculating the amount of ad material on sport pages it is possible to calculate the total amount of editorial material on the sport pages and its division into coverage of the researched sporting event and other sport coverage. Specifying the amount of ads (not considered editorial material) is important especially in comparative analysis, as the total number of sport pages alone does not reveal how much editorial sport material a newspaper contains. Figure 2 displays the summary of the newspaper sport material analysis Studies in Media and Communication Vol. 4, No. 2; 2016 49 form and contains the variables of each measurement level. The whole newspaper sport material analysis form (i.e. the classes of each variable and the definitions of each class) is published as a supplementary file to this article online in SMC: http://redfame.com/journal/index.php/smc/article/view/1703/1856 LEVEL I LEVEL II Articles (A1 – V1) Sets of articles (A4 – O4) A1. Country A4. Country B1. Newspape r B4. Newspape r C1. Event C4. Event D1. Publication date D4. Publication date E1. Article code E4. Set of articles code F1. Main headline F4. Article codes /article count G1. Topic G4. Size of a set of articles (cm²) H1. Sport H4. Photo count I1. Athlete I4. Size of all photos in a set of articles (cm²) J1. Athlete's gende r J4. Graphics count K1. Other person K4. Size of all graphics in a set of articles (cm²) L1. Other person’s gende r L4. Journalists count in a set of articles M1. Source person M4. Journalists photo count in a set of articles N1. Source person’s gende r N4. Size of journalist photos in a set of articles (cm²) O1. Location O4. Text size in a set of articles (cm²) P1. Type LEVEL III Q1. Size (cm²) Data set of materials (A5 – M5) R1. Journalist A5. Country S1. Journalist count B5. Newspape r T1. Journalists' gende r C5. Event U1. Photos of journalists D5. Publication date V1. Total photo size of journalists (cm²) E5. Publication page count Photos (A2 – G2) F5. Publication size (cm²) A2. Article code G5. Page count of other material B2. Article photo count H5. Size of other material (cm²) C2. Photo code I5. Sport pages count D2. Photo colo r J5. Size of sport pages (cm²) E2. Photo size (cm²) K5. Amount of event coverage on sport pages (cm²) F2. Photo subject L5. Amount of ads on sport pages (cm²) G2. Gender of person in photo M5. Amount of other sport coverage on sport pages (cm²) Graphics (A3 – F3) A3. Article code B3. Graphic count C3. Graphic code D3. Graphic colo r E3. Graphic size (cm²) F3. Graphic type Figure 2. Newspaper sport material analysis form/summary 6. Coding and Statistical Treatment of the Material The coding can be performed with a spreadsheet program and the statistical treatment with the SPSS (Statistical Package for the Social Sciences), one newspaper and measuring level (I–III) at a time. Before the coding can progress to the next level, the previous level must be completed. Coding on each level requires the use of a new sheet in the spreadsheet program. To demonstrate coding in practice, a coding example of the articles in Figure 1 is displayed in Figure 3. Placing variables into columns is compatible with the form in Figure 2. Figure 3 does not contain coding of photos, graphics, or sets of articles as they have been coded onto a different sheet. Figure 3. Coding example for articles in a spreadsheet program Identifier variables are central to code numbering, and are affected by the number of levels used in the meter and the scope of the material to be coded. If the meter is used in its entirety, every article, photo, graphic, and set of articles must be given a code, and any overlap in their use noted (e.g., individual articles partially overlap with sets of articles). Studies in Media and Communication Vol. 4, No. 2; 2016 50 7. The Meter's Reliability The meter's reliability refers to the ability to measure exactly what needs to be measured (validity) and the consistency, unambiguity, and repeatability related to coding the material (reliability). The meter's overall reliability is based on a balance between validity and reliability in relation to the material and formulation of the problem. As the meter's quantitative classification framework is based on qualitative and theoretical choices, the conceptual choices behind the variables can be questioned. For example, is the specified gender of a person the biological, cultural, or social gender (Seppänen, 2005, 155)? In order to achieve the best possible overall reliability, variables, their content classes, and classification rules designed for special cases have been specified in the clearest and most detailed manner. This way the research material can remain ambiguous, fewer content-based generalizations are needed, and the amount of content placed in Other or Mixed classes is minimized. The meter first sorts the material in detail, and the contained variables and classes can later be combined as necessary. This analysis method is demanding and labor-intensive, but also rewarding and reliable from a research perspective. The classification is more consistent and there is less room for interpretation. The meter's reliability has been tested using test subjects (see Lynch, & Peer, 2002, 46). The test was carried out by designing an optional study module for first-year, university-level P.E. social sciences students. The five test subjects accepted to the study module received verbal instructions consisting of three main points: 1) adopting the meter requires careful familiarization; 2) classifying and measuring the material is labor-intensive; and 3) everyone works independently and systematically from beginning to end. After the instructions, test subjects were handed printed measuring instructions and a meter, test material (Finnish newspaper Ilta-Sanomat, 24.2.2006), and measuring equipment (a ruler, pencil, and eraser). Combined with the whole newspaper sport material analysis form (length – 12 pages), information given to the subjects was similar to this article. No more information was given, as the purpose was to test how the subjects learn to use the meter themselves and to see what changes are necessary to the meter. For the coding, a Microsoft Excel file was mailed to the subjects into which background and identifier variables had been entered in order to be able to compare the test subjects' results (see Figures 2–3, variables A–F). There were 1124 coding units in the test material. They were classified as follows: articles = 672 units (42 articles x 16 variables); photos = 228 units (57 photos x 4 variables); graphics = 6 units (2 graphics x 3 variables); sets of articles = 210 units (10 sets of articles x 21 variables); and a digest of data materials = 8 units (1 x 8 variables). The test subjects measured and classified the test material, documented the results in the provided file, and returned the file. Feedback regarding use of the meter was gathered at the same time. Test subjects had internalized the use of variables based on measurement (cm²) splendidly. The coding of photos, graphics, sets of articles, and sets of data materials was also working very well. However, the classification of all article variables didn't work, partly because the detailed classifications didn’t include all the necessary information and partly because the classification of two variables (article topic and article type) was too multifaceted. The main reasons, however, were that the test subjects did not familiarize themselves with the meter well enough, or that they had coded carelessly. As an example of poor familiarization, the coding included many errors that were clearly defined in the detailed classification document (e.g., not all test subjects had coded the betting tips article type in the “service and entertainment material” class, even though this was clearly specified in the article type definitions). Carelessness was also evident in coding errors that were found in variables whose coding did not require learning the meter (e.g., the article's sport or location). After the first test phase the classification of the article topic and article type variables were simplified and new classifications were sent to the subjects. Subjects were asked to completely re-code the two redefined variables, and because of the mentioned careless mistakes also to check their other coding. In the second the phase the measuring generally went extremely well. Two of the five test subjects measured and coded the material meticulously. Three still had some carelessness errors in their coding, but a statistical check proved that all the measurements were reliable. As the results were examined, the values based on statistical comparison for the two careful test subjects and all five test subjects were separated. In analyzing the coding uniformity of two test subjects, Cohen's kappa and P value are used to examine classification-based variables. In analyzing the coding uniformity of all five test subjects, an ICC coefficient (Intraclass Correlation Coefficient), which describes relative reliability (see Atkinson & Nevill, 1998), and P value are used to examine classification-based variables. ICC is suited to instances where the group is small. Analyzing the uniformity of measurement-based variables (cm²) for two test subjects is based on the average, T test, and P value. Analyzing the uniformity of measurement-based variables (cm²) for all five test subjects is based on the ICC coefficient and P value. The calculations are done using the IBM SPSS Statistics 20 program. In interpreting Cohen's kappa, J. Richard Landis and Gary G. Koch's (1977) threshold values have been used as criteria, Studies in Media and Communication Vol. 4, No. 2; 2016 51 and in interpreting the ICC coefficient, Patrick E. Shrout and Joseph L. Fleiss' (1979) threshold values. Landis and Koch have sorted the threshold values of the kappa coefficient that reveals the conformity of measurements into six classes: < 0 = less than chance agreement; 0.01–0.20 = slight agreement; 0.21–0.40 = fair agreement; 0.41–0.60 = moderate agreement; 0.61–0.80 = substantial agreement; 0.81–0.99 = almost perfect agreement. Shrout and Fleiss assert that the threshold for excellent reliability is >0.74, good is 0.70–0.60, acceptable is 0.59–0.40, and poor is <0.40. The analysis of classification-based variables is presented in Table 1 and analysis of measurement-based variables in Table 2. Table 1. Statistical analysis of classification-based variables Variables Two test subjects Five test subjects Articles Kappa P-value ICC P-value Topic 0.834 0.000 0.592 0.000 Sport 0.971 0.000 1 Athlete 1 0.000 0.722 0.000 Athlete’s gende r 1 0.000 0.721 0.000 Other person 0.852 0.000 0.781 0.000 Other person’s gende r 1 0.000 0.734 0.000 Source person 0.933 0.000 0.697 0.000 Source person’s gende r 1 0.000 0.788 0.000 Location 0.947 0.000 0.976 0.000 Article type 0.892 0.000 0.735 0.000 Journalist 1 0.000 0.984 0.000 Journalist count 1 0.000 0.98 0.000 Journalist’s gende r 1 0.000 1 Photos of journalists 1 0.000 1 Photos Kappa P-value ICC P-value Subject 0.956 0.000 0.899 0,000 Gende r 1 0.000 0.992 0,000 Graphics Kappa P-value ICC P-value Type 1 0,000 0.614 0,017 Sets of articles Kappa P-value ICC P-value Number of articles 1 0.000 1 Number of photos 1 0.000 1 Number of graphics 1 0.000 1 Number of journalists 1 0.000 0.847 0.000 Number of journalists photos 1 0.000 1 A kappa coefficient between two measurers is, depending on the variable, 0.834–1.000, and all P values are 0.000 (Table 1). All kappa coefficients between two measurers show that the conformity of measurements is very good. The ICC coefficient among the five measurers was, depending on the variable, 0.592–1.000 and P value, wherever possible to calculate, 0.000 (Table 1). The Article Topic variable's value was Good to Acceptable (0.592), the Graphics variable (0.614) was marginally Good, and the Article Source variable Good to Excellent (0.697). The value of all other variables was Excellent. Table 2. Statistical analysis of measurement-based variables Variables Two test subjects Five test subjects Articles Mean T-test P-value ICC P-value Size -1.79 -0.78 0.442 0.993 0.000 Size of journalist photo -0.33 -1.04 0.303 0.973 0.000 Photos Mean T-test P-value ICC P-value Size 0.09 0.13 0.901 0.999 0.000 Graphics Mean T-test P-value ICC P-value Size -7.98 -1.00 0.5 0.998 0.000 Sets of articles Mean T-test P-value ICC P-value Size -3.58 -1.20 0.245 0.999 0.000 Size of all photos 0.23 0.13 0.899 1 0.000 Size of all graphics -0.76 -1.00 0.329 1 0.000 Size of all journalist photos -0.66 -1.04 0.309 0.985 0.000 Text size -2.39 -0.65 0.526 0.985 0.000 The average difference in the measurement (cm²) between two measurers was minimal, at 0.09–7.98 cm². The difference was the greatest in graphics, which don't have clear boundaries, though only minimal when comparing to the scale of the measurement. The P value range between two measurers was 0.309–0.901, so no measurements were statistically anomalous to each other (>0.05). The ICC coefficients among the five measurers were, depending on the variable, 0.973–1.000, and P values were 0.000. ICC coefficients reveal that the conformity of all measured variables was Excellent (Table 2.)