Reliability and Validity in Agricultural Research: A shiny App for Non-statisticians

October 2025, Volume 1, Issue II. doi: 10.65287/josta.202510.68E5 Reliability and Validity in Agricultural Research: A shiny App for Non-statisticians Varsha H Kerala Agricultural University Pratheesh P Gopinath* Kerala Agricultural University Abstract Survey research is essential for understanding attitudes, behaviours, and perceptions across diverse populations, providing the foundation for data-driven decisions and empirical analysis. Researchers often face difficulties in assessing the validity and reliability of survey instruments due to complex statistical procedures, limited access to integrated tools, and the need for multiple software platforms. To address these challenges, a userfriendly web application was developed to simplify and automate the testing of validity and reliability. The application integrates multiple statistical analyses into a single platform, offering real-time computation and graphical result displays for easy interpretation. This design minimises repetitive analyses, enhances efficiency, and supports users with limited statistical expertise. A user satisfaction survey involving 30 participants revealed that 90% were satisfied with the application’s performance, demonstrating its effectiveness in improving the accuracy, consistency, and overall quality of survey-based research. Keywords: Questionnaire, Validity, Reliability, Survey research, Agricultural statistics, Social science. 2Reliability and Validity in Agricultural Research: A Shiny App for Non-statisticians 1. Introduction Survey research is a widely used method for collecting quantitative data on human behaviour, attitudes, and opinions. The accuracy of insights derived from surveys largely depends on the quality of the instruments used, particularly the validity and reliability of questionnaires (Carmines and Zeller 1979;Cronbach 1951). Validity ensures that a questionnaire measures the intended construct, while reliability indicates the consistency of measurements over time. However, assessing these psychometric properties can be challenging, as traditional methods often involve manual calculations or specialised statistical software that may not be readily accessible to all researchers (Pallant 2020). To address this limitation, a user-friendly web application was developed to automate validity and reliability testing, providing real-time feedback through an intuitive interface (Kline 2023) This innovation enables researchers to perform comprehensive psychometric evaluations efficiently, regardless of their statistical expertise. In applied research contexts such as agriculture, questionnaires are among the most commonly used tools, yet many studies fail to report procedures for assessing validity (31%) and reliability (33%) (Radhakrishna 2007). By automating these assessments, the developed web application reduces measurement errors-such as sampling, selection, and frame errors-thereby improving the precision and consistency of data collection (Singh 2017). Overall, the web application enhances the methodological rigour of survey research by simplifying complex statistical evaluations, empowering researchers to design reliable and valid instruments, and strengthening the credibility of research outcomes. Its adoption can significantly improve research quality, particularly in developing regions where expertise in psychometric analysis is often limited. 2. Methodology 2.1. Reliability Reliability ensures the consistency and stability of a measurement tool, producing similar results under identical conditions (Nunnally and Bernstein 1994). It underpins the credibility of research by confirming that findings reflect true phenomena rather than measurement errors. Figure 1categorises reliability into four types: Test-retest, Parallel-form, Interrater, and Internal consistency. Each type includes specific methods such as Pearson Correlation, ICC, Cohen’s Kappa, and Cronbach’s Alpha for assessing consistency. They ensure measurement stability, agreement, and internal accuracy across tests and raters. Assessing reliability often requires complex statistical calculations and multiple software tools, making it challenging for researchers without strong statistical expertise. Interpreting results like ICC, Cronbach’s Alpha, or Cohen’s Kappa can also be confusing and time-consuming. To overcome these difficulties, this web application was developed to provide an integrated, user-friendly platform that automates reliability testing and simplifies result interpretation. Journal of Sustainable Technology in Agriculture 3 Figure 1: Types of reliability and statistical tests used to assess reliability 2.2. Validity Validity testing ensures that research instruments accurately measure the intended constructs, enhancing the credibility and quality of findings (Cronbach and Meehl 1955). It minimises measurement errors, strengthens reliability, and supports the generalization of results to broader contexts. Using valid tools ensures ethical responsibility, as accurate data lead to meaningful scientific contributions and sound decision-making in applied fields. Validity testing also refines instruments by confirming that each item measures only its intended construct, thereby advancing reliable and theory-based research. Figure 2classifies validity into Content, Construct, Criterion, and Face Validity. Each type uses specific methods like EFA, CFA, Correlation Analysis, and Expert Review to ensure instruments measure intended constructs accurately. Together, these tests strengthen measurement accuracy, relevance, and scientific credibility. Testing validity involves complex statistical methods like EFA, CFA, and correlation analyses, which require advanced statistical knowledge and multiple software tools. Interpreting these results can be time-consuming and confusing for non-experts. To overcome these challenges, this web application was developed to integrate all validity tests into one user-friendly platform, automating analysis and simplifying result interpretation. 2.3. Framework selection The Shiny was the natural choice for building this application-combining the analytical muscle of R with the sleek interactivity of modern web design (Chang et al. 2023). For researchers diving deep into survey validity and reliability, Shiny offers a powerful toolkit: real-time responsiveness, seamless integration with R’s rich statistical libraries, and effortless web deployment (Beeley 2018). But what truly sets it apart is its flexibility. Whether a seasoned data scientist or a curious newcomer, Shiny makes complex analyses accessible and intuitive (R Core Team 2020). By tailoring the interface to the unique needs of survey-based research, the platform doesn’t just crunch numbers-it tells a story, empowers exploration, and invites insight (Xie et al. 2018). 4Reliability and Validity in Agricultural Research: A Shiny App for Non-statisticians Figure 2: Types of validity and statistical tests to assess validity 2.4. Application Architecture User interface (UI) The user interface of the Shiny app was meticulously designed to strike a balance between simplicity and functionality, ensuring that users can navigate the application with ease while accessing powerful analytical tools. The UI is structured into several key sections, as shown in Figure 3. Navigation Sidebar The sidebar serves as the central hub for navigation, providing users with quick access to all the main functions of the app. This includes uploading data files, selecting statistical tests, and accessing help documentation. The sidebar’s organised layout ensures that users can find what they need without unnecessary clutter, enhancing the overall user experience. Data Upload Mechanism A robust data upload mechanism is a crucial feature of the UI. Users are guided to upload their survey data in CSV format, which the app immediately processes. The application is designed to perform an initial check on the uploaded data, identifying any potential issues such as missing values or formatting errors. This immediate feedback loop allows users to correct problems before they begin their analyses, thereby streamlining the workflow (R Core Team 2020). Test Selection Interface The app offers a dropdown menu that categorizes the available tests under broader sections such as “Reliability” and “Validity.” Within these categories, users can further select specific tests like Cronbach’s Alpha, Intraclass Correlation Coefficient (ICC), or Factor Analysis. The Journal of Sustainable Technology in Agriculture 5 structured organization of the test selection process ensures that users can easily find and apply the appropriate tests to their data, regardless of their statistical expertise. Dynamic Result Display The results generated by the selected tests are presented in a dynamic and interactive format within the main panel of the UI. Depending on the nature of the test, results may be displayed as tables, graphs, or textual summaries. The UI also provides options for users to download or export these results in various formats, facilitating further analysis or inclusion in research reports. This feature ensures that the application not only performs the necessary calculations but also presents the findings in a clear and usable format (Chang et al. 2023). Figure 3: User interface of the developed app Server-side logic The server component of the Shiny app is where the real power of the application lies. It is responsible for processing the uploaded data and executing the complex statistical analyses required for testing validity and reliability. The server’s architecture follows a systematic workflow: Data Preprocessing Once a user uploads their data, the server immediately reads the CSV file into R and conducts a series of preprocessing steps. These include checking for missing data, validating data types, and ensuring that the dataset meets the prerequisites for the selected tests. Such preprocessing safeguards data quality and reduces the likelihood of analytical errors, If any issues are identified, the server communicates these back to the user through the UI, guiding corrections. Execution of Statistical Tests 6Reliability and Validity in Agricultural Research: A Shiny App for Non-statisticians The core functionality of the server revolves around performing statistical tests. For reliability testing, this includes calculations for metrics like Cronbach’s Alpha, ICC, and Cohen’s Kappa (Cronbach 1951;Shrout 1998;Fleiss 1971;Cohen 1992). For validity, the server can compute measures such as the Content Validity Ratio (Lawshe 1975) and conduct Factor Analysis. The server is optimised for efficiency, ensuring results are produced quickly, even for large datasets. Reactive Programming One of Shiny’s key strengths is its reactive programming model, which the server leverages fully. This model ensures that any changes made by the user-such as selecting different tests, uploading new datasets, or modifying parameters-trigger immediate updates in the displayed results (Chang et al. 2023). This interactivity is essential for fostering an intuitive user experience and enabling researchers to explore different analyses in real time. Data flow within the application The data flow within the Shiny app is designed to be seamless and efficient, guiding users through the process from data upload to result interpretation in a logical sequence. Figure 4 explains the following steps of the data flow within the application. File Upload Users begin by uploading their survey data in a CSV format. The app’s UI facilitates this process, ensuring that users can easily locate and upload their files. Data Validation Upon upload, the server automatically validates the data, checking for common issues such as missing values, incorrect data types, or formatting errors. This step is crucial for ensuring that the dataset is ready for analysis. Test Selection Once the data is validated, users can select the desired statistical tests from the dropdown menu. The app’s organization of tests under broader categories like “Reliability” and “Validity” makes this process straightforward. Data Analysis The server processes the validated data and performs the selected statistical tests. Thanks to R’s powerful statistical packages (Beeley 2018), the app can handle a wide range of analyses, from simple metrics like Cronbach’s Alpha to more complex procedures like Factor Analysis. Result Presentation After the analysis is complete, the results are dynamically displayed in the UI. Users can interact with these results, explore different aspects of the data, and even download the findings for further examination or reporting. Journal of Sustainable Technology in Agriculture 7 This well-orchestrated data flow ensures that users can efficiently conduct their analyses, moving smoothly from one step to the next without unnecessary delays or complications. Figure 4: Data flow within the application 3. System Architecture The architecture of the developed web application follows a client–server model designed for efficient data processing and real-time interaction. The User Interface (UI), built using Shiny’s front-end framework, allows users to upload datasets, select analysis options, and view results. The Server component, implemented in R, handles computation by executing automated reliability and validity analyses, including Cronbach’s alpha and other psychometric tests. Data and results flow dynamically between the UI and server, ensuring quick feedback and seamless operation. The Output module displays the processed results as tables, summary statistics, and downloadable files, enhancing user accessibility and experience. Figure 5illustrates the overall system flow from user input to result generation. 4. User Gratification Test To evaluate the developed web application, user gratification was employed as the primary testing construct. User gratification, drawn from psychology and consumer behaviour theories, reflects the satisfaction users experience when interacting with a system (Ruggiero 2000). It provides a holistic framework that incorporates both functional and hedonic dimensions of user experience. For academic research platforms, gratification is shaped by factors such as ease of use (Davis 1989), trust in results (Gefen et al. 2011), aesthetic appeal (Blythe and Hassenzahl 2003), and the extent to which the system meets user requirements. Accordingly, the application was tested across these dimensions to determine its effectiveness in supporting research activities. A total of 30 participants were selected using convenience sampling, comprising postgraduate students and researchers in agricultural sciences. User gratification was measured using a 5-point Likert scale (1 = Strongly Disagree to 5 = Strongly Agree). The constructs included Technology (2 items), Hedonic (2 items), Risk (2 items), and Behavioural Intention (1 item). The functional aspects of gratification were assessed by examining accuracy, efficiency, and 8Reliability and Validity in Agricultural Research: A Shiny App for Non-statisticians Figure 5: System architecture of the developed web application automation in validity and reliability testing. These features reduce manual effort and save time, directly contributing to user satisfaction. The hedonic aspects, including enjoyment, interactivity, and engagement (Venkatesh and Bala 2008), were also measured, recognising their role in transforming data analysis into a more engaging process. Specific indicators such as convenience, authenticity of results, user engagement, privacy concerns, and behavioural intentions were incorporated into the evaluation. This ensured that both the technical performance and psychological impact of the system were captured. By adopting user gratification as a testing framework, the study provides insights into how well the application fulfils user expectations and promotes its continued use in academic research. 5. Results 5.1. Results of the Developed Web Application The developed web application demonstrated effectiveness in automating the assessment of validity and reliability for survey instruments. The platform provided real-time feedback through an intuitive interface, eliminating the need for manual computation or advanced statistical software. Application of the tool reduced measurement errors and ensured more accurate data collection. In agricultural research and related fields, the web application enabled systematic reporting of psychometric properties, addressing the prevalent issue of omitted validity and reliability procedures. Overall, the tool enhanced methodological rigour and improved the credibility of research findings, particularly in contexts with limited statistical Journal of Sustainable Technology in Agriculture 9 expertise and resources. Figure 6shows the Result of the Developed Web Application. Figure 6: Result of the developed web application 5.2. Reliability Test The reliability of the user gratification construct was evaluated using Cronbach’s Alpha. Table 1presents the results for the four key dimensions: technology (convenience), hedonic (enjoyment), risk (privacy concern), and behavioural intention. All four dimensions demonstrated strong internal consistency. Technology (convenience) showed good reliability (� = 0.80), while hedonic (enjoyment) also indicated good reliability (� = 0.89). Risk (privacy concern) achieved the highest reliability (� = 0.97), reflecting excellent consistency. Behavioral intention was similarly reliable (� = 0.91). The overall scale demonstrated high internal consistency, with Cronbach’s � = 0.91, which is considered acceptable for research instruments. Sl. No. Factors Reliability (Cronbach’s Alpha) Status 1 Technology (convenience) 0.80 Good 2 Technology (authenticity of conversation) 0.90 Good 3 Hedonic (enjoyment) 0.89 Good 4 Hedonic (pass time) 0.90 Good 5 Risk (Privacy concern) 0.94 Good 6 Risk (immature technology) 0.92 Good 7 Behaviour intention 0.91 Good 8 Statistical capabilities 0.89 Good Overall Reliability 0.91 Acceptable Table 1: Reliability analysis of user gratification dimensions 16 Reliability and Validity in Agricultural Research: A Shiny App for Non-statisticians ĺPublication & Reviewer Details Publication Information •Submitted: 18 October 2025 •Accepted: 31 October 2025 •Published (Online): 01 November 2025 Reviewer Information •Reviewer 1: Dr. K.Satheesh Kumar Assistant Professor SRM College of Agricultural Sciences SRM Nagar, Kattankulathur, Tamil Nadu •Reviewer 2: Dr. Dinesh Kumar P Assistant Professor Alliance University Anekal, Bengaluru, Karnataka ĹDisclaimer/Publisher’s Note The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of the publisher and/or the editor(s). The publisher and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. © Copyright (2025): Author(s). The licensee is the journal publisher. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits non-commercial use, sharing, and reproduction in any medium, provided the original work is properly cited and no modifications or adaptations are made. Journal of Sustainable Technology in Agriculture 17 Affiliation: Varsha H Agricultural Statistics College of Agriculture, Vellayani Thiruvananthapuram, Kerala India E-mail: [email protected] Pratheesh P Gopinath* Agricultural Statistics College of Agriculture, Vellayani Thiruvananthapuram, Kerala India E-mail: [email protected] URL: https://kau.in/people/sub-lt-dr-pratheesh-p-gopinath Journal of Sustainable Technology in Agriculture https://www.jostapubs.com/ PAPAYA Academic Press, Statoberry LLP, https://www.statoberry.com/papaya October 2025, Volume 1, Issue II Submitted: 2025-10-18 doi:10.65287/josta.202510.68E5 Accepted: 2025-10-31