Methods as data: Investigating how reporting in methods sections describing mouse models of breast or prostate cancer affects perceived replicability of models

Methods as data: Investigating how reporting in methods sections describing mouse models of breast or prostate cancer affects perceived replicability of models Anna Korzeniowskaa, Louise Saulb, c a. eLife Publishing Ltd, Cambridge CB2 1AW ; b. University of Southampton, Southampton, SO17 1BJ; c. Careers and Skills for Data-driven Research (CaSDaR) The potential for replicability and reproducibility of an experiment is vital for trustworthy research and to verify claims made when publishing the research. However, there are increasing reports of cancer study designs not being reproducible1. Detailed contents of the methods sections of manuscripts are essential for replicability. This is a moral imperative when considering experimental animal research undertaken in accordance with ASPA and for compliance with the 3Rs initiative (Reduce, Refine, Replace), as researchers should show that they are undertaking animal experimentation with an appropriate level of care and consideration towards the experimental animals. However, there is no established method for assessing the replicability of animal models during peer review2, and whilst there exist papers that describe how to embed practices to support making models replicable at the point of experimental design3, we propose that promotion of extensive reporting in methods sections may be a way of promoting perceived replicability of animal models to ensure trust in research , with a focus on mouse models of breast and prostate cancer. To support method sharing, we plan to generate a checklist by searching available literature and consulting with researchers to determine the factors that should be included in methods sections to ensure replicability, then determine the representation of those factors in mouse models of breast and prostate cancer described in manuscripts published to determine the current level of perceived replicability in these manuscripts. 1. Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. eLife, 10, e71601. https://doi.org/10.7554/eLife.71601 2. Landi, M., Everitt, J., & Berridge, B. (2021). Bioethical, Reproducibility, and Translational Challenges of Animal Models. ILAR Journal, 62(1–2), 60–65. https://doi.org/10.1093/ilar/ilaa027 3. Wilson, E., Ramage, F. J., Wever, K. E., Sena, E. S., Macleod, M. R., & Currie, G. L. (2023). Designing, conducting, and reporting reproducible animal experiments. Journal of Endocrinology, 258(1), e220330. https://doi.org/10.1530/JOE-22-0330 4. Percie Du Sert, N., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., Emerson, M., Garner, P., Holgate, S. T., Howells, D. W., Hurst, V., Karp, N. A., Lazic, S. E., Lidster, K., MacCallum, C. J., Macleod, M., … Würbel, H. (2020). Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0. PLOS Biology, 18(7), e3000411. https://doi.org/10.1371/journal.pbio.3000411 5. Sayers E. E-utilities Quick Start. 2008 Dec 12 [Updated 2018 Oct 24]. Abstract Scoping Review Survey Design We used the factors identified to create a survey for researchers currently using experimental animal models for oncology research, where we asked them to rate on a scale of 1-5 (1 meaning not important and 5 meaning essential) the importance of various factors that we have identified from a literature review as being relevant to the replicability of breast/prostate cancer solid tumour models when included in the methods sections of manuscripts. We recruited the participants as researchers who are either involved with the NC3Rs working group with a focus on oncology research or know someone who is attached to this group who has forwarded on the survey to them. The participants will be asked for additional information about their career stage, involvement in replicability studies, their geographic location, and their institution type. The ratings of the factors will support us to create a list of factors important for replicability; based upon the ratings from the survey, the factors will be ranked, and a weighting will be given to each factor. We are currently awaiting the results of the survey. Future Work Having identified the factors relevant to perceived replicability of mouse models, ranked them, and applied a weighting factor to each of them, we plan to query the PubMed database to identify the whether manuscripts describing models of breast or prostate cancer published in the last 5 years could be perceived to be replicable and therefore trustworthy by researchers, based upon the survey results. There are two options to do this work: The selected method will largely depend on the survey results and the rankings allocated. Method Description Barriers E -utilities Entrez programming utilities 5, (Eutilities) which allow search and retrieval of information from the PubMed database with the use of a public API to retrieve numbers of manuscripts from the database which use specific keywords identified from the survey. The data in the PubMed database is unstructured, meaning that it is difficult to determine if the entries returned have the keywords present in the methods or elsewhere in the text. This presents an issue as we are focusing solely on the methods sections. Manual search A systematic review of the literature by creating an exclusion criteria of specific literature, searching the PubMed database, and manually collecting data from the methods sections of the returned manuscripts. This will need to be completed by a named researcher; therefore position statements will need to be completed, and additional testing will need to be implemented to ensure impartiality in scoring. To identify which areas to include in the checklist to promote perceived replicability, the following was used: 1. Resources from institutions known for support of animal welfare in experimentation (NC3Rs4, CAMARADES3) 2. A review of manuscripts published in the last 5 years that published immunocompetent mouse models of breast or prostate models. Manuscripts with the following breast or prostate models were excluded from the scoping review: experimental models published more than 5 years ago, models with transgenic mouse models (athymic mouse models were included), and only the methods sections were examined. Information was not collected from text elsewhere in the manuscript. The above was used to generate a list of 43 factors that we propose are relevant to the replicability of mouse models used in breast and/or prostate cancer models that were divided into 3 different areas: Husbandry, experimental, and procedural. As well as recording the factors that we proposed were important for perceived replicability, we collected information on the geographical location of the lead author to ensure that we were representing a diverse sample. Figure 1. Image showing workflow for the project