All Solutions
Primary solutions - practices applied by researchers
Secondary solutions - measures to support primary solutions
Preregistration is a publicly documented research plan (for example, questions, hypotheses, data collection plan, and analysis plan) that is registered before data collection starts, before viewing the data if working with pre-existing data or before research results are known. This can be done by storing the study plan in a (commonly read-only) public repository, such as Open Science Framework Registries or the National Library of Medicine’s Clinical Trials Registry. Researchers can make pre-registration publicly accessible immediately or after an embargo period. Read more in
1) Nosek, B. A., Ebersole, C. R., DeHaven, A. C. & Mellor, D. T. The preregistration revolution. Proc. Natl Acad. Sci. USA 115, 2600–2606 (2018).
2) Rice, D. B. & Moher, D. Curtailing the use of preregistration: a misused term. Perspect. Psychol. Sci. 14, 1105–1108 (2019).
3) Hardwicke, T. E. & Wagenmakers, E.-J. Reducing bias, increasing transparency and calibrating confidence with preregistration. Nat. Hum. Behav. 7, 15–26 (2023).
Evidence:
Medicine
Several large-scale studies in medicine have shown that published studies that were previously registered are of higher methodological quality, have more complete reporting of methods, and have results that less often support the hypothesis, are significant, or large. Pre-registration has also been shown to increase the availability of results (many results are only published in a registry, but not in a journal). The overview of these studies can be found in https://rdcu.be/dJYor (Purgar, M., Glasziou, P., Klanjscek, T. et al. Supporting study registration to reduce research waste. Nat Ecol Evol 8, 1391–1399 (2024)
Psychology
Two meta-studies have shown that published studies that were pre-registered have smaller effect sizes that non-registered studies. 1) Kvarven, A., Strømland, E. & Johannesson, M. Comparing meta-analyses and preregistered multiple-laboratory replication projects. Nat Hum Behav 4, 423–434 (2020) 2) Schäfer, T. & Schwarz, M. A. The meaningfulness of efect sizes in psychological research: diferences between sub-disciplines and the impact of potential biases. Front. Psychol. 10, 813 (2019).
Set-up and fund pre-registration platforms (repositories)
Type: Infrastructure
In charge:
Publisher
Institution
Funder
Pre-registration platforms are repositories where pre-registrations are deposited before the research is conducted. Ideally, they should also allow for: 1) submitting the results of the study after it has been conducted or adding the reasons for a failure of the study; 2) linking to other related resources (e.g. datasets, publications); 3) search for registered study without the registries, but also via other sources (e.g. WoS). Examples include discipline-specific registries such as https://www.isrctn.com/ or https://clinicaltrials.gov/; and general-purpose ones, such as https://www.cos.io/products/osf-registries.
Evidence:
In progressPre-registration should be made possible by developing registration templates. The could be specific to a certain research area, or research design. They can also include options for modular (sequential) registration where not all stages of research have to be registered at the same time.
Evidence:
TBAFunders and research institutions should hire experts that would support researchers in registering their studies.
Evidence:
In progress.
Funders, research institutions and publishers can introduce pre-registration policies that mandate registration of certain research (note, not all research can be registered). For example, the International Committee of Medical Journal Editors introduced trial registration policy that then led to the implementation of laws and policies in the USA and internationally that expanded mandatory prospective trial registration (see here).
A system for checking whether policies are followed should also be established. For example, among 14 medical research funders in Europe that require prospective trial registration, only some monitor whether trials are indeed registered (9 funders) or whether results are made public (8 funders). Text mining and other AI-driven solutions could be used for such monitoring. For instance, PLoS and DataSeer have developed such a tool to monitor Open Science Indicators in PLoS journals.
Evidence:
Medicine
After the International Committee of Medical Journal Editors (ICMJE) announced in 2004 that it would require registration of clinical trials as a condition for publication, relative number of registered clinical trials (given the overall number of clinical trials published) increased five fold until 2013. See the study here.
Badges that are used to acknowledge researchers that preregistered their study and to signal to the reader that the preregistered content is available in a persistent location. Registration badges are usually provided by the journal and are clearly visible on a published manuscript. Read more here.
Evidence:
Not available.
The registered report is a publication format where a study’s design and methods undergo Stage 1 peer review by a journal. This review happens before data collection, or data access and analysis if working on already collected data. Upon passing stage 1 peer review and completion of the research, the final article with results and discussion undergoes stage 2 peer review. The acceptance of a registered report depends on the relevance of the research topic, the thorough development of the research questions/hypotheses, and the robustness of the methodological approach, rather than the results obtained. This format promotes methodological rigor, helps to reduce publication bias and enhances transparency. Read more on registered reports in
1) Chambers, C. D. & Tzavella, L. The past, present and future of registered reports. Nat. Hum. Behav. 6, 29–42 (2022).
2) Henderson, E. L. & Chambers, C. D. Ten simple rules for writing a Registered Report. PLoS Comput. Biol. 18, e1010571 (2022).
Evidence:
Psychology
Several large-scale meta-studies in psychology have shown that, compared to 'classical' published articles, registered reports have higher methodological quality, have more complete reporting of methods, and have results that less often support the hypothesis, are significant, or large. Registered reports also decrease publication bias and increase the availability of results, as a study is published regardless of the features of the results. The overview of these studies can be found in https://rdcu.be/dJYor (Purgar, M., Glasziou, P., Klanjscek, T. et al. Supporting study registration to reduce research waste. Nat Ecol Evol 8, 1391–1399 (2024).)
Medicine (biomedicine)
One meta-study showed that results of registered reports are less often in the expected directions compared to the results in the standard literature. Allen C, Mehler DMA (2019) Open science challenges, benefits and tips in early career and beyond. PLOS Biology 17(5): e3000246.
Journals should introduce registered reports as an article type. Here, the review of the intended study happens before the study is conducted, and after the study is conducted. Thus, the review process is slightly different than for a 'classical' manuscript. For journals that introduce registered reports, it is important that: 1) The handling time for the Stage 1 review is clear and relatively short, as the research cannot start before the review is finished. Further, making review time for registered reports shorter than for a classical article can further encourage the authors to submit registered reprots 2) The registered reports policy is explicit and defines the exact type of contribution that would be accepted as registered reports. While many journals offer registered reports for experimental research only, other types of research, such as observational studies or systematic reviews and meta-analyses are also suitable for this format
Evidence:
Not relevantFunders can further support registered reports by providing funding either for the publication of registered reports or for research projects that aim to be published as registered reports. For example, grant programmes of Cancer Research UK and Templeton World support research that will be published as a registered report.
Evidence:
Not relevant
Open data are data that are available to others to reuse. Open data should follow FAIR principles - be findable, accessible, interoperable, and reusable, both to machines and humans. Open data can mitigate several issues related to the inefficiencies in the research life cycle 1) even if the study was not published, the data can be published and thus of use to others 2) for studies that apply incorrect analysis, open data can be used for applying a correct analysis 3) for under-reported results, open data can help to understand the missing values either directly (e.g. sample size) or via re-analysis (e.g. effect size not reported) 4) open data can ensure wider accessibility of a study that is itself not open access, can be used within evidence synthesis, or for another purpose that increases the chances that a study reaches the intended audience
Evidence:
TBATo enable data archiving and sharing, funders and institutions should establish trusted, curated data repositories. These can be discipline-specific (e.g. FlyBase Homepage, Data Publisher for Earth & Environmental Science, SPI-Birds) or can host a variety of different types of data (e.g. Zenodo; Dryad | Home - publish and preserve your data)
Evidence:
TBD
Research institutions should hire dedicated data managers that would support researchers in creating data/software management plans, and FAIR data/software.
Evidence:
TBC
Research institutions, publishers, and journals should introduce open data / software policies, as many have already done (see here). The policies should be explicit, and compliance should be checked.
Evidence:
TBD
The review and synthesis of already existing evidence on a research topic can insure that research is relevant and also help in setting up a proper methodology for a new study. Research is relevant and needed if the existing evidence is lacking, or if the existing evidence is inconclusive. Previous studies can also inform new research on the best methodological approaches, but also on approaches that are suboptimal.
Evidence:
In progress.
Pre-print is a version of the manuscript that has not yet been peer-reviewed. Rarely, pre-print can also be the accepted version of the manuscript. Pre-prints allow for sharing of research regardless of its publication in a scientific journal. They are published on pre-print servers such as arXiv.org e-Print archive; bioRxiv.org - the preprint server for Biology; EcoEvoRxiv.
Evidence:
In progress.
For applied research, researchers should consider involving stakeholders in deciding on what research should be conducted (i.e. what research is of most interest to the stakeholders), and even when designing the study (e.g. deciding on outcome measured, sampling locations etc.).
Evidence:
Across several studies, public involvement was reported to have increased the quantity and quality of patient-relevant priorities and outcomes, enrollment, funding, design, an implementation. See more in Price A, Albarqouni L, Kirkpatrick J, Clarke M, Liew SM, Roberts N, Burls A. Patient and public involvement in the design of clinical trials: An overview of systematic reviews. J Eval Clin Pract. 2018 Feb;24(1):240-253. doi: 10.1111/jep.12805. Epub 2017 Oct 27. PMID: 29076631.
Specific journals can be started that publish articles that take stakeholder engagement seriously. For example, Research for All has brought together the research community conducting consultative work for stakeholder-informed research.
Evidence:
In progress.
Funders can request that grant applications demonstrate patient and public involvement. For example, The National Institute for Health and Care Research (NIHR), the UK’s largest funder of clinical healthcare research, requires researchers to demonstrate patient and public involvement when applying for grants, including, for example, by having patients with first-hand experience of the condition or phenomenon being researched serving on the funding panel or steering committee.
Evidence:
To be collected.
Funders and research institutions could set a training infrastructure to support researchers’ engagement efforts with stakeholders.
Evidence:
To be collected.