Deutsch
 
Datenschutzhinweis Impressum
  DetailsucheBrowse

Datensatz

DATENSATZ AKTIONENEXPORT
 
 
DownloadE-Mail
  Lokale TagsFreigabegeschichteDetailsÜbersicht

Freigegeben
Zeitschriftenartikel

Cochrane evaluation of (semi-)automated review methods: protocol for an adaptive platform study within reviews

Urheber*innen

Gartlehner,  Gerald
External Organizations;

Banda,  Susan
External Organizations;

/persons/resource/max.callaghan

Callaghan,  Max       
Potsdam Institute for Climate Impact Research;

Chase,  Jo-Ana
External Organizations;

Dobrescu,  Andreea
External Organizations;

Eisele-Metzger,  Angelika
External Organizations;

Flemyng,  Ella
External Organizations;

Gardner,  Sean
External Organizations;

Griebler,  Ursula
External Organizations;

Helfer,  Bartosz
External Organizations;

Jemiolo,  Pawel
External Organizations;

Macura,  Biljana
External Organizations;

/persons/resource/jan.minx

Minx,  Jan C.       
Potsdam Institute for Climate Impact Research;

Noel-Storr,  Anna
External Organizations;

Tahmasebi,  Noosheen Rajabzadeh
External Organizations;

Sharifan,  Amin
External Organizations;

Meerpohl,  Joerg J.
External Organizations;

Thomas,  James
External Organizations;

Externe Ressourcen
Es sind keine externen Ressourcen hinterlegt
Volltexte (beschränkter Zugriff)
Für Ihren IP-Bereich sind aktuell keine Volltexte freigegeben.
Volltexte (frei zugänglich)

1-s2.0-S0895435626002660-main.pdf
(Verlagsversion), 2MB

Ergänzendes Material (frei zugänglich)
Es sind keine frei zugänglichen Ergänzenden Materialien verfügbar
Zitation

Gartlehner, G., Banda, S., Callaghan, M., Chase, J.-A., Dobrescu, A., Eisele-Metzger, A., Flemyng, E., Gardner, S., Griebler, U., Helfer, B., Jemiolo, P., Macura, B., Minx, J. C., Noel-Storr, A., Tahmasebi, N. R., Sharifan, A., Meerpohl, J. J., Thomas, J. (2026): Cochrane evaluation of (semi-)automated review methods: protocol for an adaptive platform study within reviews. - Journal of Clinical Epidemiology, 198, 112390.
https://doi.org/10.1016/j.jclinepi.2026.112390


Zitierlink: https://publications.pik-potsdam.de/pubman/item/item_35123
Zusammenfassung
Background and Objectives: Artificial intelligence (AI) has the potential to improve the efficiency of evidence synthesis and reduce human error. However, robust methods for evaluating rapidly evolving AI tools within the practical workflows of evidence synthesis remain underdeveloped. This protocol describes a study design for assessing the effectiveness, efficiency, and usability of AI tools in comparison to traditional human-only workflows in the context of Cochrane systematic reviews.
Methods: Members of the Cochrane Evaluation of (Semi-)Automated Review Methods (CESAR) project developed an adaptive platform study-within-a-review design, modeled after clinical platform trials. This design employs a master protocol to concurrently evaluate multiple AI tools (interventions) against a standard human-only process (control) across 3 key review tasks: title and abstract screening,
full-text screening, and data extraction. The adaptive framework allows for the addition or removal of AI tools based on interim performance analyses without necessitating a restart of the study. Performance will be assessed using metrics such as accuracy (sensitivity, specificity, precision), efficiency (time on task), response stability, impact of errors, and usability, in alignment with Responsible use of AI in
evidence SynthEsis principles.
Results: The study will generate comparative data about the performance and usability of specific AI tools used in a semiautomated or fully automated manner relative to standard human effort. The protocol provides a flexible framework for the assessment of AI tools in evidence synthesis, addressing the limitations of static, one-time evaluations.
Conclusion: This study protocol presents a novel methodological approach to addressing the challenges of evaluating AI tools for evidence syntheses. By validating entire workflows rather than individual technologies, the findings will establish an evidence base for determining the viability of integrating AI into evidence synthesis workflows. The adaptive design of this study is flexible and can be adopted by other investigators, ensuring that the evaluation framework remains relevant as new tools emerge.