Vara and University of Lübeck finish largest prospective AI study (~0.5 million women) for breast cancer screening
April 2024: In participation with breast cancer screening units in the German Mammography Screening Programme, Vara and the University of Lübeck have conducted the first of its kind nationwide prospective observational study to evaluate the use of an AI application and workflow software to support breast cancer screening radiologists.
While many AI systems have demonstrated strong performance in simulations using historical screening data, prospective studies are the key to measuring the real-world effect of AI in clinical practice. These are required to determine how the use of AI will translate safely and efficiently to the clinical world.
The PRAIM Study (PRospective multicenter observational study of an integrated artificial intelligence (AI) system with live Monitoring) is led by Prof. Alexander Katalinic (University Hospital Schleswig Holstein, University of Lübeck) in collaboration with Vara. The study is embedded within the national population-based breast cancer screening programme and is guided clinically by an Advisory Board with some of the country's leading breast cancer screening radiologists. The study protocol was approved by the University of Lübeck’s Ethics Committee and is registered in the German Clinical Trials Register. Primary endpoints were screen-detected breast cancer detection and recall rates.
The study has concluded and is currently in academic peer review. 463,094 women were screened from July 2021 to February 2023 in 12 screening sites in Germany. A total of 119 radiologists participated in the study. The study did not exclude any subpopulations of women, screening sites, radiologists (for example based on their years of professional experience), and featured a wide range of five different hardware vendors. Extensive subgroup results across screening round, breast density, age, cancer invasiveness, stage, grading, and size will be reported.
To date, less than a handful of prospective studies have been conducted globally which investigate the performance of AI solutions, and more importantly, how radiologists interact with this technology. These studies are limited by small sample sizes, inclusion of only a single machine manufacturer, or a few screening sites and radiologists. The PRAIM study has sites open across Germany. The wide-scale involvement of important stakeholders in the German Mammography Screening Programme allowed for greater participation of screening units, a diverse study population, and a quick turnaround time for results. This is key for a rapidly evolving technology with growing adoption around the world.
The PRAIM study takes on a unique study design, allowing for direct observation of radiologist interaction with the technology, and comparison to current standard double reading. Important screening-related metrics can be compared between mammograms read with AI assistance, and those not. The study is observational, meaning women will undergo routine mammography screening following local guidelines.
Prof. Alexander Katalinic, University Hospital Schleswig Holstein, Lübeck, said: “We wanted to understand how this CE-marked technology is currently being used in German mammography screening units: what works, and what doesn't. Using an observational study design was an important strategic decision to take. Such a study design was feasible, reflects the real healthcare situation and we could observe outcomes for more women in a quite short period of time. This way we could disseminate important findings more quickly to the radiologists who await this evidence.”
Vara's AI uses a unique decision referral approach that leverages the strengths of both the radiologist and the AI algorithm. This two-part system incorporates triage of normal exams with high accuracy, while also introducing a “safety net” to maintain a high degree of sensitivity by performing predictions on the presence or absence of cancerous findings as post-hoc decision support. The decision referral approach has already been shown to improve the screening accuracy (sensitivity and specificity) of an average German radiologist based on a retrospective evaluation of screening data from eight screening units in the German Mammography Screening Programme (peer-reviewed publication in Lancet Digital Health).
The PRAIM study is being supervised by an Advisory Board made up of the following breast imaging experts:
- Dr. Gerold Hecht, Reference Center Mammography North
- Professor Sylvia Heywang-Köbrunner, Reference Center Mammography Munich
- Professor Katja Siegmann-Luz, Reference Center Mammography Berlin
- Dr. Timo Gomille, Visiorad, Pinneberg
- Dr. Thilo Töllner, Klinik Dr. Hancken, Stade
- Dr. Toni Vomweg, Radiologisches Institut Dr. von Essen, Koblenz
- Regine Rathmann, Praxis Schwarzer Bär, Hannover