Advertisement

Deep Natural Language Processing of Oncology Radiology Reports

Advertisement

Key Points

  • Researchers compared human and computer measurements of outcomes such as disease-free survival, progression-free survival, and time to improvement or response, and found that the AI algorithm could replicate human assessment of these outcomes.
  • Deep learning algorithms were then applied to annotate another 15,000 reports for 1,294 patients whose records had not been manually reviewed. The authors found that computer outcome measurements among these patients predicted survival with similar accuracy to human assessments among the manually reviewed patients.
  • The human curators were able to annotate imaging reports for about three patients per hour—a rate at which one curator would need about 6 months to annotate all of the nearly 30,000 imaging reports for the patients in the cohort. By contrast, the AI model that the researchers developed could annotate the imaging reports for the cohort in about 10 minutes.

Scientists have demonstrated that an artificial intelligence (AI) tool can perform as well as human reviewers—and much more rapidly—in extracting clinical information regarding changes in tumors from unstructured radiology reports for patients with lung cancer. These findings were published by Kehl et al in JAMA Oncology.

Background

The AI tool performed comparably to trained human “curators” in detecting the presence of cancer and whether it was responding to treatment interventions, stable, or worsening. 

The goal of the study, said corresponding author Kenneth Kehl, MD, MPH, a medical oncologist and faculty member of the Population Sciences Department at Dana-Faber Cancer Institute, was to determine whether AI tools can extract the most high-value cancer outcomes from radiology reports. Dr. Kehl noted that electronic health records now collect vast amounts of information on thousands of patients. However, unless the patients are enrolled in clinical trials, information about their outcomes is recorded only in the text of the medical record. Historically, this unstructured information is not amenable to computational analysis and therefore could not be used for research into the effectiveness of treatment. 

Many recent studies also analyze patient tumor samples and accumulated molecular information about patients’ cancers, “but it can be difficult to apply this information to understand what molecular patterns predict benefit from treatments without intensive review of patients’ medical records to measure their outcomes. This is a critical barrier to realizing the full potential of precision medicine,” said Dr. Kehl.

Methods

For the current study, Dr. Kehl and colleagues obtained over 14,000 imaging reports for 1,112 patients and manually reviewed records using the PRISSMM framework. PRISSMM is a phenomic data standard developed at Dana-Farber that takes unstructured data from text reports in electronic health records and structures them so that they can be readily analyzed. PRISSMM structures data pertaining to a patient’s pathology, radiology/imaging, signs/symptoms, molecular markers, and a medical oncologist’s assessment.

Human reviewers analyzed the imaging text reports and noted whether cancer was present and, if so, whether it was worsening or improving, and if the cancer had spread to specific body sites. These reports were then used to train a computational “deep learning” model to recognize these outcomes from the text reports. “Our hypothesis was that deep learning algorithms could use routinely generated radiology text reports to identify the presence of cancer and changes in its extent over time,” the authors wrote. 

Findings

The researchers compared human and computer measurements of outcomes such as disease-free survival, progression-free survival, and time to improvement or response, and found that the AI algorithm could replicate human assessment of these outcomes. The deep learning algorithms were then applied to annotate another 15,000 reports for 1,294 patients whose records had not been manually reviewed. The authors found that computer outcome measurements among these patients predicted survival with similar accuracy to human assessments among the manually reviewed patients.

The human curators were able to annotate imaging reports for about three patients per hour—a rate at which one curator would need about 6 months to annotate all of the nearly 30,000 imaging reports for the patients in the cohort. By contrast, the AI model that the researchers developed could annotate the imaging reports for the cohort in about 10 minutes.

“To create a true learning health system for oncology and to facilitate delivery of precision medicine at scale, methods are needed to accelerate curation of cancer-related outcomes from electronic health records,” said the authors. If applied widely, the investigators said, “this technique could substantially accelerate efforts to use real-world data from all patients with cancer to generate evidence regarding effectiveness of treatment approaches.”

Next steps will include testing this approach on electronic health record data from other cancer centers and using the data to discover which treatments work best for which patients.

Disclosure: The research was supported by grants from the National Cancer Institute, the Claude Dauphin Philanthropic Research Fund, and the Simeon J. Fortin Foundation. For full disclosures of the study authors, visit jamanetwork.com.

The content in this post has not been reviewed by the American Society of Clinical Oncology, Inc. (ASCO®) and does not necessarily reflect the ideas and opinions of ASCO®.


Advertisement

Advertisement




Advertisement