Informatics association challenge elicits de-identification solutions

Article

Medical records are an important source for clinical researchers, but text records used outside the hospital must first be de-identified.

Medical records are an important source for clinical researchers, but text records used outside the hospital must first be de-identified.

Several automatic schemes to achieve de-identification recently surfaced as the result of a challenge issued jointly by the American Medical Informatics Association and i2b2 (informatics for integrating biology and the bedside), an NIH-funded National Center for Biomedical Computing project based at Partners HealthCare System. An overview of participating systems, details of evaluation metrics, and findings can be found in the Association's journal (J Am Med Inform Assoc 2007;14:550-563).

In one effort, researchers at the University of Szeged in Hungary developed a de-identification model that successfully removes personal health information from hospital records, in conformance with the Health Insurance Portability and Accountability Act.

The system is a machine learning-based iterative that uses a named entity recognition approach on semistructured documents. Named entity recognition (NER) is a subtask of information extraction that locates and classifies elements in text into predefined categories.

"Our named entity approach is based on a complex feature set and boosted decision trees, and it uses a different feature representation from other state-of-the-art NER systems," said György Szarvas, Ph.D., of the university's informatics department.

Szarvas's method identifies personal health information in several steps (J Am Med Inform Assoc 2007;14:574-580). First, it labels all entities whose tags can be inferred from the structure of the text, and it then utilizes this information to find further personal health information phrases in the flow text parts of the document.

Szarvas said that customizing his system took only a few weeks.

"Such systems can be built quite rapidly for any institute for de-identification or other NER-like tasks," he said.

Elsewhere, researchers at MITRE (Bedford, MA), along with Harvard, Brandeis, and Stanford universities, took a different approach, focusing instead on rapid adaptation of existing toolkits for named entity recognition. They used two existing tools: Carafe and LingPipe.

The researchers report that the out-of-the-box Carafe system achieved a good score (phrase F-measure of 0.9664) with only four hours of work to adapt it to the de-identification task (J Am Med Inform Assoc 2007;14:564-573). With further tuning, they were able to reduce the token-level error term by over 36% through task-specific feature engineering and the introduction of a lexicon, achieving a phrase F-measure of 0.9736.

Challenge organizer Özlem Uzuner, Ph.D., of the University at Albany, SUNY, said the efforts show that most private health information can be de-identified with more than 98% accuracy.

Whether 98% accuracy is good enough is an open question best left to policymakers, she said.

"The results are nevertheless encouraging from a technical perspective and show that much can be accomplished to de-identify data with the best techniques," Uzuner said.

AMIA was so encouraged it is now in the process of organizing similar challenges on other open research questions in medical language processing.

Newsletter

Stay at the forefront of radiology with the Diagnostic Imaging newsletter, delivering the latest news, clinical insights, and imaging advancements for today’s radiologists.

Recent Videos
SNMMI: Emerging PET Insights on Neuroinflammation with Progressive Apraxia of Speech (PAOS) and Parkinson-Plus Syndrome
Improving Access to Nuclear Imaging: An Interview with SNMMI President Jean-Luc C. Urbain, MD, PhD
SNMMI: 18F-Piflufolastat PSMA PET/CT Offers High PPV for Local PCa Recurrence Regardless of PSA Level
SNMMI: NIH Researcher Discusses Potential of 18F-Fluciclovine for Multiple Myeloma Detection
SNMMI: What Tau PET Findings May Reveal About Modifiable Factors for Alzheimer’s Disease
Emerging Insights on the Use of FES PET for Women with Lobular Breast Cancer
Can Generative AI Reinvent Radiology Reporting?: An Interview with Samir Abboud, MD
Mammography Study Reveals Over Sixfold Higher Risk of Advanced Cancer Presentation with Symptom-Detected Cancers
Combining Advances in Computed Tomography Angiography with AI to Enhance Preventive Care
Study: MRI-Based AI Enhances Detection of Seminal Vesicle Invasion in Prostate Cancer
Related Content
© 2025 MJH Life Sciences

All rights reserved.