一种文本挖掘工具在早期可切除非小细胞肺癌常规临床数据提取中的验证
Validation of a Text-Mining Tool for Extracting Routine Clinical Care Data in Early-Stage Resectable Non-Small Cell Lung Cancer.
作者
作者单位
- Department of Clinical Pharmacy and Toxicology, Leiden University Medical Center, Leiden, the Netherlands.
- Department of Pathology, Leiden University Medical Centre, Leiden, the Netherlands.
- Department of Pulmonology, Leiden University Medical Centre, Leiden, the Netherlands.
摘要
中文
对电子健康记录(EHR)进行人工图表回顾(MR)耗时、易出错,且限制了真实世界数据(RWD)研究的可重复性和可扩展性。使用自然语言处理(NLP)实现自动化和标准化可提高效率和可扩展性。CTcue是一种基于NLP的软件平台,旨在从EHR中提取结构化和非结构化数据。本研究在早期可切除非小细胞肺癌(NSCLC)患者中评估了CTcue相对于MR的准确性和效率。纳入对象为2018年1月至2021年12月在荷兰莱顿大学医学中心接受肺切除术的所有I-III期NSCLC患者。收集人口统计学、肿瘤特征、治疗及结局数据。使用加权F1值、准确率、精确率和召回率对分类变量比较CTcue与MR的表现,使用Bland-Altman分析对连续变量进行比较。两种方法均识别出85例患者(占人工队列的70.2%),纳入比较分析。CTcue在15个分类变量中的7个上实现了>0.85的加权F1值,包括性别、肿瘤位置和死亡状态,尽管部分得分基于两个队列中观察例数较少。对于文档中术语多变的变量(如ECOG状态和病理N分期),表现较差。连续变量的平均差异可忽略不计,提示一致性良好。两组数据集的生存结局完全一致。CTcue在患者筛选方面的表现低于预期。然而,其在早期NSCLC中能够准确、高效地提取结构化和非结构化EHR数据。对于术语多变的变量,人工验证仍有必要。进一步开发基于人工智能的工具(特别是针对自由文本数据提取的工具)对于提升未来RWD研究的准确性和可扩展性至关重要。
English
Manual chart review (MR) of electronic health records (EHRs) is time-consuming, error-prone, and limits the reproducibility and scalability of real-world data (RWD) research. Automation and standardization using natural language processing (NLP) could improve efficiency and scalability. CTcue is an NLP-based software platform designed to extract structured and unstructured data from EHRs. This study evaluated the accuracy and efficiency of CTcue versus MR in patients with early-stage resectable non-small cell lung cancer (NSCLC). Included were all patients with stage I to III NSCLC who underwent lung resections between January 2018 and December 2021 at the Leiden University Medical Center, the Netherlands. Demographics, tumor characteristics, treatment, and outcomes were collected. CTcue performance was compared with MR using weighted F1-scores, accuracy, precision, and recall for categorical variables and Bland-Altman analysis for continuous variables. Eighty-five patients (70.2% of patients from the manual cohort) were identified by both methods and included in the comparison. CTcue achieved weighted F1-scores >0.85 for seven of 15 categorical variables, including sex, tumor location, and deceased status, although some scores were based on low number of observations in both cohorts. Lower performance was observed for variables with varying terminology in documentation, such as Eastern Cooperative Oncology Group status and pathological N-stage. Continuous variables showed negligible mean differences, indicating good agreement. Survival outcomes were identical in both data sets. CTcue performance for patient selection was lower than anticipated. However, it enables accurate, efficient extraction of structured and unstructured EHR data in early-stage NSCLC. Manual validation remains necessary for variables with varying terminology. Further development of artificial intelligence-based tools-particularly for free-text data extraction-will be crucial to enhance the accuracy and scalability of future RWD research.
分类与指标
- 研究类型
- AI/ML
- 病种
- 肺癌
- JCR 分区
- Q2
- 影响因子
- 3.6
- 新锐分区
- 3区