Autor(es): Curioso, Isabel ; Santos, Ricardo ; Ribeiro, Bruno ; Carreiro, André ; Coelho, Pedro ; Fragata, José ; Gamboa, Hugo
Data: 2023
Identificador Persistente: http://hdl.handle.net/10362/155244
Origem: Repositório Institucional da UNL
Projeto/bolsa: info:eu-repo/grantAgreement/FCT/3599-PPCDT/DSAIPA%2FAI%2F0094%2F2020/PT;
Assunto(s): Clinical data; Correlation; Machine learning; Missing data; Missing data imputation; Computer Science(all)
Funding Information: This work was done under the project “CardioFollow.AI: An intelligent system to improve patients’ safety and remote surveillance in follow-up for cardiothoracic surgery”. Publisher Copyright: © 2023 The Author(s)
Clinical data are essential in the medical domain. However, their heterogeneous nature leads to many data quality problems, notably missing values, which undermine the performance of Machine Learning-based clinical systems. Hence, there has been a growing interest in strategies that address this challenge in order to build trustworthy systems to improve the quality of care and benefit clinical decision-making. In particular, missing value imputation is a common approach. This paper proposes three novel imputation techniques that leverage correlation in an innovative manner by exploring the relationship between values and missingness patterns. Experiments were carried out on three publicly available datasets, under three missingness mechanisms with different missing rates, and on two real-world medical datasets. The imputation precision and the classification performance of the proposed techniques were evaluated in a comprehensive comparative study, which included diverse existing methods. The developed techniques outperformed state-of-the-art methods on several assessments while overcoming current flaws shared by correlation-based imputation strategies in real-world medical problems.
