Recurrent Neural Networks for Complex Survival Problems

Abstract

In this study, we introduce a novel deep learning technique (CmpXRnnSurv_AE) that obliterates the limitations imposed by traditional approaches and addresses the limitations of the existing deep learning systems to jointly predict the risk-specific probabilities and survival function for recurrent events with competing risks. We introduce the component termed Risks Information Weights (RIW) as an attention mechanism to compute the weighted cumulative incidence function (WCIF) and an external auto-encoder (ExternalAE) as a feature selector to extract complex characteristics among the set of covariates responsible for the cause-specific events. We train our model using synthetic and real data sets and employ the appropriate metrics for complex survival models for evaluation. As benchmarks, we selected both traditional, and machine learning models and our model demonstrates better performance across all datasets with the best weighted time dependent concordant index score of 92% and the weighted time dependent Brier score of 20%. Keywords: Cumulative Incidence Function (CIF), Risk Information Weight (RIW), Autoencoders (AE), Survival analysis, Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM).