CAT-Former: A Causal Attention Transformer Framework for User Retention Prediction and Incremental Value Attribution in Advertising Systems Attacks

Abstract

In modern advertising systems, accurately predicting user retention while quantifying the incremental impact of advertising interventions is critical for optimizing budget allocation, improving return on investment, and enhancing long-term user engagement in large-scale digital platforms. However, existing approaches largely rely on correlation-based modeling, which fails to distinguish true causal effects from spurious associations induced by biased ad exposure and user selection mechanisms. To address these challenges, this study proposes CAT-Former (Causal Attention Transformer), a unified framework for user retention prediction and incremental value attribution in advertising systems. The proposed model integrates a Transformer-based encoder for capturing long-range dependencies across multi-channel advertising exposures and a Bidirectional Long Short-Term Memory (BiLSTM) network for modeling short-term behavioral dynamics. Furthermore, a counterfactual causal inference module is introduced to estimate potential outcomes under treated and untreated conditions, enabling precise uplift-based attribution of advertising effects. Extensive experiments conducted on large-scale advertising datasets demonstrate that CAT-Former achieves superior performance, improving AUC by 4.13%. These results highlight the effectiveness of incorporating causal reasoning into deep sequential models for advertising decision-making. The proposed framework provides a robust and interpretable solution for real-time retention optimization in complex advertising environments.