Off-Policy Evaluation and Learning for Survival Outcomes Under Censoring

1 Introduction

Evaluating and implementing policies (treatment strategies) aimed at maximizing survival outcomes presents a critical challenge in decision-making in diverse real-world domains, ranging from extending survival in healthcare [Geng et al., 2015; Murphy, 2003] to reducing customer churn in marketing [Ascarza et al., 2018]. The core focus of these applications is not limited to evaluating short-term effects, but encompasses a broader understanding of a policy's impact on the long-term survival trajectory.

Off-policy evaluation (OPE) provides a powerful framework to estimate the performance of new decision-making policies using only offline logged data, which is crucial in domains where online experiments are costly or impractical [Wang et al., 2017]. Conventional approaches, such as Inverse Propensity Scoring (IPS) [Horvitz and Thompson, 1952; Rosenbaum and Rubin, 1983] and Doubly Robust (DR) [Dud´ık et al., 2011; Robins and Rotnitzky, 1995], have been widely adopted for evaluating policies using offline logged data collected under different (logging) policies.

However, despite their broad applicability, these typical estimators are not designed to handle right-censored survival outcomes. This is because these estimators are designed primarily for immediate rewards or outcomes defined over a fixed time horizon. They also implicitly assume that all outcomes are fully observed, which is directly violated by right-censoring, a defining characteristic of survival data. A naive application of these estimators cannot distinguish censoring from true event occurrence, and therefore wrongly treats observed times as final event times. This induces systematic underestimation bias because typical methods fail to account for potential survival beyond the censoring point.

To address this issue, we propose a novel framework for off-policy evaluation and learning under practical constraints designed for censored survival outcomes. Specifically, we develop new statistical estimators, namely Inverse Probability of Censoring Weighted-Inverse Propensity Scoring (IPCW-IPS) and IPCW-Doubly Robust (IPCW-DR), which explicitly consider censoring induced bias. IPCWIPS additionally re-weights logged data via estimated censoring probabilities beyond the typical importance weighting regarding policy probabilities. IPCW-DR further incorporates outcome modeling, which enjoys the doubly robust property with respect to the propensity score and the outcome model. Consequently, it ensures reliability in practical applications and yields lower variance compared to IPCW-IPS. Furthermore, we extend our framework to the problem of constrained Off-Policy Learning (OPL), enabling policy optimization for survival outcomes subject to practical budget constraints.

Finally, we empirically demonstrate the effectiveness of our proposed framework. The simulation results demonstrate that our proposed estimators, particularly IPCW-DR, achieve superior accuracy than standard OPE estimators. Moreover, the application to the Starbucks Customer Reward Program dataset highlights the practical effectiveness of our approach in complex scenarios. Specifically, we demonstrate how our framework can better optimize promotional offers to minimize the customer purchase cycle under budget constraints. Our key contributions can be summarized as follows.

• We develop censoring aware estimators for off-policy evaluation of survival outcomes leveraging the technique of inverse probability of censoring weighting and provide their theoretical advantages.
• We introduce an iterative algorithm for off-policy learning with practical constraints that targets optimization of survival outcomes.
• We demonstrate the effectiveness of the proposed methods in terms of both evaluation and learning tasks, through simulation studies and a real world application.

Due to space limitations, we provide a comprehensive review of related work in Appendix A.

2 Preliminaries

2.1 ProblemFormulation

Here we consider the OPE problem for an evaluation policy ωe under right-censoring, using data collected by a logging policy ω0. Let x → X be the context vector representing individual characteristics, a → A be the discrete action selected according to a policy ω(a ♣ x). The logged dataset D = ¶(xi↪ ai↪ Ti↪ ri)♦ni=1 is collected under a logging policy ω0, where the underlying random variables are generated as

Here, Li and Ci denote the true (latent) survival time and the censoring time, respectively. The observed data consist only of the observed time Ti = min¶Li↪Ci♦ and an event indicator ri = I¶Li ↓ Ci♦. Thus, for censored individuals (ri = 0), we only know that their true survival time exceeds the observed time Ti. Our goal is to accurately estimate the performance (policy value) of an evaluation policy ωe regarding the true survival time using only the logged data D.

Policy Value Definition. A key quantity of interest in our work is the expected survival probability at an arbitrary time t. Let S(x↪ a↪ t) := P(L > t ♣ x↪ a) be the survival function given context x and action a. The expected survival probability under policy ωe at time t, as a type of policy value, is defined as

Another important quantity of interest is the Restricted Mean Survival Time (RMST) [Uno et al., 2014], which is the expected survival time up to a pre-specified finite time horizon ε . This version of the policy value is defined as follows.

Note that ε is not a tunable hyperparameter but a domeinspecific objective that defines the evaluation metric itself.

Assumptions. We begin by outlining some regularity assumptions necessary for analyzing the theoretical properties of OPE estimators under censoring. First, the following common support assumption, standard in the OPE literature [Sachdeva et al., 2020], ensures that the evaluation policy does not place a positive probability on actions that the logging policy never takes. It is formally stated as follows.

Assumption 2.1 (Common Support). If ωe(a ♣ x) > 0, then ω0(a ♣ x) > 0 for all x → X and a → A.

We also introduce an assumption regarding censoring.

Assumption 2.2 (Conditional Independent Censoring). Given the context x and action a, the true survival time L and the censoring time C are conditionally independent

This assumption serves as the theoretical basis for conventional methods in survival analysis, such as the Kaplan-Meier estimator and the Cox proportional hazards model [Fleming and Harrington, 2013].

2.2 Naive OPE Under Censoring

A naive approach to addressing this estimation problem is to directly apply conventional estimators in OPE, such as IPS or DR, to the logged dataset ignoring the censoring issue [Saito and Joachims, 2021]. These standard estimators are typically designed for fully observed outcomes and, when naively applied to survival data, they treat the observed time Ti as the true event time regardless of the censoring indicator ri. Specifically, by treating the observed indicator I¶Ti > t♦ as the reward, the naive IPS estimator is given by

We can also define the naive DR estimator by combining IPS with an outcome model ˆ S(x↪ a↪ t) trained on the observed times. 1 Since these estimators are not designed to account for the censoring mechanism, they suffer from systematic underestimation bias, explicitly characterized as follows.

Proposition 2.3. Under the presence of censoring (that is, P(L > C) > 0) and Assumptions 2.1 and 2.2, the naive IPS and DR estimators are biased as

where G(t ♣ x↪ a) := P(C > t ♣ x↪ a) is the censoring survival function.

1 We formally define naive DR in Eq. (D.1) in Appendix D.

The proof is provided in Appendix D.1. Since G(t ♣ x↪ a) ↓ 1, the bias term is always non-positive, meaning that these naive estimators systematically underestimate the policy value, as visually depicted in Figure 1. We can also see that the bias increases under severe censoring conditions, such as short follow-up periods or low event rates. In such cases, G(t ♣ x↪ a) becomes significantly smaller than one, resulting in a more severe underestimation bias.

3 Proposed Approach: Censoring-Aware Off-Policy Evaluation and Learning

To obtain unbiased estimates of expected survival probability or RMST, it is essential to incorporate the censoring mechanism directly into the OPE framework. This section proposes two new estimators that properly account for censoring and yield unbiased or at least bias-reduced policy value estimates.

3.1 Unbiased Estimation of Survival Policy Value

Our first proposed estimator, namely Inverse Probability of Censoring Weighted-Inverse Propensity Scoring (IPCWIPS), directly corrects for the censoring bias. As shown in the derivation of Proposition 2.3 (see Appendix D.1), the expectation of the naive outcome I¶Ti > t♦ yields the confounded quantity S(xi↪ ai↪ t)G(t ♣ xi↪ ai) rather than the true survival probability S(xi↪ ai↪ t). To deal with this bias, we propose to employ the Inverse Probability of Censoring Weighting (IPCW) technique [Robins et al., 1995; Robins and Finkelstein, 2000]. Specifically, we define IPCW-IPS by additionally weighting each observed indicator I¶Ti > t♦ with the inverse of the estimated probability that the sample remains uncensored at time t, denoted by ˆG(t ♣ xi↪ ai) as

where w(xi↪ ai) := ωe(ai ♣ xi)◁ω0(ai ♣ xi).

While IPCW-IPS is designed to be unbiased, it may be susceptible to high variance particularly when censoring is heavy (i.e., when ˆG(t ♣ x↪ a) is small), as the inverse probability weight becomes large. To mitigate this variance inflation and improve estimation reliability further, we extend IPCW-IPS to IPCW-Doubly Robust (IPCW-DR) defined as

ˆ

The IPCW-DR estimator extends the standard DR framework to the setting of censored survival outcomes. Structurally, this estimator combines a model-based estimate derived via the direct method (DM) [Beygelzimer and Langford, 2009], with a bias-correction term based on the weighted residual between the IPCW-adjusted reward and the outcome model.

We first establish the unbiasedness of our proposed estimators. The following theorem guarantees that both IPCW-IPS

and IPCW-DR provide unbiased estimates of the policy value, with IPCW-DR further enjoying double robustness.

Theorem 3.1. Under Assumptions 2.1 and 2.2, provided that the censoring model G(t ♣ x↪ a) is correctly specified

1. The IPCW-IPS estimator is unbiased for the expected survival probability V (ωe↪ t) if the propensity score ω0(a ♣ x) is correctly specified.
2. The IPCW-DR estimator is unbiased for V (ωe↪ t) if either the propensity score ω0(a ♣ x) or the outcome model ˆ S(x↪ a↪ t) is correctly specified.

The proofs are provided in Appendix D.2.

While unbiasedness is a desirable statistical property for OPE, the variance of an estimator is an equally critical component of the MSE. The following theorem explicitly characterizes the variance of our estimators and theoretically establishes the variance reduction property of IPCW-DR.

Theorem 3.2. Under Assumptions 2.1 and 2.2, provided that the propensity score ω0(a ♣ x), the censoring model G(t ♣ x↪ a), and the outcome model S(x↪ a↪ t) are correctly specified, the variance of the IPCW-IPS estimator is

where ϑ2 (x↪ a↪ t) := S(x↪ a↪ t)◁G(t♣x↪ a) ↗ S(x↪ a↪ t)2 represents the variance inflation due to censoring. Furthermore, the variance of the IPCW-DR estimator is

Consequently, IPCW-DR achieves a variance reduction of Ep(x)[Vω0(a♣x)[w(x↪ a)S(x↪ a↪ t)]] ↘ 0, ensuring V[ ˆ VIPCW-DR] ↓ V[ ˆ VIPCW-IPS]. The proofs are provided in Appendix D.3.

This theorem provides a formal justification for the instability of IPCW-IPS under heavy censoring. Specifically, the term ϑ2 (x↪ a↪ t) reveals that the variance is inversely proportional to the censoring survival function G(t ♣ x↪ a). Thus, when censoring is heavy (i.e., G(t ♣ x↪ a) is small), IPCWIPS can suffer from significant variance. In contrast, IPCWDR consistently reduces variance by leveraging the outcome model, making it a more reliable choice in practice.

While we have so far focused on estimating the pointwise survival probability V (ωe↪ t), our framework naturally extends to the estimation of the RMST V ε (ωe) defined in Eq. (3). Since the RMST is the integral of the survival probability, we can construct unbiased estimators for RMST by integrating IPCW-IPS and IPCW-DR over the time horizon [0↪ ε]. The detailed formulations and theoretical guarantees for these RMST estimators are provided in Appendix B.1.

3.2 Off-Policy Learning for Survival Maximization

In addition to the problem of OPE, we can formulate the problem of learning an optimal policy to maximize the survival outcome under censoring. Let ! be a predefined policy class such as neural networks. The goal of off-policy learning is to find a policy ωϑ → ! that maximizes the policy value based on the logged data D. Due to space limitations, we will focus on the policy learning objective for the expected survival probability V (ω↪ t). 2 Using our proposed estimators, we can define the policy learning objectives as ϖ := argmaxϑ ˆ V (ωϑ↪ t; D).

A typical approach to solve this optimization problem is to use gradient-based approach [Huang and Jiang, 2020; Saito et al., 2024], which updates the policy parameter via iterative gradient ascent as ϖk+1 ≃ ϖk + ⇐ϑV (ωϑ↪ t). Because the true gradient

is unknown, we need to estimate it from the logged dataset D where we can apply our proposed estimators. For example, IPCW-IPS for the policy gradient can be constructed by replacing the true survival probability S(x↪ a↪ t) in Eq. (6) with the IPCW-corrected outcome I¶T > t♦◁ ˆG(t ♣ x↪ a) as

where w→ (x↪ a) := ωϑ(a ♣ x)◁ω0(a ♣ x). Similarly, the IPCWDR estimator for the policy gradient can be constructed by replacing the true survival probability S(x↪ a↪ t) in Eq. (6) with the IPCW-DR outcome model as follows.

These estimators are unbiased against the true policy gradient ⇐ϑV (ωϑ↪ t) under the same conditions as Theorem 3.1. IPCW-DR for the policy gradient is superior to IPCW-IPS in terms of variance under the same conditions as in Section 3.1, leading to a more efficient policy learning.

Extension to Constrained Off-Policy Learning. We can further extend our OPL framework to optimize survival outcomes under a budget constraint, a common requirement in many real-world applications [Cai et al., 2023; Zhang et al., 2021; Zhou et al., 2023].

Let c(x↪ a) be the observed cost for a given (x↪ a). We aim to find ωϑ that maximizes V (ωϑ↪ t) subject to C(ωϑ) := Ep(x)ωω(a♣x)[c(x↪ a)] ↓ B. This constrained optimization problem can be transformed using Lagrangian duality [Geoffrion, 1974] into a min-max problem as minϖ↑0 maxϑ L(ωϑ↪ t↪ ς), where L(ωϑ↪ t↪ ς) = V (ωϑ↪ t) ↗

2 Note that the formulation for maximizing RMST proceeds analogously by integrating the objective over the time horizon.

ς(C(ωϑ) ↗ B). To solve this problem, we can employ an iterative algorithm that alternates between updating ϖ (gradient ascent on L) and the Lagrangian multiplier ς. The true gradient of the Lagrangian is

where Rϖ(x↪ a↪ t) = S(x↪ a↪ t) ↗ ςc(x↪ a). Since c(x↪ a) is fully observed and independent of censoring, we apply the IPCW correction exclusively to the survival component S(x↪ a↪ t). Thus, we can naturally obtain the IPCW-IPS and IPCW-DR estimators for ⇐ϑL by simply replacing the reward term in Eqs. (7) and (8) with the Lagrangian reward (I¶Ti > t♦◁ ˆG(t ♣ x↪ a) ↗ςc(xi↪ ai)) and its modelaugmented counterpart, respectively. The unbiasedness of these estimators against the true Lagrangian policy gradient is proved in Appendix D.7. This approach enables stable offline policy learning that rigorously satisfies budget constraints while handling the censoring issue.

4 Simulation Studies

This section demonstrates the effectiveness of our proposed methods in controlled simulated environments, which allow us to systematically analyze their behavior under known and carefully varied experimental conditions.

Data Generating Process. We construct the logged dataset D = ¶(xi↪ ai↪ Ti↪ ri)♦ni=1 to simulate OPE/L for survival outcomes under right-censoring. The data-generating process samples context vectors x, selects actions a using a logging policy ω0, and generates latent survival times L along with censoring times C. This design enables direct control over the degree of censoring. Detailed descriptions of the datagenerating process are provided in Appendix C.1.

OPE Performance Comparison. In the OPE experiments, we compare IPCW-IPS and IPCW-DR with the naive baselines including DM, IPS and DR, all of which ignore the censoring mechanism. We investigate the OPE accuracy under varying experimental conditions defined by three key factors: (i) training sample size n, (ii) censoring rate φ1, and (iii) policy divergence ↼. First, we vary n → ¶500↪ 1↪ 000↪ 2↪ 000↪ 5↪ 000↪ 10↪ 000♦ to evaluate asymptotic behavior of the estimators. Second, we vary the censoring rate φ1 = P(L > C) from ¶0▷1↪ 0▷2↪ 0▷3↪ 0▷4↪ 0▷5♦ to test robustness against heavy censoring. We obtain each target φ1 by numerically adjusting the censoring distribution parameters, as described in Appendix C.1. Third, to analyze the impact of the discrepancy between the logging and evaluation policies, we define the evaluation policy ωe using the ↼-greedy rule:

We configure the logging and evaluation policies in accordance with the synthetic experimental protocols in prior relevant studies [Kiyohara et al., 2024; Peng et al., 2023]. By varying ↼ → ¶0▷1↪ 0▷2↪ 0▷3↪ 0▷4↪ 0▷5♦, we examine how the divergence between the logging policy ω0 and the target policy ωe affects estimation variance.

(c) Impact of policy divergence ε on OPE performance.

Figure 2: OPE performance comparison across key experimental factors. The figure illustrates how MSE changes as we vary (a) the logged data size n, (b) the censoring rate ω1, and (c) the divergence between the logging and evaluation policies ε.

We evaluate and compare the MSE, squared bias, and variance of the estimators over 100 independent trials. In practice, constructing our estimators requires estimating nuisance components such as the censoring model ˆG(t ♣ x↪ a). To ensure computational stability under heavy censoring, we estimate this using Cox proportional hazards models with L2 regularization. Additional implementation details are provided in Appendix C.2 and our code to reproduce the results is shared as a supplementary material.

OPL Performance Comparison. We evaluate the performance of OPL methods in learning a policy ωϑ, parameterized by a Multi-Layer Perceptron (MLP) [LeCun et al., 2015], to maximize the expected RMST. Our comparison includes two learning paradigms, including a regression-based approach [Jeunen and Goethals, 2021], which selects actions using RMST predictions from an outcome model, and policy gradient-based approaches [Huang and Jiang, 2020], which optimize differentiable OPE estimators directly as objective functions. We analyze the learning performance under varying training sample sizes n and censoring rates φ1. We evaluate the learned policies on an independent large-scale test set using the improvement ratio over the logging policy, defined as V ε (ˆω)◁V ε (ω0). Additional implementation details are provided in Appendix C.3.

Simulation Results. Figure 2 illustrates the OPE performance comparison results across varying logged data sizes n, censoring rates φ1, and policy divergences ↼.

Figure 2a interestingly shows that naive IPS and DR do not reduce squared bias even when we increase the sample size, and their MSE remains high. This observation directly reflects Proposition 2.3, which explains how censoring induces systematic underestimation for those typically unbiased estimators. In contrast, IPCW-IPS and IPCW-DR steadily lower both MSE and squared bias as n grows, even in the existence of severe censoring, and these trends align with the unbiasedness guarantees in Theorem 3.1. DM achieves the smallest variance, but it retains a non-vanishing and substantial bias because the outcome model alone cannot fully capture the underlying survival dynamics. IPCW-DR corrects this bias while maintaining a strong bias-variance balance.

Next, in Figure 2b, we observe that the performance of naive estimators deteriorates rapidly as the censoring rate increases. Our IPCW-based estimators, however, continue to produce much lower MSE even under heavy censoring.

Figure 3: OPL performance comparison. The improvement ratio V ω (ˆϑ)◁V ω (ϑ0) quantifies how effectively each method learns a superior policy compared to the logging policy optimalities.

IPCW-DR, in particular, avoids the variance inflation that often appears in high-censoring settings, which highlights its robustness. Moreover, IPCW-DR remains highly stable under high policy divergence as shown in Figure 2c, whereas naive estimators continue to show unreliable accuracy.

Figure 3 summarizes the OPL results. In Figure 3a, policies learned based on IPCW-IPS and IPCW-DR achieve substantially higher improvement ratios compared to policy learners based on naive IPS and DR. Naive methods frequently underestimate survival times and thus fail to choose optimal actions, while IPCW-based learners appropriately deal with this bias and consistently identify better policies. Our methods continue to outperform naive learners as censoring increases as well (Figure 3b). These results demonstrate that our IPCW-based approaches achieve robust and reliable policy improvements across a range of realistic conditions.

5 Real-World Case Study: Starbucks Customer Retention

This section explores the practical effectiveness of our proposed estimators through a semi-synthetic experiment based on the Starbucks Customer Reward Program dataset 3 . Realworld datasets capture rich behavioral patterns, but they rarely offer counterfactual outcomes or complex censoring structures that allow a rigorous comparison of off-policy estimators or learners. To address these limitations and to perform yet a meaningful experiment, we first construct a datadriven "Oracle" environment directly from the Starbucks mobile app logs. The dataset contains detailed customer information such as age, income, membership tenure, and gender, together with offer attributes such as difficulty, reward amount, delivery channel, and duration. We define a discrete action space A containing ten offer types. In this environment, the survival time T measures how long a user takes to make the first purchase after receiving an offer. Our goal in this environment is to evaluate policies that shorten this purchase cycle, which corresponds to minimizing RMST over the validity period. We treat a purchase within the period as an event and regard all remaining observations as censored.

3 https://www.kaggle.com/datasets/blacktile/ starbucks-app-customer-reward-program-data

Furthermore, to examine estimator robustness under challenging censoring mechanisms, we introduce artificial censoring that depends on user demographics as their features. The original dataset mainly reflects simple Type I censoring due to offer expiration, which does not adequately stresstest modern OPE estimators. By generating censoring times that interact with demographic features, we create scenarios where the random censoring assumption fails in a controlled and interpretable way. This setting allows our experiment to reveal when naive estimators fail and how IPCW-based corrections address those failures. We describe the full data construction procedure and the censoring mechanism for this experiment in Appendix C.5.

OPE Results. Table 1 summarizes the OPE results on the Starbucks dataset. As shown in Table 1, IPCW-IPS and IPCW-DR deliver a clear and substantial reduction in MSE compared with IPS and DR. Both IPS and DR accumulate substantial squared bias, a pattern that directly reflects the theoretical insight shown in Proposition 2.3. In contrast, IPCW-IPS and IPCW-DR control squared bias effectively, demonstrating that the proposed correction mechanism performs reliably even in realistic settings that include complex and feature-dependent censoring structures.

We can also see from the table that DM records the smallest variance, but its large bias exposes its heavy reliance on the predictive accuracy of the outcome model. When the outcome model is misspecified, DM suffers from substantial bias regardless of the presence of censoring. IPCW-DR, however, counterbalances this weakness, i.e., although it shows slightly larger bias than IPCW-IPS, it significantly reduces variance and achieves the lowest overall MSE among all estimators. This result highlights how the doubly robust structure stabilizes the offline estimation process. Thanks to its doubly robust structure, IPCW-DR significantly reduces variance compared to IPCW-IPS. These findings highlight the practical value of our estimators for real-world marketing and retention. By consistently outperforming baselines even under complex censoring patterns and non-linear dynamics, IPCWIPS and IPCW-DR empower practitioners to derive effective interventional strategies from logged data that shorten purchase cycles and strengthen long-term engagement.

Constrained OPL Analysis. We further investigate the practical utility of our framework by evaluating the constrained OPL methods on the semi-synthetic Starbucks dataset. This experiment follows the same "Oracle" construction used in the OPE analysis.

To set up the constrained optimization problem, we first define the cost function c(x↪ a) based on the reward amount attached to each offer. In line with common performancebased marketing practices, we charge this cost only when a user converts; otherwise, the cost remains zero. Next, we set the budget constraint B to 80% of the average cost incurred by the logging policy (B = 2▷82), creating a realistic and challenging setting in which the learner must deliver faster customer engagement (lower RMST) while operating under a substantially reduced spending allowance.

We compare our Constrained IPCW-IPS and Constrained IPCW-DR with three constrained baselines: Constrained Regression, Constrained IPS, and Constrained DR. We solve the constrained optimization problem using Lagrangian duality to enable gradient-based learning. Appendix C.5 provides full details regarding the baseline implementations, neural network architecture, and optimization hyperparameters.

Table 2 reports the constrained OPL results on the Starbucks dataset. We observe that our IPCW-DR learner delivers the strongest RMST performance and clearly outperforms both naive OPE-based learners and the model-based regression approach. The naive IPS and DR learners underestimate survival outcomes as they ignore the censoring issue during policy learning, and the regression baseline performs even worse because it relies entirely on an imperfect outcome model. In contrast, IPCW-DR combines IPCW correction with a doubly robust structure, allowing the learner to optimize survival outcomes more effectively even under strict resource constraints. The IPCW-IPS and IPCW-DR

learners also reduce the standard deviation of RMST substantially compared with naive approaches, indicating that our censoring-aware corrections create more stable gradient signals throughout the policy optimization process. Naive estimators unintentionally inject noise into the learning signal because they interpret censored samples as early events, leading to their unstable learning performances. Our IPCWbased learners avoid this distortion and update the policy using signals that reflect the underlying counterfactual survival outcomes more accurately. As a result, they repeatedly derive high-quality policies across different data samples and demonstrate strong reliability in practice.

We also examine the feasible rate, the proportion of experiment runs in which the learned policy respects the budget constraint. IPCW-IPS achieves the highest feasible rate of 0.99, and IPCW-DR follows closely at 0.95. Because the IPCW correction provides more accurate survival estimates, the learner can track the budget boundary more precisely and maintain consistent constraint satisfaction within the Lagrangian optimization framework. Overall, these results show that our censoring-aware OPL framework not only improves survival-based outcomes but also stabilizes the learning dynamics even with challenging constraints. This combination of predictive accuracy, robustness, and reliability positions our approach as a practical and powerful tool for real-world decision-making regarding survival outcomes.

6 Concluding Remarks

This paper proposes a novel framework for Off-Policy Evaluation (OPE) and Off-Policy Learning (OPL) with survival outcomes subject to right censoring. Specifically, we introduce two novel estimators, IPCW-IPS and IPCW-DR, which incorporate the statistical technique of Inverse Probability of Censoring Weighting to correct for systematic bias induced by censoring. Theoretically, we establish the unbiasedness of our proposed estimators and prove the doubly robust property and variance reduction effects inherent in IPCW-DR. Extensive simulation studies demonstrate that, while conventional OPE methods suffer from severe underestimation bias, our estimators maintain high estimation accuracy across a wide range of experiment conditions. Furthermore, real-world experiments using the Starbucks dataset validate that our proposed estimators are robust against complex real-world data structures and censoring mechanisms, and substantially outperform existing baselines in terms of both OPE and OPL.