ADVISER: AI-Driven Vaccination Intervention Optimiser for Increasing Vaccine Uptake in Nigeria

1 Introduction

The state of maternal and infant health in Nigeria is appalling. The estimated maternal mortality rate in Nigeria is about 814

∗Corresponding Author

per 100,000 live births; in comparison, Poland and Italy's maternal mortality rate is 2 deaths per 100,000 live births. In fact, Nigeria alone accounts for more than 10% of maternal deaths globally, while only accounting for 2.6% of the world's population [World Health Organization, 2014]. Infant deaths in the country are also shockingly high-Nigeria loses 2300 children under five years of age daily [Okwuwa and Adejo, 2020]. The sustainable development goals (SDG 1 and SDG 3) aim to mobilize resources to the developing world to address inequity due to poverty and end preventable deaths of infants completely [United Nations, 2020]. However, we are far from achieving these goals.

In collaboration with HelpMum, a large non-profit organization based in Nigeria, we identify three significant challenges contributing to high mortality rates among mothers and infants. First, with an immunization rate of 13% for children between 12-23 months, Nigeria has the lowest vaccination rate in Africa. While vaccination is available for free in Nigeria, lack of awareness about the importance of vaccination is one of the major concerns for the low uptake of vaccination. Second, HelpMum identified that a primary driver for mothers not taking their children for vaccination is the high transportation cost relative to their income; we found that 46% of families analyzed as a part of this study earned less than $25 per month. Third, although several organizations, such as HelpMum, strive to design interventions for at-risk mothers and children, there is a gross imbalance between resource availability and demand for healthcare services.

HelpMum works closely with several local and state governments in Nigeria. Even though it uses a vaccination tracking system to remind mothers of upcoming vaccination, it has proven to be ineffective in practice. As part of this project, four new interventions were designed to increase vaccination uptake in Nigeria. HelpMum (including its advisory board) and domain experts guided the design of each intervention. The interventions (described later in the paper) are geared towards increasing the awareness of vaccination, reminding mothers about upcoming vaccinations for their children, providing accessibility to vaccination centers by operating a pick-up and drop-off service, and conducting a door-todoor vaccination delivery program. However, matching the interventions to specific individuals presents a challengethere exist far too many eligible recipients for the interventions as compared to the available resources. Moreover, the interventions do not necessarily guarantee successful vaccination. For example, HelpMum has observed that, at times, parents fail to bring their children for immunization despite repeated phone calls about upcoming vaccination schedules. These domain-specific challenges require that the allocation of limited resources is optimized under uncertainty of the outcomes.

We present a principled framework for optimizing the allocation of heterogeneous health interventions under uncertainty. Our approach, named ADVISER: AI Driven Vaccination Intervention OptimiSER, can guide non-profit organizations and government agencies to increase vaccination uptake in resource-constrained geographies, that are crucial to achieving key goals outlines in SDG 1 and SDG 3 of the United Nations [United Nations, 2020]. Our approach is based on formulating an integer linear program (ILP) to maximize the cumulative probability of success of heterogeneous interventions under uncertainty. However, in our setting, it is infeasible to generate or solve the ILP directly. In this paper, we tackle the challenges in a principled manner through the following contributions: 1) We present a formulation for optimizing the allocation of heterogeneous health resources under uncertainty. 2) We present a heuristic approach to prune the decision space of the ILP by leveraging the structure of the problem. We also present a theoretical bound on the objective value attainable by the heuristic with respect to the optimal solution to the ILP. 3) We show how guided local search can be used to generate promising vehicle routes based on the probability of specific individuals requiring the pickup service for vaccination. 4) We estimate the success of interventions through historical data and community surveys. 5) Experimental results using real-world data demonstrate that the proposed approach significantly outperforms baseline approaches. HelpMum is currently developing a pilot plan to deploy ADVISER in the largest city in West Africa. To the best of our knowledge, our solution would be the first AI-enabled program for increasing vaccination uptake in Nigeria.

2 RelatedWork

We present a condensed literature review here; a more extensive description of prior work is provided in the appendix (see section A.1) 1 . The general problem of resource allocation has been well studied in literature in the context of health interventions [Bishop et al., 2021; Mukhopadhyay et al., 2017]. Augmenting traditional vaccination campaigns by increasing awareness and optimizing vaccine supplies has shown increased uptake in South Africa [Bishop et al., 2021]. Dynamically adapting the vaccine delivery under limited resources and an evolving epidemiological situation has also been explored in the context of the COVID-19 pandemic [Matrajt et al., 2021]. The optimization of healthcare resources can either be done in a single-shot manner or by considering the sequential nature of the decision-making problem, depending on the specific use-case. Single-shot optimization problems have been used for the optimization of patient admissions in hospitals [Hulshof et al., 2013], allocating home healthcare services [Aiane et al., 2015], and redistribution of patients among hospitals in case of a surge in demand [Parker et al., 2020]. In principle, our problem setting and formulation is most similar to the work done by Aiane et al. [Aiane et al., 2015]; however, their setting only optimizes for travel times of resources (e.g., healthcare providers) and is not scalable (the number of decision variables and constraints in our problem is 106 times higher). Prior work has also explored performing sequential decision-making in the context of resource allocation in healthcare settings [Mate et al., 2021; Nishtala et al., 2021]. The key difference of such approaches to our setting is that we focus on a single-shot optimization problem within a fixed time window (which can be repeated to multiple time windows). The sequential nature of the problem is irrelevant in our setting-each individual in our case is only provided with a single intervention during the time period considered.

3 ProblemFormulation

ProblemSetting. Our problemsetting involves resource allocation to M individuals (e.g., mothers) over T days. We use [M] and [T] as shorthand for [1, . . . ,M] and [1, . . . , T] respectively. While our goal is to ensure that children get vaccinated, mothers typically take their children for vaccination in our geographic area of interest. As a result, we say that the interventions are designed for mothers. We assume that each mother is eligible for an intervention for a fixed number of contiguous days within these T days (depending on the last date when her child was vaccinated). The binary variable amt denotes whether mother m ∈ [M] is eligible at time t ∈ [T ]; amt = 1 if and only if the mother is eligible on day t, and is 0 otherwise. In order to get vaccinated, mothers can either travel to designated health centers, or healthcare officials can visit a mother's house. We divide the region of interest into a grid G consisting of equally sized cells. Each mother's residence and each health center therefore map to unique cells in G. We use dmg to denote the distance of mother m's residence from cell g ∈ G.

1 https://ayanmukhopadhyay.github.io/files/adviserAppendix.pdf

Interventions. In collaborationwithHelpMumand domain experts, we design four new interventions: 1) Phone call: A phone call is made to the mother reminding her about upcoming vaccination. We denote this intervention by ic. 2) Travel Voucher: A travel voucher is provided to the mother to commute to vaccination centers. We denote this intervention by it. 3) Bus Pickup: A bus can pick up a mother (and her child) from her residence and drop them at a vaccination center. Each bus has a capacity of γℓ (for ease of exposition, we assume that γℓ denotes the number of mothers that a bus can accommodate with their children). We denote this intervention by iℓ. F denotes the set of buses. 4) Vaccine Drive: A health worker goes to a designated locality and vaccinates mothers (children) living nearby who are eligible for vaccination. Naturally, there is a cap on the number of vaccinations a health worker can provide in a day. We denote this cap by γv, and denote this intervention by iv.

We use I to denote the set of interventions, and for notational convenience, add no-intervention/empty-intervention, denoted as in, to this set. HelpMum considers iv to be highly effective in practice, followed by ib, it and ic (in decreasing order of effectiveness). Each intervention has a cost associated with it; we use ej to denote the cost associated with intervention ij ∈ I. Naturally en = 0 (the cost of no-intervention is 0), and eℓ > ev ≫ et > ec. In particular, employing a bus pickup or the cost of conducting a vaccine drive is relatively much more expensive than giving a travel voucher or making a phone call to a single mother.

Outcomes. Let pmj be the probability of mother m taking her child for vaccination given intervention ij ∈ I.

Decision Variables. Given grid G, mothers M, and a time horizon of [T] days, we optimize over the allocation of interventions I. We use g and t to denote an arbitrary cell in G and an arbitrary day in [T] respectively. Let xtg be a binary variable that denotes the decision to conduct a vaccine drive, i.e., xtg is 1 if and only if there is a vaccine drive at cell g on day t, and 0 otherwise. We point out that a vaccination drive at a cell does not necessarily target every mother in that cell. A healthcare official can only visit a fixed number of households, and our optimization formulation must optimize which mothers to target during a drive. If possible, the healthcare worker will travel to nearby cells as well.

Let Rf denote the set of routes that a bus f ∈ F can operate (we explain constraints specific to routes later; we first present our optimization formulation here for ease of exposition). We use a binary variable qtfr to denote the routes that are chosen for operation, i.e., qtfr = 1 if bus f operates on route r ∈ Rf on day t ∈ [T]. Note that a specific route can only potentially target a subset of the mothers based on their locations. We use binary values smtfr to denote whether mother m can be picked up by a bus f ∈ F operating on route r ∈ Rf on day t ∈ [T ].

We use additional u, y, and z variables to match specific interventions to each mother. The variable ymtj = 1 if mother m is given intervention ij ∈ {in, ic, it} at time t. For interventions iℓ and iv, we have variables u and z such that: a) umtfr = 1 if mother m is picked up by bus f employing route r ∈ Rf on day t ∈ [T ], and b) zmtg = 1 if mother m is targeted on day t by a vaccination drive conducted at cell g.

Objective Function. Formally, we seek to optimize the following objective:

We seek to maximize the cumulative probability of successful vaccination given a fixed overall budget b by finding the optimal allocation of interventions I among the M mothers.

Constraints. We need to enforce the following constraints given our problem setting:

1. Eligibility Constraints: Each mother must be eligible for the vaccine when she is being targeted for an intervention.

2. Vaccine Drive Constraints: a) If a mother is being targeted for a vaccination drive at a given location and time, there must exist such a drive, b) only mothers that live within distance σ of a drive can be targeted for the drive, and c) at most γv mothers can be targeted by a single drive. The last two constraints denote operational limitations of conducting door-to-door vaccination drives.

3. Route Constraints: If a mother is being being picked by a bus on a route on a particular day, then a) the mother should be eligible to be picked on that route, b) the bus must employ that route on that day, and c) each bus can pick up at most γℓ mothers, and d) in addition, given current resource limitations of our partner agency, we consider that a bus can only operate a single route on a given day.

Note that each vehicle route must obey general routing constraints, e.g., there are restrictions on the earliest pick-up times and the latest drop-off times in our setting. We assume that all routes in Rf , ∀f ∈ F obey these constraints (for now) to simplify the discussion (discussed in detail in section 3.1). 4. Intervention constraint: We consider that each mother can be targeted for at most one intervention, i.e. for all m ∈ [M],

5. Budget Constraint: The total cost of the interventions can not exceed the monetary budget b of the organization.

3.1 Routing Formulation

We formulate a vehicle routing problem with time windows (VRPTW) [Toth and Vigo, 2002] to schedule vehicles to pick up mothers (and their children) and take them to a vaccination center. In practice, the mothers need to be taken to the health centers and dropped back to their resp. residences. However, we only discuss routing to the health centers to simplify the discussion. All vehicles begin operation from fixed spots (parking locations rented by HelpMum) called depots. Note that on day t, only a subset of mothers are eligible for vaccination, i.e., amt = 1. Let βe(m) and βl(m) denote the earliest and latest times on which mother m can be picked up. The times vary across the population based on occupation and other beneficiary specific constraints. Let the set of vaccination centers operating on day t day be St. HelpMum requires that mothers are dropped off at a vaccination center early so that there is sufficient time for them to get their children vaccinated. Let the earliest and latest drop-off times for a vaccination centre s ∈ St be denoted βe(s) and βl(s) respectively. The set of pick-up locations (mothers' residences) and drop-off locations (vaccination centers) represent the nodes (N) of a graph with the road network being the edges.

A route plan is denoted by an ordered sequence of nodes θ = {n1, n2, ...}, where nj ∈ N is an arbitrary node that the vehicle needs to visit en-route. We attach the following information with each node in a route plan. First, βθ,e(nk) and βθ,l(nk) is the earliest and latest the vehicle can arrive at location nk, and is set to βe(m) and βθ,l(nk) respectively, when the node corresponds to a pickup location for a mother m ∈ M, or βe(s) and βl(s) respectively when the node corresponds to a vaccination site s ∈ St. Second, the scheduled arrival time for the vehicle servicing route plan θ to location nk is δθ(nk). A route plan is feasible if all time window constraints are satisfied and all mothers who are picked up (up to a maximum capacity of each vehicle) are dropped off at a vaccination site. The time window constraints are satisfied for each location if βθ,e(nk) ≤ δθ(nk) ≤ βθ,l(nk), ∀nk ∈ θ. For each vehicle f ∈ F, the set of feasible routes contain all routes that obey all the routing and capacity constraints.

4 Approach

We face three specific challenges in directly solving optimization problem (1). First, the ILP consists of more than 109 decision variables and constraints. Second, we must generate the set of feasible routes as an input to the ILP. In our problem setting, the number of routes exceed 10200 . Third, we must estimate the success of each intervention for each mother. We tackle these challenges in a principled manner below. Our approach (shown in Figure 1 (b)) is based on pruning the search space of the decision variables by greedily conducting the most efficient intervention; specifically, we greedily use the given budget to conduct the intervention that has the highest success-to-cost ratio. In this case, conducting a vaccination drive is relatively more expensive than making a phone call, but it enables HelpMum to target more mothers and guarantee more successful vaccinations. After the greedy allocation of the vaccination drives we use guided local search to generate promising vehicle routes, which is fed to the ILP as an input.

Greedy Pruning. We assume that the probabilities pmj are known for each mother and intervention. We describe how such parameters can be estimated later. We use an iterative approach for pruning the size of the ILP. At each step, a set of mothers are chosen for intervention and removed from consideration. Let [Mw] denote the set of mothers at the beginning of iteration w. Let Hw denote a matrix of size G× T, where each entry in the matrix (denoted by Hw gt ) captures the utility of conducting a vaccination drive at cell g ∈ [G] and time t ∈ [T]. At iteration w, let g∗w on day t∗w denote the optimal cell and day to conduct an intervention, given [Mw]. The matrix Hw is used to choose cell-time combinations to conduct vaccination drives in each iteration. The cell-time positions chosen in the previous iterations are updated to -1 in the matrix to remove them from consideration in future iterations, i.e., g∗w′ , t∗w′ for all w′ < w is set to -1. We point out that conducting a vaccination drive at a cell can potentially target mothers from nearby cells as well, depending on the number of households in consideration and the manner in which door-to-door vaccine delivery is done. Let Mgt denote a subset of mothers in [Mw] who live within some exogenously specified distance σ of cell g and are eligible for vaccination at time t. Hw is then computed such that Hw gt = -1 if g = g∗w′ ∧ t = t∗w′ ∀w′ < w and U w (g, t) otherwise, where U w (g, t) = maxS⊆Mgt,|S|≤γv ∑m∈S (pmv - pmn) denotes the utility of conducting a vaccine drive at cell g at time t on iteration w by targeting the subset of mothers who provide the most gain over no interventions. Note that the utility depends on w, because mothers who are targeted for intervention in an iteration are removed from consideration on the next iteration. As an example, conducting a vaccination drive at the same location on two successive days won't be ideal, if every mother eligible for vaccination is already benefited by the service on the first day. Then, g∗w , t∗w = argmaxg,t Hw g,t . In our formulation, pmv = 1. We denote the mothers targeted as part of the conducting a drive at g∗w , t∗w as Sw g∗ t∗ = arg maxS⊆Mg∗t∗ ,|S|≤γv ∑m∈S (pmv - pmn).

We drop references to w to simplify the discussion. In the proposed heuristic, we decide to conduct a vaccine drive at g∗ , t∗ if et · |Sg∗ t∗| ≥ ev, i.e., if the cost of conducting a vaccine drive is at most the cost of giving travel vouchers to the mothers being benefited by the drive. Note that conducting a vaccine drive is a better intervention than providing travel vouchers even when they cost the same (as children are guaranteed to be vaccinated through the former strategy). However, our pruning strategy is lazy; we commit to conducting a drive only if the aforementioned condition is satisfied and leave other decisions for the pruned ILP. The mothers who are mapped to vaccine drives are removed from consideration at the next iteration and the budget is updated accordingly. We stop pruning after we are left with some exogenously specified budget parameter (b′ ). The complete algorithm is presented in the appendix (see section A.6).

Performance Bounds. Let k denote the number of vaccination drives determined in the greedy pruning phase, and MVH denote the set of mothers targeted by these k vaccination drives. We bound the loss incurred through greedy pruning relative to the optimal solution of the ILP. We assume that the optimal solution has at least k vaccination drives of size at least ev/et; we verify this empirically in multiple parameter settings. We arbitrarily choose k vaccine drives from such an optimal solution, and denote by MV I the set of mothers targeted by these k vaccine drives. Also, let mother m ∈ M be given intervention i∗m by such an optimal solution. We begin by proving the following proposition, which is an outcome of our greedy choice made at every iteration during pruning.

Proposition 1. ∑m∈MV H (pmv - pmn) ≥ ∑m∈MV I (pmv - pmn) (proof in Appendix A.2).

Let MV H \MV I (resp. MV I \MV H) denote the set of mothers in MVH (resp. MV I) but not in MV I (resp. MVH). We use Proposition 1 to prove the following theorem.

Theorem 1. Let OH be the objective value of the solution derived from our heuristic procedure and O∗ be the objective value of the optimal ILP solution. Then OH ≥ O∗ - (∑m∈MV H\MV I (pmi∗m - pmn)) (proof in Appendix A.3).

The lower bound on the objective value of the heuristic approach in Theorem 1 depends on the interventions provided in the optimal solution to mothers in MV H but not in MV I .

Route Generation. In principle, we could generate all the feasible routes given the routing constraints, which can then be provided as an input to the optimization problem. However, route generation is intractable in our setting. The total number of routes in our setting exceeds 10200 . As a result, we focus on generating a smaller subset of promising routes. Recall that our overall goal is to maximize the cumulative probability of successful vaccination; as a result, it is imperative that given the limited number of vehicles, we pickup mothers that need the ride the most. We capture this idea to define the utility of a route plan. Let p(θ, nk) = pmℓ - pmn denote the utility of an arbitrary pick-up node nk ∈ N in the routing graph (recall that each pickup node corresponds to a unique mother). The quantity pmℓ -pmn captures the importance of providing a bus pickup to mother m over giving no intervention. Given the utility function, we use guided local search [Kilby et al., 1999] to generate a subset of routes that maximize the utility (see section A.9 in the appendix).

Parameter Estimation. Note that optimization problem (1) requires estimates of the probability of success of each intervention for each mother. However, estimating the probabilities presents a challenge-the interventions - conducting vaccine drives, operating vehicle routes, and providing travel vouchers, are designed as part of this research; as a result, we lack exact historical data about the interventions. We only have data about phone calls that HelpMum made to all mothers. We compute the probability of success of untested interventions (i.e., vaccine drives, bus pickups, and travel vouchers) through a community survey that HelpMum performed. We estimate the probability of successful vaccination through phone calls and the effect of no interventions by learning a regression model on historical data. For brevity, we describe the estimation procedure in the appendix (see section A.5).

5 Experiments

Data. We collect anonymous information from HelpMum for 500 mothers registered as part of a vaccination tracking system operated by HelpMum. Each data point consists of several features such as the income level of the family, whether the mother received a reminder about the upcoming vaccination appointment, whether she took her child for vaccination, and the age of her child, among others. HelpMum obtained consent from each beneficiary for anonymous data sharing. We present a detailed description of each feature in the appendix (see A.4). We also collect the geographic locations of all 32 vaccination centers in our area of interest. A map with the vaccination centers and the rented parking locations is presented in the appendix (see section A.4).

HelpMum plans to deploy the ADVISER framework to all mothers in their vaccination tracking system (about 40000 mothers). As a result, we use the available data to generate two synthetic datasets (D1 and D2), consisting of 40000 mothers each. We generate features of mothers in D1 and D2 by sampling each feature independently and uniformly at random from the original data of 500 mothers. For the mothers in D1, we compute the probabilities of success for each mother given an intervention as follows: for each m ∈ [M], we choose pmn uniformly at random from (0, 1); followed by pmc uniformly at random from (pmn, 1); pmt uniformly at random from (pmc, 1); and pmℓ uniformly at random from (pmt, 1). We set pmv = 1 based on community feedback (see section A.4). D1 essentially captures the domain knowledge that we have about the interventions, specifically, for a mother m, pmn ≤ pmc ≤ pmt ≤ pmℓ ≤ pmv = 1. For D2, we estimate the probability of vaccination given no intervention (and vaccination given phone calls) by training a logistic regression model on the original data (details in Appendix A.4). The probabilities for remaining interventions, pmt, pmℓ, and pmv are chosen in a similar manner as in D1.

Baseline Algorithms. While prior work does not consist of approaches that optimize the allocation of heterogeneous health resources under uncertainty, we consider the following baselines: 1) Real-world Baseline (RWB) For the first baseline, we asked HelpMum to allocate the interventions solely based on domain expertise. HelpMum identified key areas of importance to conduct vaccination drives in each local government (similar to administrative jurisdictions), operate fixed routes to each vaccination center, distribute travel vouchers to mothers living more than 10 kms from a vaccination center, and call mothers by prioritizing the age of the child (the younger the child, the higher the priority). 2) Hierarchical ILP (HILP) Motivated by the use of hierarchical planning to create tractable approaches for resource allocation [Zhang et al., 2016; Pettet et al., 2021], we design a baseline that solves optimization problem (1) in a hierarchical manner. First, we leverage the geographic density of the beneficiaries to identify clusters (using k-means [MacQueen and others, 1967]). The overall budget is distributed across clusters in proportion to the number of mothers in each cluster. A separate ILP is then solved directly for each cluster. We describe the exact parameters of both the approaches and results from the clustering algorithm in the appendix (see section A.7).

Experiment Setup. In consultation with HelpMum, we set the costs as follows: ec = $0.1, et = $1.1, ev = $15, and eℓ = $20. We optimize the allocation of resources for T = 30 days and γv = 100. We vary the overall budget b between $7000 to $8400, and use b - $1000 as a threshold for the greedy pruning procedure. Our implementation is available online 2 . All experiments were run on a Linux machine with 64GB RAM and an 8-core AMD processor. We implement ADVISER using the Python programming language and solve the ILP using Google OR Tools with SCIP solver.

Results. We show the expected number of successful vaccinations in D1 and D2 in Figure 2 (a) and Figure 2 (b) respectively. We observe that the expected vaccine uptake achieved via ADVISER is more than 39970 for all the budgets considered in the experiment, whereas the average vaccine intake achieved by the baseline algorithm is at most 26000 (we performed the simulation on 40000 mothers). The average number of mothers who received intervention through ADVISER is 39672, in comparison to 28426 and 20588 through HILP and RWB respectively. We also observe in Figure 2 (c) the distribution of the interventions; as expected, both the ILP-based approaches capitalize on solutions with more vaccination drives. However, we point out the importance of the other interventions as well. In practice, the number of vaccination drives is bound by the number of available healthcare workers for the service, whose regular job is to work at healthcare centers. We observed that when the number of vaccination drives is restricted to 400 per month, the average number of mothers who are targeted for pickups more than triples as compared to Figure 2 (c) (result presented in appendix A.8). Moreover, HelpMum seeks to utilize ADVISER to improve antenatal care for pregnant mothers as well, which will lower the realization of γv (number of children who can be vaccinated during a drive). Additional empirical results, such as the distribution of each intervention for each budget, outputs of the clustering approach, and running time of the approaches are provided in Appendix A.8.

2 See https://github.com/ayanmukhopadhyay/ADVISER IJCAI

Deployment. HelpMum is currently planning a pilot program in the largest city of Nigeria. We show an initial version of the tool based on the ADVISER framework in Figure 3.

6 Conclusion

In collaboration with HelpMum, a non-profit organization in Nigeria, we present ADVISER: AI Driven Vaccination Intervention OptimiSER. Our framework can accelerate our progress towards goals in SDG 1 and SDG 3 by increasing access to healthcare services and vaccination, and by reducing maternal and infant mortality in resource-constrained settings.

Ethical Statement

Our goal in this project is to improve vaccination uptake in Nigeria. The specific interventions were designed by HelpMum in collaboration with domain experts and its advisory board. HelpMum is currently planning to pilot the ADVISER framework in collaboration with the local governments. The implications of the solution quality of any AI-driven framework for public intervention need to be studied carefully. While a large part of the interventions suggested by ADVISER are targeted towards low-income individuals (by construction), the objective function of our framework can be modified to add a score function that measures other parameters, e.g., fairness of an allocation. Also, it is possible to add arbitrary constraints on such parameters. However, the very nature and form of such constraints needs to be determined in collaboration with local stakeholders and governments.

Acknowledgements

We would like to acknowledge funding from the Google AI for Social Good Grant, Google AI for Social Good "Impact Scholars" program, National Science Foundation grant CNS1952011, HelpMum, and Vanderbilt University. We would like to thank the staff at HelpMum for helping conduct community surveys and perform data collection.