Forecasting the Number of Tenants At-Risk of Formal Eviction: A Machine Learning Approach to Inform Public Policy

1 Introduction

Eviction is an urgent societal issue, which severely affects the lives of low-income individuals in the U.S. from multiple perspectives. In particular, it puts evicted families into material hardship [Desmond and Kimbro, 2015] and could increase the risk of various health issues (such as depression and parental stress) and reduce their prospects of future decent housing [Desmond, 2012; Eviction Lab, 2018; Desmond and Kimbro, 2015; Himmelstein and Desmond, 2021]. Furthermore, it could intensify various types of social problems such as poverty and housing inequality [Gromis, 2019; Eviction Lab, 2018]. Therefore, tackling the eviction crisis plays a critical role in improving the lives of this vulnerable population, and helps make a progress on the SDG #1 "No Poverty" and SDG #11 "Sustainable Cities and Communities" laid out by United Nations 1 to create a better world by 2030.

To mitigate the eviction crisis, several eviction prevention/diversion programs (such as the Emergency Rental Assistance Program 2 ) have been designed and implemented in the field. In particular, the federal government has allocated various financial resources (such as cash assistance, vouchers, etc.) to help households who have difficulty paying their rents. In spite of their availability nationwide, there is a large variability in the use of those resources; i.e., while the allocated resources have been used completely in some regions, parts of the allocated resources have been returned to the federal government from some other regions. This observation suggests a need for a more efficient resource allocation strategy, which in turn, requires more accurate forecasts of the future number of tenants at-risk of eviction in target regions. Thus, any attempt to improve the accuracy of the forecasted number of tenants at-risk of eviction could have substantial impacts on the effectiveness of existing policies to disperse resources.

To this end, this paper leverages recent advances in the ML domain [Yao et al., 2018; Tabar et al., 2021] to propose an ML-based solution to forecast the number of tenants atrisk of formal eviction in various census tracts 3 at a temporal resolution of one month. Our model, named as MARTIAN (Multi-view model forcAsting the numbeR of Tenants at-rIsk of formAl evictioN) leverages data sources of various spatial and temporal resolutions (namely, eviction filing records, American Community Survey (ACS) data 4 , and labor statistics) to forecast the total number of tenants that are at-risk of eviction in each census tract n months into the future. Then, we evaluate the predictive performance of MARTIAN under various conditions using a real-world dataset consisting of information about eviction cases filed across Dallas County, TX since 2019. The results of our experiments show that MAR- TIAN outperforms a wide variety of baseline models in all considered situations; in particular, it achieves 5% lower Root Mean Square Error (RMSE) than the best-performing baseline model (on average). Further, it achieves a Spearman of 0.685, which shows that the ranking of census tracts is preserved to high extent in MARTIAN's forecasts. Additionally, the results of our cross-region test show that MARTIAN's superior predictive performance is generalizable to unseen census tracts. This research has been conducted in collaboration with CPAL 5 , which is an NGO aiming at tackling povertyrelated issues across Dallas County, TX.

1 https://sdgs.un.org/goals

2 http://tiny.cc/3vhouz

3 A census tract is a sub-region of a county, and is defined by the

U.S. Census Bureau for taking surveys and representing its results.

4 https://www.census.gov/programs-surveys/acs/data.html

2 RelatedWork

As a pathway into various social problems (such as homelessness) [Bieretz et al., 2020; van Laere et al., 2009], the eviction crisis has drawn the attention of scholars from several disciplines. In particular, there has been extensive research in social science literature on understanding the risk factors 6 of eviction and its consequences. As a result, past literature found three key categories of risk factors: (1) individual-level factors such as the number of children, job loss, and drug use disorder [Desmond et al., 2013; Desmond and Gershenson, 2017; Montgomery et al., 2017; Stenberg et al., 2020]. (2) neighborhood-level factors such as crime rate, and eviction rate in a neighborhood [Desmond and Gershenson, 2017]. (3) network-level factors such as the number of disadvantaged people in a tenant's network [Desmond and Gershenson, 2017].

Additionally, there has been a growing body of knowledge on the consequences of eviction and its impacts on individuals' lives. For example, prior work found that eviction could result in various health issues such as parental stress and depression [Desmond and Kimbro, 2015; Hatch and Yun, 2021]. Furthermore, once getting evicted, tenants' credit rating gets debased, which in turn, put more distance between them and the public housing program, and could exacerbate the housing inequality in society [Greiner et al., 2012]. Although these empirical findings are informative and conducive to understanding the whole context of eviction, these studies do not focus on the problem of forecasting the number of tenants at risk of eviction. In contrast, this paper leverages the findings of prior work in social sciences as well as ML techniques to forecast the number of tenants at risk of eviction, which could assist the government and NGOs in proactively tackling the eviction crisis in a more efficient and effective manner.

In addition to the social science studies, there has been some research from the Artificial Intelligence (AI) community on mitigating the housing problems. For example, Ye et al. [2019] and Tan [2020] employed ML techniques to predict the risk of landlord-harassment and the eviction rate, respectively. However, these studies have some limitations: (1) the developed ML models forecast at the temporal resolution of one year, which limits their usability in our problem domain, where a forecasting tool with a higher temporal resolution (such as one month) is needed, or (2) they mainly relied on classical ML models and did not consider differences in the nature of various data sources in their design, e.g., timeseries data and static data are treated the same. To address these limitations, we build a deep learning-based model that leverages various data sources with different spatial and temporal resolutions to forecast the number of tenants at-risk of getting formally evicted at the monthly resolution. Further, we conduct extensive experiments under various conditions to assess the superiority of MARTIAN to a wide variety of baseline models.

5 https://childpovertyactionlab.org

6 Risk factors denote factors that are linked to the higher chance of a negative outcome.

3 A Problem Statement

This paper aims at building an ML model to precisely forecast the number tenants at-risk of formal eviction (i.e., the number of eviction filings) at each census tract n months into the future.

Assume that Ec t refers to the total eviction cases filed at census tract c in month t and Lct is a vector of length q representing the labor statistics at census tract c in month t (q refers to the total number of features in the labor statistics data). Also, suppose that ACSc t is a vector of length r representing the most recent values of ACS factors available at month t for census tract c (r refers to the total number of features selected from the ACS data). Note that as the U.S. Census Bureau releases the ACS data with a delay of about two years, at each point of time, ACSc t contains statistics of two years ago. Then, this paper aims at building a forecasting model M such that:

In this paper, the value of k is chosen through hyperparameter tuning, and a separate experiment has been conducted for different values of n.

4 Datasets

This work uses three data sources: (1) Eviction filing records, (2) Labor statistics, and (3) American Community Survey (ACS). We extract our input features using these data sources, which are then used by an ML model to compute the value of the target variable. Table 1 provides detailed information on the input features extracted from each data source. In the following paragraphs, we introduce each data source and explain why we incorporate them into the model.

(1) Eviction Filing Records. This dataset consists of detailed information about eviction cases filed in judicial courts across Dallas County, TX since 2019. We get access to this dataset via CPAL, which receives daily updates (except for holidays) on new eviction cases filed in Dallas County. Each eviction record contains detailed information, e.g., the plaintiff's name, the defendant's name and address (geographical coordinates), the filing date, etc. However, the court's final decisions regarding each case is not available in our dataset.

In this work, we use eviction filing data to compute the target variable and extract input features; in particular, we use the historical data on the number of eviction filings as input because overall eviction rate in a neighborhood is found to be associated with greater likelihood of individuals' eviction [Desmond and Gershenson, 2017].

(2) Labor Statistics. The U.S. Bureau of Labor Statistics releases monthly data on labor statistics 7 , which contains various pieces of information related to the economy of a region, e.g., the unemployment rate and number of employees in various non-farm industries (e.g., manufacturing and government). This data enables policymakers to monitor the economic/employment status over time, and to make appropriate policies accordingly. Given a strong association between work status and the risk of eviction [Desmond and Gershenson, 2017; Stenberg et al., 2020], we believe that this data would provide useful signals to MARTIAN regarding monthly work status. This data is mainly released for each metropolitan area (rather than each census tract) and we use the data of the "Dallas-Fort Worth-Arlington" area.

(3) American Community Survey. The U.S. Census Bureau releases the ACS data, which is basically an annual report on various demographic/housing characteristics of different regions across the U.S. In particular, for renterinhabited housing units (and their householders), it summarizes the value of the following metrics, which are found to have some associations with the risk (or number) of eviction and housing instability: work status [Desmond and Gershenson, 2017; Stenberg et al., 2020; Puckett et al., 2002], educational attainment [Stenberg et al., 2020; Bassuk et al., 1997], income level, and monthly housing cost per income [Eggers et al., 2010]. Accordingly, we utilize the 5-Year Experimental Estimates ACS data, which is available for each census tract in our study. Although the ACS data is not available at the monthly resolution, we think that it could still provide an insightful big picture of the situation in various census tracts.

Pre-processing. To pre-process our data, we take three main steps: (1) similar to prior work [Desmond et al., 2018], we remove eviction filing records with commercial defendants and duplicate records from the dataset of eviction filing records, (2) we compute the total number of eviction filings in each census tract (out of 529 census tracts within Dallas County, TX) per month, (3) we scale the data of each input feature and the target variable into the range of [0, 1] using the Min-Max normalization. Please note that the predictive performance of all models is calculated after transforming the data to the original range (the parameters of min-max normalization are computed using the training data).

7 https://www.bls.gov/eag/eag.tx.htm#eag tx.f.2

5 The ForecastingModel: MARTIAN

In this section, we explain our forecasting model. Leveraging recent advances in the ML domain [Yao et al., 2018; Tabar et al., 2021], we build a multi-view neural network to incorporate data sources of different spatial/temporal resolutions into the prediction process. Figure 1 represents the architecture of MARTIAN. As illustrated, it has three views, each of which extracts features from one of the three aforementioned data sources: (1) The first view employs a Long Short-Term Memory (LSTM) network [Hochreiter and Schmidhuber, 1997] followed by two fully-connected layers to learn patterns from the time-series data of eviction filings in the census tract of interest, (2) the second view extracts features from the time-series data of labor statistics using an LSTM network followed by two fully-connected layers, and (3) the third view employs a Multi-Layer Perceptron (MLP) to learn features from the factors selected from the ACS data. Then, the outputs of these three views are concatenated and given to the output layer to forecast the value of the target variable.

6 Empirical Validation

In this section, first, we explain our experimental set-up and baseline models. Then, we compare the predictive performance of MARTIAN with that of baseline models and conduct an ablation study. Finally, we conduct a cross-region test to evaluate its generalizability.

6.1 Set-Up

To have a trustworthy and robust evaluation of the predictive accuracy of different models, we employ the walkforward testing approach [Kaastra and Boyd, 1996]. The walk-forward testing approach is a variant of K-fold crossvalidation adopted for the time-series domain; i.e., in this testing approach, the input dataset is split into a sequence of time-shifted training, validation, and test sets (instead of randomly splitting the data) with a window length of w (we set w to three, which is equivalent to three months as the length of each time-step is one month). Then, for each performance metric, we compute and report the average performance over all test sets. In addition, to train neural network models, the batch size, loss function, and maximum number of epochs are set to 32, MSE, and 200, respectively. We also utilize an Adam optimizer [Kingma and Ba, 2014] with a learning rate of 2 × 10-4 and the early stopping approach [Prechelt, 1996] with a patience value of 10 epochs. Finally, as a result of hyper-parameter tuning, the value of k (i.e., the length of time-series inputs) is set to 6.

6.2 Baseline Models

We compare the predictive performance of MARTIAN with that of an extensive set of baselines. The first set of baselines consists of the following classical ML models: (1) Ridge regression [Hilt and Seegrist, 1977] (2) Support-Vector Machine (SVM) [Cortes and Vapnik, 1995], (3) XGBoost [Chen and Guestrin, 2016], (4) Random Forest [Breiman, 2001], and (5) LightGBM [Ke et al., 2017].

Additionally, we considered various deep learning-based models in our study as well. In particular, we conduct a performance comparison between MARTIAN and its building blocks, i.e., LSTM and MLP, to show the effectiveness of the multi-view architecture in this problem domain. We also compare its predictive performance with some strong deep learning models, namely TabNet [Arik and Pfister, 2021] and Gated Recurrent Unit (GRU) [Cho et al., 2014], which are shown to work well on the tabular data and time-series data, respectively. Please note that the input of time-series models at time-step t is a concatenation of Ec t ,Lct , and ACSc t . However, for the remaining models, the input is a concatenation of static features and all k steps of time-series inputs.

6.3 AComparison with Baseline Models

Table 2 compares MARTIAN with that of various baseline models for n ∈ {1, 2, 3}. We use two metrics to evaluate the predictive performance of forecasting models: (1) RMSE, which intuitively measures the average difference between each model's predictions and the actual number of eviction filings, and (2) Spearman correlation 8 , which intuitively shows the extent to which the forecasted values preserve the actual orders of census tracts in terms of the number of eviction filing values. In this table, one row is considered for each ML model of interest and each column corresponds to the value of a performance metric for a specific value of n. Also, the best performance is shown in bold and the last row (Gain) shows the percentage of improvement that MARTIAN achieves over the best-performing baseline model. According to the results, MARTIAN outperforms all baselines for all different values of n; in particular, on average, MARTIAN outperforms the best-performing baseline model by achieving 5.00% smaller RMSE and 0.88% higher Spearman, which shows its superiority against several strong ML models for this problem domain.

8 The value of Spearman ranges between -1 and 1. A higher Spearman shows a better performance of a forecasting model.

Additionally, MARTIAN achieves a Spearman value of 0.685 (on average), which shows that the ranking of census tracts in terms of the number of tenants at risk of formal eviction is preserved to high extent in MARTIAN's output.

Furthermore, MARTIAN outperforms both MLP and LSTM models, which form its building blocks; i.e., on average, it achieves 6.00% lower RMSE and 12.29% higher Spearman than MLP and improves the predictive performance of LSTM by 5.16% and 7.70% in terms of RMSE and Spearman, respectively. This could show the value of using multi-view architecture for incorporating data sources of various resolutions, rather than treating all inputs the same.

Moreover, comparing the performance of classical models, we see that decision-tree based models outperforms the other ones (i.e., SVM and Ridge) significantly; i.e., on average, decision-tree based models achieve 28.31% and 29.42% better RMSE and Spearman, respectively. This could show that, in case of any difficulty in using deep learning, decisiontree based ensemble models could be more appropriate ML choices for this task. Also, in spite of its high performance in several other domains, TabNet achieves the poorest performance among all our deep learning-based baselines and ensemble models. Therefore, this attention-based model does not seem to be an appropriate choice for this case.

6.4 Ablation Study

We also conduct an ablation study to assess the impact of each view on the MARTIAN's predictive performance. To this end, we remove one view each time, train the new model, and then, evaluate its performance. Table 3 represents the results of our ablation study for n ∈ {1, 2, 3}. According to the results, removing view1 (i.e., features extracted from the time-series data of eviction filings) leads to a significant decrease in the predictive performance of MARTIAN; i.e., it results in 32.34% increase in RMSE and 159.27% decrease in Spearman (on average). In particular, we observe that MARTIAN-w/o-View1 cannot preserve the rank of census tracts with respect to the number of eviction filings as it has a negative spearman value. This makes sense because the timeseries data of eviction filings is the only input data available at our forecasting spatial and temporal resolutions, and the other two data sources (i.e., labor statistics and ACS) are unavailable either at the census tract level or at the temporal resolution of one month. Therefore, two other data sources can only provide a big picture and alone are not enough for accurately forecasting the eviction crisis at high spatial and temporal resolutions.

Additionally, removing view2 (i.e., features extracted from the labor statistics data) results in 7.69% increase in RMSE

and 10.21% drop in Spearman (on average). Thus, as expected, incorporating the monthly status of employment helps enhance the predictive performance of MARTIAN, even though it is not available for each census tract and it only reports the work status for "Dallas-Fort Worth-Arlington". Furthermore, excluding view3 (i.e., features extracted from the ACS data) leads to 4.83% increase in RMSE and 5.83% decrease in Spearman (on average). Therefore, although the ACS data reports annual conditions of each census tract, its information on renter-inhabited housing units (and their householders) are still helpful for predicting the number of eviction filings at the census tract level for each month. In conclusion, as a result of this ablation study, we find that both labor statistics and ACS data are useful auxiliary input signals for our forecasting task.

6.5 Cross-Region Test

In all our previous experiments, we trained forecasting models on the training portion of the Dallas data, and then, evaluated their performance on the testing portion of the same data. We now conduct a cross-region test, in which the training and testing datasets are created from the data of two disjoint sets of census tracts. This helps us evaluate if MARTIAN's superior predictive performance is generalizable to unseen regions (whose data has not been seen by the model in the training phase). To this end, we take the following steps: (1) we create two disjoint sets of census tracts (with almost equal size) through random sampling such that the statistics (minimum, maximum, median, and average) of the total number of eviction filings for these two sets look similar, (2) we train the forecasting models on the training portion of the first set, and (3) we assess the performance of forecasting models on the testing portion of the second set. Please note that we still use the walk-forward testing approach and the time frame of training and test sets is the same as before.

Table 4 shows the results of our cross-region test. According to the results, MARTIAN outperforms all baseline models for different values of n; in particular, on average, it achieves 3.32% lower RMSE and 3.77% higher Spearman than the best-performing baseline model. This shows that MARTIAN's superior predictive performance is generalizable to various unseen regions.

7 Real-World Use Case

Our tool could serve as an AI assistant to (1) shed light on the number of tenants at risk of getting formally evicted in the future; e.g., the output of MARTIAN can be used to generate a heatmap of the forecasted number of tenants at-risk of eviction for each month in the future (similar to Figure 2), and (2) make a more well-informed resource allocation plan to mitigate evictions in a more efficient and effective manner. In particular, we contacted officials at Texas Housers (i.e., Texas Low Income Housing Information Service) 9 , which is an organization aiming at mitigating housing problems in Texas. Ben Martin, who is an official at Texas Housers and is working on the eviction and foreclosure data, stated that:

"Knowing where evictions are being filed helps advocates, administrators, elected officials, and legal aid to identify where they need to direct their efforts, funds, and other resources in order to keep renters housed"

In particular, he elaborated on the potential impacts of such forecasting tools in the real world as follows:

"The number of eviction cases filed or of a certain outcome, might, for example, be used as a baseline for setting program funding levels. If a somewhat accurate tool could be developed, it would be incredibly useful for advocacy with the legislators, elected officials, and agencies responsible for eviction court and eviction diversion"

Therefore, all these pieces of evidence show the value of an accurate eviction forecasting tool, and the extent to which it could help policymakers enhance the eviction diversion/prevention programs in the field. We are in discussion with CPAL and officials at Texas Housers in deploying MARTIAN among domain experts to make broader impacts.

8 Conclusion

In this paper, we developed a neural network model, named as MARTIAN, that leverages data sources of various resolutions and forecasts the number of tenants at-risk of getting formally evicted at the census tract level n months into the future. The results of our empirical evaluation show that MARTIAN outperforms various baseline models in terms of RMSE and Spearman in all considered situations. Additionally, the results of our cross-region test show that MARTIAN's superior predictive performance is generalizable to unseen census tracts. MARTIAN could help policymakers direct funding and other resources in a more efficient manner and enhance the existing eviction prevention/diversion programs by providing data-driven insights on the future condition of each census tract in terms of eviction filings. This research has been done in collaboration with CPAL, and the usability of MARTIAN in the field is under review.

9 https://texashousers.org

Acknowledgements

This work was in part supported by NSF awards #1940076 and #1934782 and the Bill and Melinda Gates Foundation Grant #141840.