The Good, the Bad, and the Explainer: A Tool for Contrastive Explanations of Text Classifiers

1 Introduction and Contribution

Consider a text classifier ψ1, retrained with new data and resulting into ψ2. The underlying learning function of the newly trained model might lead to outcomes considered as contradictory by the end users when compared with the previous ones, as the system does not motivate why the logic is changed. Hence, such a user might wonder "why do the criteria used by ψ1 result in class c, but ψ2 does not classify on c anymore?". This is posed as a T-contrast question, namely, "Why does object A have property P at time ti, but property Q at time tj?" [Miller, 2019; Van Bouwel and Weber, 2002].

Contribution. ContrXT (Contrastive eXplainer for Text classifier) is a tool that implements the approach we proposed in [Malandri et al., 2022a]. ContrXT computes modelagnostic global T-contrast explanations from any black box text classifiers. ContrXT, as a novelty, (i) encodes the differences in the classification criteria over multiple training phases through symbolic reasoning, and (ii) estimates to what extent the retrained model is congruent with the past. ContrXT is available as an off-the-shelf Python tool on Github, a pip package, and as a service through RESTAPI. [Malandri et al., 2022c]

To date, there is no work that the authors are aware of that computes T-contrast explanation globally, as clarified by the most recent state-of-the-art surveys on XAI for supervised ML (see [Burkart and Huber, 2021; Mueller et al., 2019]).

∗Contact Author. fabio.mercorio@unimib.it - mercorio.com

2 ContrXT

ContrXT aims at explaining how a classifier changes its predictions through time. We describe the five building blocks composing ContrXT, as in Fig.1: (A) the two text classifiers, (B) their post-hoc interpretation using global, rule-based surrogate models, (C) the Trace step, (D) the eXplain step and, finally, (E) the generation of the final explanations through indicators and Natural Language Explanations (NLE).

(A) Text classifiers. ContrXT takes as input two text classifiers ψ1,2 on the same target class set C, and the corresponding training datasets D1,2. As clarified in [Sebastiani, 2002], classifying Di under C consists of |C| independent problems of classifying each d ∈ Di under a class ci for i = 1, . . . , |C|. Hence, a classifier for ci is a function ψ : D × C → B approximating an unknown target function ˙ψ.

Output: Two black-box classifiers on the same class set.

(B) Post-hoc interpretation. Following the study about ML post-hoc explanation methods of [Burkart and Huber, 2021], one of the approaches consists in explaining a black box model globally by approximating it to a suitable interpretable model (i.e., the surrogate) solving the following:

where I represents a set of possible white box models to be chosen as surrogates, and S is the fidelity of the surrogate pg, that measures how well it fits the predictions of the black box model ψ. In addition to [Burkart and Huber, 2021], ContrXT adds Ω(pg) ≤ Γ as a constraint to Eq. 1 to keep the surrogate simple enough to be understandable while maximising the fidelity score. The constraint measures the complexity of the model whilst Γ is a bounding parameter. 1 In the global case, the surrogate model pg approximates ψ over the whole training set X taken from D which is representative of the distribution of the predictions of ψ.

Output: Two white-box, rule based surrogates p1,2 of ψ1,2

1 In case the surrogate is a decision tree, Ω(pg) might be the number of leaf nodes whilst it could be the number of non-zero coefficients in case of a logistic regression.

(C) Trace. This step aims at tracing the logic of the models p1,2 while working on a datasets D1,2. It generates the classifiers' patterns through a global interpretable predictor (i.e., the surrogate), then it is encoded into the corresponding Binary Decision Diagram (BDD) [Bryant, 1986]. A BDD is a rooted, directed acyclic graph with one or two terminal nodes of out-degree zero, labelled 0 or 1. BDDs are usually reduced to canonical form, which means that given an identical ordering of input variables, equivalent Boolean functions will always reduce to the same BDD. Reduced ordered BDDs allow ContrXT to (i) compute compact representations of Boolean expressions, (ii) apply efficient algorithms for performing all kinds of logical operations, and (iii) guarantee that for any function f : Bn → B there is one BDD representing it, testing whether it is true or false in constant time.

Output: two BDDs b1,2 representing the logic of pg1,2.

1/1 (D) eXplain. This step takes as input the BDDs b1,2, that formalises the logic of the surrogates pg1,2, and computes the BDDs encoding the differences between the two. Step D manipulates the BDDs generated from the Trace step to explain how ψ1 and ψ2 differ (i) quantitatively by calculating the distance metric defined below (aka, Indicators), and (ii) qualitatively by generating the BDDs of the added/deleted patterns over multiple datasets Dti . As this is the key idea of ContrXT, we formalise the following.

Definition 2.1 (T-contrast explanations through BDDs)

Given f1 : Bn → B and f2 : Bm → B we define:

The goal of the operator � (�) is to obtain a boolean formula that is true iff a variables assignment that satisfies (falsifies) f1 is falsified (satisfied) in f2 given f1 (f2). Let b1 and b2 be two BDDs generated from f1 and f2 respectively, we synthesise the following BDDs:

bb1,b2 � = b1 � b2 (4) bb1,b2 � = b1 � b2 (5) where b� (b�) is the BDD that encodes the reduced ordered classification paths that are falsified (satisfied) by b1 and satisfied (falsified) by b2. We also denote as

• var(b) the variables of b;
• sat(bb1,b2 � ) all the true (satisfied) paths of bb1,b2 � removing var(b1) \ var(b2);
• sat(bb1,b2 � ) all the true (satisfied) paths of bb1,b2 � removing var(b2) \ var(b1).

Both bb1,b2 � and bb1,b2 � encode the differences in the logic used by b1 and b2 in terms of feature presence (i.e., classification paths). Indeed, bb1,b2 � (bb1,b2 � ) can be queried to answer a T-contrast question like "Why does a path on b1 had a true (false) value, but now it is false (true) in b2?". Clearly, features discarded (added) by b2 are removed from paths of bb1,b2 � (bb1,b2 � ) as they are used by ψ1.

Output: Two BDDs bb1,b2 � and bb1,b2 � encoding the rules used by b2 but not by b1 and vice-versa.

(E) Generation of final explanations. Starting from bb1,b2 � and bb1,b2 � , the final explanations are provided through a set of indicators and Natural Language Explanations.

Indicators estimate the differences between the classification paths of the two BDDs through the Add and Del values (see Eq. 6 and 7). To compare add and del across classes, we compute the Add Global (Del Global) as the number of paths to true in b� (b� ) over the corresponding maximum among all the bc� (bc� ) with c ∈ C. In the case of a multiclass classifier, as for 20newsgroup, ContrXT suggests focusing on classes that changed more with respect the indicators distribution.

Natural Language Explanations (NLE) exhibits the added/deleted paths derived from b� and b� to final users through natural language. ContrXT uses the last four steps of six NLG tasks described by [Gatt and Krahmer, 2018], responsible for microplanning and realisation. In our case, the structured output of BDDs obviates the necessity of document planning which is covered by the first two steps. The explanation is composed of two main parts, corresponding to Add and Del paths. Content of each part is generated by parsing the BDDs, extracting features, aggregating them using Frequent Itemsets technique [Rajaraman and Ullman, 2011] to reduce the redundancy, inserting the related parts in the predefined sentences [Rosenthal et al., 2016].

3 Results on a Benchmark Dataset

Evaluation. ContrXT was evaluated in terms of approximation quality to the input model to be explained (i.e., the fidelity of the surrogate) on 20newsgroups, a well-established benchmark used in [Jin et al., 2016] to build a reproducible text classifier, and in [Ribeiro et al., 2016], to evaluate LIME's effectiveness in providing local explanations. We ran ContrXT over different classifiers, trained through the most used algorithms, such as linear regression (LR), random forest (RF), support vector machines with RBF (SVM), Naive Bayes (NB), Bidirectional Gated Recurrent Unit (biGRU) [Cho et al., 2014], and BERT [Devlin et al., 2019] (bert-base-uncased) with a sequence classification layer on top. Results are shown in Table 1. We considered and evaluated all the global surrogate models surveyed by [Burkart and Huber, 2021], represeting the state of the art. Approaches falling outside the goal of ContrXT (e.g., SP-LIME [Ribeiro et al., 2016] and k-LIME [Hall et al., 2017] whose outcome is limited to the feature importance values) and papers that did not provide the code were discarded.

To date, ContrXT relies on decision trees to build the surrogate, though it can employ any surrogate algorithms.

Results Comment for 20newsgroup. One might inspect how the classification changes from ψ1 to ψ2 for each class, i.e., which are the paths leading to class c at time t1 (before) that lead to other classes at time t2 (now) (added paths) and those who lead to c at t2 that were leading to other classes at time t1 (deleted paths). Focusing on the class atheism of Fig. 2 the number of deleted paths is higher than the added ones. Fig. 3 reveals that the presence of the word bill leads the ψ2 to assign the label atheism whilst the presence of such a feature was not a criterion for ψ1. Conversely, ψ1 used the feature keith to assign the label, whilst ψ2 discarded this rule. Actually, both terms refer to the name of the posts' authors.

The example of Fig. 3 sheds light on the goal of ContrXT, which is providing to the final user a way to investigate why ψ2 classified documents to a different class with respect to ψ1, as well as monitoring future changes. NLE allows the user to discover that -though the accuracy of ψ1 and ψ2 is high - the underlying learning functions (i) learned terms that should have been discarded during the preprocessing, (ii) ψ2 persists in relying on those terms, which are changed after retraining (using bill instead of keith), and (iii) having political atheist is no longer enough to classify in the class.

Evaluation through Human Subjects. We designed a study to assess if - and to what extent - final users can understand and describe what differs in the classifiers' behaviour by looking at NLE outputs. We recruited 15 participants from prolific.co [Palan and Schitter, 2018] that were asked to look at NLE textual explanations and to select one (or more) statements according to the meaning they catch from NLEs. Results showed that the participants understood the NLE format and answered with an 89% accuracy on average, and an F1score of 87%. Finally, we computed Krippendorff's alpha coefficient to estimate the extent of agreement among users. We reached a value of 0.7, which [Krippendorff, 2004] considers as acceptable to positively assess the subjects consensus.

Getting ContrXT. ContrXT can be used either as a pip Python package [Malandri et al., 2022b] or as a service through REST API. The API is written using Python and the Flask library [Grinberg, 2018] and can be invoked using a few lines code shown in Listing 1. A load testing has been performed using locust.io to measure the quality of service of the ContrXT's API, adding a virtual user every 10 sec. Our architecture reached a throughput of 2.55 users per second. Beyond this value, the API service keeps working, putting additional requests into a queue.

Demo Video. Available at https://tinyurl.com/ContrXTIJCAI

1 impor t r eque s t s , i o 2 from z i p f i l e impor t Z i pF i l e 3 f i l e s = {' t ime 1 ' : open ( t 1 c s v pa t h , ' rb ' ) , ' t ime 2 ' : open ( t 2 c s v pa t h , ' rb ' )} 4 r = r e qu e s t s . pos t ( ' [ see d e t a i l s on g i t hub repo ] ' , f i l e s = f i l e s ) 5 r e s u l t = Z i pF i l e ( i o . BytesIO ( r . con t en t ) )

4 Conclusion, Limitations and FutureWork

We demonstrated ContrXT [Malandri et al., 2022a] a novel model-agnostic tool to globally explain how a black box text classifier change its learning criteria with regard to the past (T-contrast) by manipulating BDDs. The evaluations have been performed on a multiclass benchmark, i.e., 20newsgroup. Our evaluation using different learning algorithms revealed ContrXT can work with the state-of-the-art learning algorithms, reaching an F1-weighted surrogate fidelity up to 0.87 and never below 0.73. The Spearman correlation test revealed the accuracy is not correlated with the ADD/DEL indicators, confirming they provide additional insights beyond the quality of the trained models. ContrXT is available as a Python package on Github. To date, ContrXT is bounded to explain text classifiers. We are working to extend ContrXT in dealing with tabular classifiers.