Taking Situation-Based Privacy Decisions: Privacy Assistants Working with Humans

1 Introduction

We are surrounded with Internet of Things (IoT) applications that enable various devices to work together. These devices operate by collecting, storing, and processing many different types of user data. The main medium for handling privacy constraints with these devices is to agree on their privacy policies, where a user is consulted to give consent, as with the policies in General Data Protection Regulations (GDPR) [Voigt and Von dem Bussche, 2017]. While giving consent enables to reflect users' preferences to some extent, it has also been shown that these approaches create an incredible decision load on the user, usually leading to users to ignore the details to be able to use the systems [Utz et al., 2019]. The problem is amplified for IoT applications. The number of devices and the way they would want to use data creates an explosion in the number of situations that might arise, requiring humans to be constantly probed for consent. Previous work has shown that privacy assistants can learn the privacy preferences of the users [Squicciarini et al., 2017;

Kurtan and Yolum, 2021] and take actions to inform and help them appropriately [Das et al., 2018; Ulusoy and Yolum, 2021]. Recent work has shown that users are welcoming to use privacy assistants beyond this; e.g., when privacy assistants take some of the decisions on behalf of the users [Colnago et al., 2020]. Here, we propose a new model for privacy assistants that can make privacy decisions on behalf of the users per situation, even when the user has not explicitly been in a similar situation before. We identify three major challenges that need to be addressed to realize this:

Context. An alternative formulation of privacy is contextual integrity [Nissenbaum, 2004], where the privacy is understood to be preserved if appropriate information flows take place as designated by contexts. Inspired by this, we investigate various definitions of context [Alegre et al., 2016] and follow the widely accepted definition of Dey [2001]: "any information that can be used to characterize the situation of an entity". In terms of privacy in IoT, various types of information have been identified as important. For example, it has been established that users tend to give access to sharing data if they know it will be used for a short duration but are uncomfortable to share data when they do not know how long it will be kept [Leon et al., 2013]. In a similar vein, knowing the purpose of data collection or the benefit it will bring affects the privacy decisions of users. The location in which the data collection is taking place is sometimes important; e.g., restroom has been identified to be a blacklist concept [Naeini et al., 2017]; while other locations, such as store or library do not have a particular connotation in user studies. Consider the following scenario [Naeini et al., 2017] that expresses a situation that combines information about a user as well as an IoT device.

Example 1. You are at a department store. This store has presence sensors to detect whether someone is present. The store management uses this data to keep track of when there are few customers in the shop to determine whether they can reduce the number of staff at these times. You are not told how long the data will be kept.

When making a decision about Example 1, the contexts that would influence the decision can be the limited nature of personal data being collected (e.g., only presence), the purpose of collection, as well as retention of data. Thus, multiple contexts have to be formulated. Given such situations, how can a privacy assistant formulate the context(s) of a situation in hand? Is it better to consider a dominant, single context or factor in all contexts that a situation belongs to?

Decision. When contexts are decided, the privacy assistant needs to make a decision to share information or not. Making a privacy decision in a situation requires factoring in whether the particular set of IoT devices involved are indeed trustworthy [Chung et al., 2017]. Ideally, with increasing number of interactions, a privacy assistant would learn whom to trust and to what extent. Many successful trust models exist in the literature to address this [Teacy et al., 2012]. However, in an IoT system, a user will interact with the same device rarely and will have to decide on whether to trust without prior interactions with that device [K¨okciyan and Yolum, 2020]. How can a privacy assistant make sharing decisions based on trust in contexts, rather than trust in an entity?

Collaboration. Privacy is personalized; so are the privacy assistants. Some users are happy to allow data collection in Example 1, while some are not, requiring the privacy assistant to take personalized sharing decisions. Moreover, it is even possible for humans to make conflicting privacy decisions on occasions that are similar [Acquisti and Grossklags, 2005]. These require privacy assistant to assess whether its decision could be faulty and if so delegate the decision back to the user. However, delegating too much to the user will make the use of assistants pointless. How can a privacy assistant decide when to delegate to the user? Would different types of users benefit from different delegation frequencies from their privacy assistants?

We propose a situation-based privacy model to realize agent-based privacy assistants (PAS). PAS derives contexts automatically from the situations that the user has previously been in. Using subjective logic, PAS determines the trust for a context, based on positive and negative privacy experiences of the user in that context. This handles the problem of not having many interactions with the same device. By explicitly modeling inconsistencies between experiences of the user, PAS determines when to make a decision on behalf of the user, and when it can abstain and delegate the decision to the user. We implement the proposed agent and experimentally evaluate its workings over a case study that uses an anonymized IoT dataset [Naeini et al., 2017]. We show that PAS can capture policies that pertain to multiple privacy contexts and combine them to reach a sharing decision as well as personalize it for different users.

2 Situation-Based PrivacyModel

Typical approaches to trust would make use of the experiences with an individual to create a model. The more experiences there are (direct or indirect), the more accurate the model [Teacy et al., 2012]. However, in an IoT setting, an agent would have few experiences with the same device, but overall many experiences with different devices. Thus, modeling the trust in a device based on the experiences of that device would create an insufficient model.

One possible way to study this question is through categories by identifying the device as belonging to a particular group and assigning trust based on that. Recent work has demonstrated that in certain domains assigning trust based on categories and indirect experiences have been successful [Sapienza and Falcone, 2020]. However, in IoT applications, collecting indirect experience is also difficult and would require interactions with unknown entities.

We propose to estimate trust by only considering the context that the interaction will take place in. This has the advantage that even when there is no specific prior evidence about a device, a decision can be made by considering the context. For example, in Example 1, the user might have never entered this store before. Hence, it is not possible to model trust in the sensor based on previous interactions. However, the user might have been exposed to situations where she did not know how long her data will be stored. If the user can formulate this as a context, it can use it to decide whether to share or not share information with the sensor.

2.1 Identifying Contexts

A typical way of thinking of contexts is to pre-define them at design time, so that the agent can categorize the policies based on their contextual properties [K¨okciyan and Yolum, 2017; Fogues et al., 2017]. However, the set of contexts would depend on the domains that the agent engages in, the granularity of experiences, and the variance between them. For some settings a context that can be associated with medical situations would be sufficient to capture interactions, whereas sometimes there would be a difference between intensive care unit and check-up situations, as they pertain to different information. This creates a need to derive contexts dynamically.

In our model, each privacy situation consists of a set of features that describe the IoT device (e.g., its type) as well as the dealings (e.g., purpose of interaction) (Definition 1).

Definition 1 (Privacy Situation). P x y = ⟨text, F⟩ denotes a situation of x concerning y, where x is of type PAS and y is an IoT device, text is a textual representation of a situation, F denotes a set of features derived from text. P x is the set of all situations of x, and pi refers to the ith item in P x .

The privacy situation in Example 1 as perceived by x concerning presence sensor ps can be represented as P x ps = ⟨t, {location = store, data = presence, device = ps, retention = unspecified, purpose = reducestaff}⟩, where t is the privacy text used in the example.

PAS interacts with IoT devices all the time, collects various privacy situations, and groups similar situations together to derive relevant contexts (Definition 2).

Definition 2 (Context). A context is a collection of similar privacy situations based on a similarity metric (Method 1). Given a context c and two situations pi and pj, if pi ∈ c and sim(pi,pj) holds then pj ∈ c also holds, where c ∈ C x and {pi, pj} ⊆ P x . C x is the set of all contexts known by x.

Method 1 (Situation Grouping). PAS implements a clustering technique to take the set of privacy situations and a similarity metric sim for generating the set of contexts.

2.2 Assigning Trust to Contexts

PAS assigns trust not to individuals but to specific contexts. To do this, PAS processes a new situation by first deciding on which contexts it belongs to, and then by considering how much it trusts these contexts. Our model uses subjective logic (SL) [Jøsang et al., 2006] to associate trust to contexts. SL is a belief calculus to capture trust among individuals as opinions. In Definition 3, we describe an opinion (wx c ) as the trust of PAS (i.e., x) in context c.

Definition 3 (Opinion). wx c = ⟨b, d, u, α⟩ denotes an opinion in context c, with the degrees of belief (b), disbelief (d), uncertainty (u) and a base-rate parameter (α); where b+d+u = 1 and b, d, u, α ∈ [0, 1].

PAS needs to construct experience bases first before it computes context-based opinions to make a sharing decision. Each interaction with a device results in the user allowing data collection or not, which we call an experience (Definition 4). In Example 1, if x declines data collection request, ⟨P x ps ,-⟩ would represent this negative experience.

Definition 4 (Experience). exc,i = ⟨P x y ,v⟩ denotes ith experience in context c, where P x y is a privacy situation, v shows if the interaction with y was perceived as positive (+) or negative (-). Ex c denotes all experiences of x in context c.

In IoT applications, each PAS values privacy differently and each experience is unique to the agent. We focus on aggregating the experiences of PAS with IoT devices that are deployed in the same context (Definition 5). Figure 1(a) depicts the grouping of situations and construction of contextbased experience bases.

Definition 5 (Context-based Experience Base). EBxc = ⟨r, s⟩ denotes a context-based experience base, where r and s are the numbers of positive and negative experiences in context c; i.e., c ∈ C x , r = ∑|Ex c | i=1 exc,i .+ and s = ∑|Ex c | i=1 exc,i .-. EBx denotes the context-based experience bases of x.

Given a context-based experience base, we derive opinions of agents about contexts using Wang and Singh's formulation, which supports that fact that uncertainty does not always decrease with more data and that uncertainty remains high when the numbers of positive and negative experiences are close to each other [Wang and Singh, 2007]. Definitions 6 and 7 show the mapping from an experience base to opinions.

Definition 7 (Mapping). Given EBxc = ⟨r, s⟩, the mapping to an opinion wx c = ⟨b, d, u, α⟩ is defined as: b = β×cert(r, s), d = (1 - β)×cert(r, s), u = 1 - cert(r, s); where α is the base-rate and β = (r + 1)/(r + s + 2).

2.3 Multi-context Decision-making

PAS should be able to make a decision for unseen privacy situations. First, PAS will compute which contexts an unseen situation belongs to (Method 2).

Method 2 (Situation Categorization). PAS trains a multilabel classifier to take a privacy situation and the set of contexts for assigning a privacy situation to every context with a varying degree between 0 and 1. The sum of all degrees is 1.

When PAS determines the opinions about multiple contexts (Definition 3), it then needs to reach a decision (i.e., allow data collection or not). PAS handles interactions in two categories. Category 1 is the set of interactions where PAS chooses not to make a decision on behalf of the user because its knowledge about the situation is uncertain. This corresponds to a case where PAS identifies one dominant context (i.e., a context with a degree higher than 0.5) for a privacy situation, but the level of conflict in that context is high (Definition 8). Hence, PAS delegates the decision to the user.

Definition 8 (Conflict). Given EBxc = ⟨r, s⟩, b, d, u values can be computed according to Definitions 6 and 7. The conflict is then computed as |b - d|, and is evaluated against a threshold ψ (a value between 0 and 1). If |b - d| < ψ holds, there is a high conflict in experience in a given context c.

Category 2 includes all other cases, where PAS makes a decision on behalf of the user. An opinion's probability expectation value can be computed according to Equation 1 (adapted

Algorithm 1: decide(p, ψ, θ, γ)

Data: EBx , set of context-based experience bases Data: C x , set of contexts 1 dec ← 0; i ← 0;wProb ← 0; cat1 ←False 2 confSet ← categorizeSituation(p,C x ) 3 domC ← getDominantContext(confSet) 4 if domC then 5 b, d, u ← computeBDU(Ex domC ) 6 if |b - d| < ψ then 7 cat1 ← T rue 8 if cat1 then 9 dec ← askUser(p) 10 else 11 foreach c ∈ C x do // p is of type cat2 12 b, d, u ← computeBDU(Ex c ) 13 α = |0.6 - d| 14 prob ← getProb(b, u, α) 15 wP rob ← wP rob + confSet[i] ∗ prob 16 i ← i + 1 17 if wProb > θ then 18 dec ← 1 19 i ← 0 20 foreach c ∈ C x do 21 if confSet[i] > (max(confSet) - γ) then 22 EBxc ← updateExpBase(Ex c , dec) 23 i ← i + 1 24 return dec

from [Jøsang et al., 2006]). A base-rate parameter α is used to define the contribution of the uncertainty to the probability expectation value. If PAS interacts with a context few times (i.e., uncertainty is high), α ensures that an initial trust is assigned for the new context.

Since PAS considers multiple contexts in making a decision, it computes a context-based weighted probability expectation value (P ) according to Equation 2. The weights are the degrees associated with each of the contexts. Note that the degrees are normalized values, dc denotes the degree of belonging to context c.

PAS then compares this value (P) against a threshold θ (a value between 0 and 1) to make a final decision while dealing with a situation of type Category 2. The sharing decision gets a value of 1, when P is above θ (i.e., PAS allows data collection); and 0 otherwise. Higher θ values will result in less data sharing by PAS. After each decision, PAS updates its context-based experience bases according to a defined confidence threshold γ (e.g., Algorithm 1). Each interaction with an IoT device is treated as a positive experience if the data collection is allowed and negative otherwise. For each relevant context, r and s values are incremented accordingly.

3 The Formal Model in Action

The decision-making process of PAS is depicted in Figure 1(b) and realized with Algorithm 1. The algorithm takes the situation p to be categorized, the conflict ratio ψ, the decision threshold θ and the confidence threshold γ as input. The decision dec, the counter i, the combined probability expectation value wP rob and the category information cat1 are initialized first (line 1). In line 2, PAS uses categorizeSituation function (Method 2) to assign the situation p to all the contexts with varying confidence scores denoted as confSet. The nth value in confSet gives the confidence score for p to belong to context n. PAS first checks if there is a dominant context given confSet (line 3), if so it checks the level of conflict in the dominant context regarding the specified conflict ratio ψ. In the case of a high-conflict (Definition 8), the situation p is of type Category 1 (lines 4-7). If PAS deals with a situation of type Category 1, the user will be asked to decide to allow data collection or not (line 9). Otherwise, PAS will make an automated decision based on previous contextual experiences (lines 11-18). For each context, PAS computes belief, disbelief and uncertainty values for the specific context-based experience base (Definitions 6 and 7). α is set to |0.6-d| per context, which ensures that when the disbelief is low, uncertainty contributes more to the computed probability expectation value (line 15) (Equation 1). The probability value is updated for each context so that if PAS is more confident about a context assignment, the corresponding context probability will influence the resulting decision more. If wProb is above the decision threshold θ, the decision dec is set to 1 to allow data sharing. Then, PAS updates the experience base for each context with acceptable confidence scores according to the decision. The confidence interval is set to (max(confSet) - γ), which defines the distance to the maximum confidence score for a context (lines 20-23). Finally, the algorithm returns the privacy decision (line 24).

4 AReal-World Case Study

To apply the model in real life, an IoT stack should be in place [Chow, 2017] to enable communications. We focus on the application layer and study the workings of the model using an anonymized dataset [Naeini et al., 2017], which has been collected through surveys with users of IoT devices. The dataset includes 380 privacy scenarios that are split into 40 surveys, where each survey includes 14 scenarios. The participants are asked questions about particular scenarios (e.g., if the user would allow data collection) and their privacy concerns. Each survey includes responses from 20-25 participants. A scenario contains a textual description of features such as data type being collected, the purpose of data collection, the location where the data collection is happening and how long the data will be kept (e.g., Example 1). In our experimental setup, we keep one survey apart to conduct a qualitative and quantitative analysis (Section 4.2). Our supplementary material demonstrates how our approach works on two different examples for different participants as well as additional information on the participants' data. This material together with our code base is available online 1 .

1 https://git.ecdf.ed.ac.uk/nkokciya/pas-privacy

4.1 Situation Clustering, Context Classification

We use 366 scenarios from remaining surveys to: (i) generate contexts using clustering techniques, (ii) train a multi-label classifier to infer multiple contexts for unseen privacy scenarios. We use well-known Python libraries such as NLTK, Gensim and scikit-learn to implement our approach.

We train a Doc2Vec model to represent each scenario as a numeric vector by providing the normalized scenarios [Le and Mikolov, 2014]; such a model helps PAS to capture the semantic similarity of words in the text-based scenarios. The Elbow method suggests grouping scenarios into four clusters. In our setting, PAS uses a hierarchical clustering algorithm to discover clusters by using ward linkage criteria, where the similarity function sim is set to euclidean (Method 1). Hence, the algorithm minimizes the sum of squared differences within the clusters while merging or splitting clusters. The four clusters identified are C0(140), C1(27), C2(39), C3(160); where the number of instances belonging to that particular cluster is shown in parentheses.

Most scenarios fall into C0 and C3, where the former is broadly focused on non-personal data (e.g., temperature sensing in a room) and the latter unspecified data collection. C1 spans situations where sensitive data (e.g., face) are used for identification and C2 covers situations where the users are told how they will benefit from data collection. Each cluster corresponds to a context in our setting, where the context does not only capture location, or a topical setting but situations that differ from each other. This satisfies the points raised in Section 1. For example, library as a location is part of each context that we identify; showing that being in a library by itself is not significant for making privacy decisions.

To classify unseen situations into multiple contexts, we train a multi-label classifier. We have tried several classification models (SVM models with linear/rbf kernel, logistic regression models and so on), applied 5-fold cross-validation for model selection, and chose the model performing the best on average. In this case, this is the SVMmodel with a linear kernel (Method 2). The macro-averaged precision, recall and f1-score values on the test set are 0.91, 0.88, 0.9 respectively with an average accuracy of 0.86.

4.2 Experimental Results

We apply PAS for 25 participants to make a privacy decision on the scenarios specific to one survey. Each participant has only 14 scenarios labeled with privacy decisions, far less than a realistic setting. For this reason, for this case study, we populated the experience base of each participant by using similar participants' experiences, where the similar participants were chosen from the remaining 39 surveys. Two participants are considered similar if the difference between their average Internet Users' Information Privacy Concerns (IUIPC) scores is less than 1. This information was already present in the dataset. Note that in real life only the experiences of the user will be used. Since there is no dataset with this information, we are resorting to using similar users' data as well, while knowing that this will create approximation and lower accuracy in our results.

We categorize the participants into three categories based on their actual sharing behavior using the sharing decisions [Baarslag et al., 2017]: Behavioral Unconcerned (Group 1-g1) if they share more than 66% of the scenarios, Behavioral Pragmatists (Group 2-g2) if they share between 33%- 66% of the scenarios, and Behavioral Fundamentalists (Group 3-g3) if they share less than 33% of the scenarios. The dataset consists of 5 users (20%) in g1, 16 users (64%) in g2 and 4 users (16%) in g3. Such a categorization is useful to understand how to set and update model parameters automatically as we discuss later.

Table 1 shows results from our experiments with different conflict thresholds (0.1, 0.2, 0.3, 0.4) with a fixed set of 250 experiences. We include results for one participant from g1 (p1), two participants from g2 (p4 and p3) and one participant from g3 (p6). We also include average values for each of the groups. For each run, we report results for three agents: PAS-S that does the computation based on single context (i.e., the context that has the highest confidence score) with no human intervention, PAS-M that does the computation based on multiple contexts with no human intervention (i.e., Algorithm 1 with ψ = 0), and PAS (Section 2). In Table 1, the number in the brackets is the number of times PAS delegates the decision to the user. We report accuracy values for all agents. The accuracy value for PAS is the value computed when the user cooperates with PAS to make sharing decisions. For example, in g2, when the conflict ratio is set to 0.4; the users could see an average accuracy value of 0.74

by just making decisions for 4.3 scenarios in average. Hence, PAS would handle the remaining scenarios. Our group-based analysis could be summarized in three major themes.

Context. Deciding based on multiple contexts as opposed to a single context helps the agent to achieve better results. That is, PAS-M almost always performs either equally or better than PAS-S. For g1, they perform equally. For g2, for all values of conflict ratio ψ, PAS-M slightly outperforms PAS-S. For g3, the difference is more pronounced; e.g., when ψ is 0.4, PAS-M achieves 0.83, whereas PAS-S can only achieve 0.6. Note that these values do not factor in cases when the agent detects uncertainty and consults the user.

Collaboration. For any conflict ratio, when PAS consults its user, PAS outperforms the other agents (PAS-S and PAS-M) that run with no human intervention. For the previous case, when the agent consults the user for the ambiguous cases the accuracy increases to 0.89. Regardless of the group, when the conflict ratio increases, PAS accuracy values also increase. This shows that the instances that PAS consults the user for are mostly the ones it is making a mistake in. Thus, PAS identifies ambiguous cases correctly and by delegating those to the user enables PAS to reach a higher accuracy.

Personalization. We observe that different groups require different values for threshold θ and conflict ratio ψ for their PAS. g1 has a tendency to share data more than other groups. This suggests that θ value should be set to a value less than 0.5. In other words, keeping θ high as in Table 1 requires keeping the conflict ratio high for better PAS accuracy values. For g3, PAS performs well even for low conflict ratios; hence, human intervention could be minimized for such users by choosing low ψ values. For g2, θ and ψ should be adjusted according to where the user stands (i.e., closer to g1 or g3). Thus, by observing its user's sharing pattern and adapting its parameters accordingly, PAS can help its user sufficiently while achieving a good accuracy.

5 Discussion and Conclusion

Most of the existing work on trust are based on the idea to build a model per agent over many interactions. Teacy et al. [2012] attack the problem by devising an algorithm that benefits from capturing hierarchical Bayesian modeling. Such an approach is successful in building trust models, but require large number of interactions and thus not immediately applicable in IoT settings. Liu and Datta [2012] develop a context aware dynamic trust, where they use a Hidden Markov Model to predict a service provider's next move. They consider interactions that are similar in certain features, rather than all interactions. Our intuition to capture context rather than previous interactions is similar but we estimate trust in devices that might never have been seen.

Burnett et al. [2013] study trust in short-lived interactions, similar to a setting as we have here. They propose a model of stereotypes, which enables agents to generalize their experiences with others over observable features. Fang et al. [2018] generalize this idea using a fuzzy semantic process and machine learning methods. In privacy, the context in which an agent resides has a tremendous effect, even when the agents exhibit similar observable properties; e.g., a user can trust a camera in a hospital but not in a bar. Thus, rather than stereotyping, our approach associates trust for each context.

Various techniques to enable agents to classify whether a content in question is private or not using various supervised machine learning algorithms or information retrieval techniques exist [Squicciarini et al., 2017; Kurtan and Yolum, 2021]. These approaches either leverage big data sets or interactions with others to make privacy decisions. However, in IoT settings these are not available; hence, our focus has been on situations where there is little data and others' privacy opinions are not accessible.

Daidone et al. [2021] develop a blockchain-based framework to check privacy compliance of devices. They assume both user preferences and device policies are available in a structured form. Baarslag et al. [2017] design a negotiation strategy, where an agent makes partial offers based on utility-based heuristics to manage app permissions. Contrary to these, here we assume that the user preferences are derived based on the trust from contexts and situations are not provided in structured form but in natural language. K¨okciyan and Yolum [2020] propose a multi-agent model, where an agent collects information from other agents to evaluate the trustworthiness of IoT devices before revealing its user's data. Differently here PAS does not keep individual trust values for each device but instead compute opinions about contexts.

Ajmeri et al. [2020] design norm-aware agents to make ethically appropriate decisions in social contexts. Mosca and Such [2021] propose an agent model to support multiuser privacy in online social networks. Our focus here has been on decision making with limited prior knowledge in new situations. The algorithm developed here is orthogonal to these works such that it would be interesting to incorporate personal values in conjunction with contexts.

We propose a novel agent-based privacy assistant PAS to handle interactions with IoT devices. PAS makes a sharing decision on behalf of the user, or it delegates the decision to the user, by modeling trust in multiple contexts. We show the applicability of our approach on an IoT case study. This work opens up interesting directions for research. Semantic relations between contexts could signal certain order to process contexts when making a decision. Obtaining more elaborate feedback from the user; e.g., "prefer not-share because the purpose is not specified." would enable PAS to assess a given situation in more depth than the textual representation alone.

Acknowledgments

This research was supported by the UKRI Strategic Priorities Fund via the REPHRAIN Research Centre, https://www.rephrain.ac.uk/; and by the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research, https://hybridintelligence-centre.nl, grant number 024.004.022.