Self Monitoring, Goal Driven Autonomy Agents

1 Research Problem

As intelligent systems become a regular part of our everyday lives, robust and safe operation is ever more important. My research focus is to endow agents with the ability to monitor themselves in order to detect when their behavior has exceeded their boundaries. Previously, we have explored different forms of expectations for anomaly detection in agents operating in Real-Time Strategy (RTS) games, as well as dynamic domains involving planning and execution. My current work aims to achieve agents that can reason about and use expectations in both dynamic and partially observable domains, as well as investigating meta-cognitive expectations for detecting anomalies in the agent's own cognitive processes (reasoning, planning, etc) instead of anomalies in the world.

2 Goal Driven Autonomy

Goal-driven autonomy (GDA) is an agency model where an agent revises its goals by reasoning over discrepancies. Discrepancies arise when the agent's own expectations do not match the agent's observations. Such discrepancies arise when acting in dynamic environments (i.e., changes occur independently from the agent's actions). When discrepancies occur, the GDA agent will suggest alternative goals. An example, adapted from [Molineaux et al., 2010], involves an agent performing navy operations. A naval convoy is in route to deliver some equipment and along the way an escort vessel identifies an unknown contact. At this point the agent could pursue one of multiple alternative goals including (1) abort the mission and route back the vessels to the departing port or (2) hold the convoy and send escort vessels to identify the contact.

Figure 1 shows a GDA situated agent. The core GDA process is shown within the controller. After discrepancy detection identifies an anomaly, the discrepancy d is sent to the explanation generator which hypothesizes one or more explanations e. These explanations are then used by the goal formulator which may decide to formulate new goals g. Finally the goal manager selects which goals for the agent to pursue. An underlying motivation for GDA is that in the face of an anomaly, it may be better for an agent to change its goal(s) instead of replanning.

player0 is the agent). This kind of expectation is possible with the use of rules, such as a description logic rule: "A player controls a region if there is at least 1 unit in the region and that player owns every unit in the region." (Partitioning a map into polygon shaped regions is possible with the API Brood War Terrain Analyzer [Perkins, 2010]). The discrepancy detector can identify if this expectation was met by adding the expectation statement (player X controls region Y) to an ontology of facts of the current game state, running it through a reasoner, and checking consistency. Upon an inconsistency the reasoner provides an explanation trace of what facts are conflicting. In this trace would be the DL rule which would have as its consequent the fact that player X controls region Y. The trace also shows which antecedents of the rule are inconsistent, which enables useful explanation.

We have also explored evidence scoring functions to determine strategies in individual battle scenarios in RTS games [Dannenhauer and Mu˜noz-Avila, 2013b].

4 HTN Based Expectations

In more recent work we have examined new techniques for expectations generated from Hierarchical Task Network (HTN) planning domain knowledge for agents operating in dynamic domains [Dannenhauer and Mu˜noz-Avila, 2015a]. We introduced Informed expectations: expectations that are the accumulated effects from the planning actions executed thus far. This approach filters out the atoms in the state that are irrelevant to the agents current goal, while still maintaining effects that are needed to reach the goal. In that work, we assumed that enough of the state is known to enable generating concrete grounded plans (i.e. a complete plan is generated for the agent's goals).

We are interested in exploring domains where perception is limited enough such that it requires the agent to explore its environment to determine what resources it has available before being able to achieve its goals. Generating a grounded plan from the beginning would be impossible, because the agent would not know where it needs to go or what objects it will encounter to use to accomplish its goals. For example, suppose an agent is operating in a domain similar to the Marsworld domain described in [Dannenhauer and Mu˜noz-Avila, 2015a].

The overall goal of the agent is to create a signal. The agent must either activate a specified number of beacons, drop and light a certain number of flares, or create a specified number of smokefires using piles of wood found nearby. If the agent has limited perception, then only after it explores the state will it know if it is better to activate beacons or go about dropping flares. Thus classical grounded planning is not possible. In these environments agents would likely have models for sensing actions and associated costs. We aim to explore sensing vs. acting as a form of self-monitoring. Additionally, we are interested in dynamic domains that have significant changes over time, requiring the agent to adapt in order to maintain performance.

5 Metacognitive Expectations

Anomalous or failed behavior of an agent may result from a problem in a cognitive process instead of a world state. To detect and correct such a problem relies on metacognitive capabilities. Previously we have explored using a cognitive trace that records data from different cognitive processes (e.g. preception, interpretation, goal selection, planning) and show preliminary results that enable an agent to swap out it's planning faculty for an alternative planner [Cox, 2016]. We would like to explore metacognition in other cognitive processes including but not limited to: goal selection, memory, and perception.

Acknowledgements: This work was supported in part by grants NSF 1217888 and ONR N00014-15-1-2080