A Goal-Directed Dialogue System for Assistance in Safety-Critical Application

1 Introduction

In an environment where a human and an assistive intelligent system cooperate to achieve predefined goals, the assistant system should be able to provide contextual help, either on request or proactively, to the human in the loop. The form in which the help is offered can be parameterised by the system designer and relies on the pre-defined encoding of the world model. The inference of the context for contextual assistance is normally carried out passively, by observing the environment and the human [Jamakatel et al., 2023; Honecker and Schulte, 2017]. However, in human-in-theloop systems, it is desirable that the context is not only passively inferred, but is actively confirmed, verified and completed by the human, so that the system is more transparent to the human. Engaging in a conversation with the human is one measure for active context inference.

Human conversations can be broadly classified into two categories: transactional conversations, in which a practical quantifiable goal is pursued and known to the interlocutors [Brown and Yule, 1983], and social conversations conducted to fulfil social purposes (e.g. chit-chat and small talk [Clark et al., 2019]). With recent advances, conversational agents powered by Large Language Models (LLMs) have shown remarkable conversational capabilities and have proven useful both in transactional and social conversations [Onorati et al., 2023; Guan et al., 2023]. In transactional conversations, LLMs have to reason over the world state to determine if the conversation goal has been reached. Although claims about the reasoning capability of LLM-powered conversational agents have been made [Kojima et al., 2022; Wei et al., 2022], these claims are disputed and LLMs have been shown to perform poorly in planning and sequential decision-making, a type of the well-studied reasoning tasks [Valmeekam et al., 2023; Kambhampati, 2024], making them unsuitable for goaloriented conversations. Furthermore, hallucinations and lack of transparency render LLM-powered conversational agents ill-equipped for applications requiring goal-oriented transactional conversations, especially in domains with smaller faulttolerance thresholds.

In safety-critical human-in-the-loop systems that should provide correct and sound contextual assistance (assistance that adapts to the context defined by variables representing the state of the world), accurate inference of the context is crucial. While this context can be inferred passively, supplementing the passively inferred context by actively engaging in a conversation with the human with contextual questions will ensure that the inferred context is also grounded on the world model perceived by the human. Furthermore, this allows the human to actively verify and correct the inferences carried out passively only through observation data.

In this paper, we introduce a goal-directed dialogue planning approach for active context determination in a safetycritical assistance system. We validate its usability in the UL domain 1 [Kiam and Jamakatel, 2023] as a running example. The domain knowledge is modelled using hierarchical task networks (HTNs). To infer the current scenario (e.g. landing due to a medical emergency, engine on fire), in which assistance is to be provided, besides relying on passively observed data on the human operator, the human be engaged actively in a conversation flow. For this, we propose the use of a Fully Observable Non-Deterministic (FOND)- planning-powered dialogue agent to actively infer the context required for instruction generation [Muise et al., 2019; De Venezia and Muise, 2023]. Although abundant flight control data is available for the UL domain, the decision-making and the dialogue subdomains are data-scarce. We utilize the generative capabilities of LLMs to augment the limited data and to train the Natural Language Understanding (NLU) system for the dialogue agent to increase the NLU capability. Thus, we combine symbolic AI (FOND planning and HTN planning) with aspects of neural AI for natural language comprehension and data augmentation, leveraging the corresponding strength of each approach.

1 The UL domain was used in the 2023 International Planning Competition for Hierarchical Planning and is available here: https://github.com/ipc2023-htn/ipc2023-domains/tree/main/ partial-order/Ultralight-Cockpit

This paper is organised as follows: In Section 2, a brief background on the techniques used in this paper is given. This is followed by the formulation of the problem and solution approach in Section 3. Then, the application of the proposed approach is demonstrated in Section 4, which is then followed by the evaluation in Section 5, as well as related work and conclusion in Sections 6 and 7 respectively.

2 Background

2.1 Hierarchical Task Network Planning

Although HTN planning is not the focus of this work, we provide a brief background, in order to make the required context to be inferred for the HTN planner clear. An HTN planner is used to generate a plan provided as a form of assistance to the human operator (i.e. pilot) [Kiam and Jamakatel, 2023]. Using the formalism by [H¨oller et al., 2016], a hierarchical planning problem can be described as a tuple P = (F,C,A,M, s0, tnI, g, δ, ). F is a set of propositional state features. A state s is defined by the subset of state features that hold in it, s ∈ 2F ; all other state features are assumed to be false. s0 ∈ 2F is the initial state of the problem, and g ⊆ F is the state-based goal description. A state s is a goal state if and only if g ⊆ s. A is a set of symbols called primitive tasks (also actions), while C is the set of compound tasks which can be decomposed into primitive tasks using the decomposition methods M. For an HTN-planner to generate a plan (meant as assistance to the pilot), the ongoing context, also known as the problem instance for the planner consisting of the current (initial) state s0, the goal g, and the initial task network tnI must be inferred.

2.2 FOND Planning

FOND planning allows the modelling of non-deterministic action outcomes. We adopt the formalism used by [Muise et al., 2012]. A FOND planning problem ⟨F, I, A, G⟩ consists of fluents F; the propositional state features in a particular domain, I ⊆ F; the initial state, A; a set of actions, and G ⊆ F; the partial assignment of the fluents state. A subset of the fluents F that are presumed to be true while assuming the remaining fluents are false, is a complete state, while a subset of the fluents F without any presumption about the truth of fluents outside the set is a partial state. An action is a tuple with a = ⟨Prec, Eff ⟩, where the precondition Prec is a partial state, and the effect Eff is a finite set of partial states. The non-deterministic outcome of an action results from the non-deterministic selection of Eff from the set of possible effects. An action a is applicable in state s if the state s entails the preconditions of a.

A solution to a FOND planning problem can be expressed either as a policy that maps a state to actions in such a way that an agent can reach a goal or in the form of a graph, where nodes and edges respectively correspond to actions and possible outcomes [Geffner and Bonet, 2013]. Then a solution to the FOND planning problem (also contingent plan) is a graph ⟨N , E, no⟩ [Muise et al., 2019]. The nodes of the graph N correspond to the actions a ∈ A, while the edges E correspond to the possible outcomes of each node. The initial execution node is n0 ∈ N.

2.3 Execution of Contingent Plan-Based Dialogue Agent

An approach for deploying a contingent plan-based dialogue agent is introduced in [Muise et al., 2019]. Here, at each iteration, the execution of the dialogue agent based on a contingent plan involves retrieving the relevant context (note that the context for dialogue execution and context for providing contextual assistance, i.e. planning problem instance, are not the same) and the state for the action corresponding to the contingent plan's current node. This is followed by the execution of the action with the relevant context and then the determination of the action's effect. The outcome is determined by running the relevant subset of the callback functions in the correct sequence and by updating the state of the world based on the outcome [Muise et al., 2019]. The selected outcome from all the possible outcomes is called the realization of the action. After the new state, new context and new realization have been determined, the current node can be updated, and the dialogue is carried out until the goal node has been reached.

2.4 Natural Language Understanding

Natural Language Understanding is an umbrella term for various tasks, e.g. sentiment analysis, question answering, intent classification etc. and involves the extraction of semantics from human language. For dialogue systems, the task of extracting intents and entities is of central importance, in order to classify what the human wants to express with his/her utterance, which contains relevant information associated with the intent. The latter task is also called slot-filling or slotlabelling. Recently, joint models such as Bert [Chen et al., 2019] that can perform both intent classification and entity extraction have gained popularity since they allow for easier training and inference workflow.

2.5 Data Augmentation Using LLMs

Broadly two approaches, either fine-tuning the LLMs followed by the generation of synthetic data generation [Kumar et al., 2019; Anaby-Tavor et al., 2020] and data generation without any task-specific fine-tuning [Yue et al., 2022], can be used for data augmentation usingLLMs. In settings, where enough data is not available for fine-tuning, zero-shot data generation and few-shot data generation can be employed.

For zero-shot data generation, using a customized context prompt, synthetic generation context is set while in few-shot data generation, LLM is provided with some examples of the data and then instructed to generate synthetic data.

3 Active Context Inference Using a Dialogue for Instruction Generation

3.1 ProblemDescription

Let us consider a human-in-the-loop assistance system that can generate contextual strategic plans on how to perform certain goal tasks by engaging in a dialogue with the human. For this, we define a set of unique identifiers for each goal task T = {T1, T2, . . . , Tn} and the corresponding set of contextual strategic plans I ∗ = {I ∗ 1 , I ∗ 2 , . . . , I ∗n }. Consider that for a goal task denoted by Ti, j different strategic plans are possible, i.e., if I ∗ i is the strategic plan for Ti, then I ∗ i = {Ii,1, Ii,2, . . . , Ii,j}, where each instruction is a sequence of k actions to be performed. Let X = {X1 = 1 or 0,X2 = 1 or 0, . . . ,Xn = 1 or 0} be the set of variables that define world state for strategic plan generation context, i.e. the generated plan depends on the state of these context variables. Let Xk ⊂ X be the set of the known variables, whose states are known through passive context inference. Then Xu = X \ Xk is the set of unknown state variables which are relevant for context inference. Then, the humanin-the-loop system that can engage in a conversation with a rational human agent using natural language so that the arbitrary human utterances U can be used to determine the plan generation context Xu should be designed.

3.2 Solution Approach

We design a system with an integrated dialogue executor with NLU capabilities that can map human speech to the state variable symbols and can thus produce a known user-defined state for the variables Xu to be used for strategic plan generation. Figure 1 shows the system-level overview of the designed system. The human interacts with the system in a multi-modal manner, using the Human Machine Interface (HMI). The speech-based communication is transcribed using an Automatic Speech Recognition (ASR) module while the human's interaction with their environment is continuously monitored to infer the state of the world passively. Depending either on the cues from passive observation or on the active request of the human, a dialogue is then initiated and executed by the Dialogue Executor to refine the context inference (i.e.

to supplement the passively inferred context) so that contextual instruction can be generated and communicated via the User Interface (UI).

First, we encode the domain knowledge containing strategic plans or different scenarios as HTNs using the Hierarchical Domain Definition Language (HDDL) [H¨oller et al., 2020] in the Instruction Generator module. Let D = (F,C,A,M, δ) be that domain. Once the planning problem P = (D, so, g, tnI) is specified, the strategic plans I ∗ i on how to perform a goal task can be generated using an HTNPlanner. Here, so is the initial state, g is a state-based goal description and tnI is the initial task network. We define a set of state-based goal descriptions G = {g1, g2, . . . , gn}, such that ∀gi ∈ G, g ⊆ 2F and initial task network corresponding to the goal task for HTN planning T = {tn1, tn2, . . . , tnn}. f1 is a mapping function that maps the state-based goal description to a unique task identifier of each goal task, i.e, f1 : T ↦→ G, while f2 maps the unique task identifiers to the initial task networks, i.e, f2 : T ↦→ T. We also define a set of fluents containing all the required states that needs to be defined for plan generation to each goal task S∗ o = {So ∗1 , So ∗2 , . . . , So ∗n } | ∀So ∗i ∈ S∗ o , So ∗i ⊆ 2F and a mapping function f3 : G ↦→ S∗ o that maps the different goal tasks identifiers in the domain to S∗ o 2 .

The instantiation of f4 : X∗ u ↦→ S∗ o is to be carried out with active engagement of the human who is in the loop. The description of the dialogue agent is formulated in such a way that, for the FOND-Planning problem PF = ⟨F, I,A, G⟩, where G is the goal state 3 defined over a partial state of F, F = FH ∪FC. Here FH contains the fluents that are mapped to the fluents of the HTN planning problem using a mapping function f5 : FH ↦→ Xu. Fc contains the fluents that are responsible for conversational logic. The values for FH are to be inferred by the NLU system from arbitrary human utterances U, i.e. the NLU system can be used to perform the mapping s1 : U ↦→ FH. The solution to this FOND planning problem is a contingent plan, which can be executed by a dialogue executor and will result in a known user-defined state for Xu. With this, the instruction generation problem for a hierarchical planner is defined as P = (D, S∗ o i , gi, tni), such that S∗ o i ∈ S∗ o , gi ∈ G, and tni ∈ T, and can be solved using a hierarchical planner. The mapping of the sequence of the actions obtained from the hierarchical planner into a human-readable strategic plan can be performed by using the generative abilities of LLMs, but is beyond the scope of this paper, and is communicated through an HMI to the human.

4 Application of the Proposed Approach in the Ultralight Domain

4.1 Domain Description

Ultralight aircraft are smaller and lighter aircraft geared mainly towards amateur pilots and thus have less strenuous li-

2 In practice, an HTN planner can find the corresponding hierarchical plans without this mapping function by grounding the HTN planning problem.

3 The local goal of the conversation is different from the goal task, for which the instruction is to be generated.

• (a) HDDL encoding of domain knowledge

(b) Declarative syntax for describing the dialogue agent

Figure 2: Encoding of the world knowledge for instruction knowledge and dialogue flow

censing requirements. An onboard contextual assistance systems for a UL aircraft should be capable of providing strategic plans to perform typical cockpit tasks to the pilot in various situations, thus increases flight safety. We reiterate that although it can be expected from a pilot with several years of training to know and to rationally follow the best course of actions in any situation, in emergency scenarios, especially due to panic, historical evidence shows that this is not always the case [Li et al., 2001]. A contextual assistance system can help provide relevant and rational instructions in emergency scenarios when the pilot is partially impaired but still functional. A goal-directed dialogue system integrated into the assistance system enables the pilot to complete missing pieces of information to determine the context. For the evaluation of the dialogue system, we broadly consider the following two types of emergencies:

Non-nominal flight situations: Various non-nominal flight situations might occur over the course of a flight; for example, an in-flight fire, engine fire etc. It is crucial that appropriate and immediate measures are taken swiftly to prevent catastrophic developments. The measures depend on the characteristics of the fire (its location, color, smell etc.), and are prescribed by the Pilot's Operating Handbook (POH).

Another example of such is a landing gear malfunction. It is the state in which the landing gear of the aircraft is not functioning as expected. Since the landing gear cannot be deployed in this state, a normal landing is not possible and, depending on the severity of the malfunction. An appropriate alternative landing plan must be generated and executed.

General health or cognitive emergencies: Alongside the non-nominal flight situations, in a single-piloted UL cockpit, general health or cognitive emergencies (mental overload, physical impairment etc.) need to be identified and, depending on the severity of the emergency, further flight plans should be generated by conversing with the pilot. Interactive dialogues are even more important in this setting because there is no hard-and-fast rule on how to react to these types of emergencies. Thus, engaging in a conversation allows humans to partake in a guided decision-making process.

The domain knowledge on how to react in these scenarios (i.e. strategic plans) is encoded using HDDL 4 . Figure 2a shows a snippet of the decomposition method for reacting to a fire emergency. As seen in the figure, this decomposition is only applicable if the fire is on the wing of the airplane. The location of the fire is determined from the dialogue. The decomposition of the method results in four atomic actions that should be executed by the pilot. Similar decomposition methods for various states are defined in the domain model for various scenarios. We use the ARIES planner to generate hierarchical plans 5 .

4.2 Dialogue Design and Execution

The dialogue is designed so that the named entities and state variables relevant to each scenario can be extracted from the pilot's utterances expressed in natural language or from formal radio communication so that they can be mapped to the corresponding fluents for the HTN planning problem instance. While it is reasonable to expect that a pilot with some level of formal training will communicate in an unambiguous manner 6 , especially in emergency situations involving inexperienced pilots, divergence from the conventions cannot be excluded. Besides, these conventions are yet to be established for talking to an on-board assistance system 7 .

4 https://github.com/UniBwM-IFS-AILab/FRICO

5 https://github.com/plaans/aries/tree/master

We explain the dialogue design with the example where the characteristics of the fire are to be determined to generate strategic plans. As explained in Section 4.1, this requires determining the location of the fire, the colour of the fire etc. These entities are contained by the utterances which express the pilot's intent to communicate a fire emergency. For example, the fire can either be defined by the state variable defining its color fire description or the variable defining the smoke type smoke description, to infer if the fire is either a fuel-fed fire or an oil-fed fire. Figure 2b shows a snippet of the declarative syntax of a slot fill type action used to extract a smoke description from user utterance. If the pilot's utterance is ambiguous, pre-defined clarifying question clarify message variants can be asked so that the detected entities can be confirmed by the pilot. Alongside the slot fill type dialogue actions, we use dialogue actions for utterances that do not contain entities and api calls to make API calls to external systems. The dialogue agent is a combination of different instances of these types of actions. Figure 3 shows the generated dialogue graph using the PRP planner [Muise et al., 2012] for the dialogue agent. The initial starting point of the dialogue is represented by the red node, while the goal position is indicated by the blue node. As seen in the figure, the human interlocutor can take various paths starting from the initial node to reach the goal node and depending on the path taken, the values of the fluents in FH change. Note that the dialogue graph is generated from the declarative dialogue agent described using thirty different actions. We execute the generated dialogue graph using a modified version of IBM's contingent plan executor called Hovor 8 . The dialogue executoer executes the dialogue graph (Figure 3) by processing the input received and selecting the next actions based on the input until the conversation goal is reached. Even when the intent is not recognised or the slots cannot be filled by the input data, there are provisions in the dialogue graph for re-asking the questions concerning these slots until they are correctly identified. Higher accuracy in intent recognition and entity extraction tasks means that there is no need to repeat the questions, resulting in a smoother dialogue flow and this accuracy is dictated by the performance of the NLU system.

NLU System

The data we use to train the NLU system is grounded on the data solicited from five human pilots with pilot licences (3 SPL, 1 PPL and 1 LAPL) 9 . In the survey, we asked the partic- Human data Synthetic data

6 Pilots, when communicating with the Air Traffic Controller (ATC), are instructed to communicate directly, adhering to a basic communication template.

7 Although it is reasonable to assume that the communication might not diverge greatly from the ATC communication pattern, the cases in which the divergence is present, must be accounted for too. 8 https://github.com/IBM/contingent-plan-executor

9 Note that collecting data at a much larger scale in this type of domain is financially not viable in our current research work.

0 2 · 10-2 4 · 10-2 6 · 10-2 8 · 10-2 0.1

(b) Chamfer distance

Figure 4: Diversity of human utterances and synthetic data

ipants to verbalize how they would express their intent in the different aforementioned scenarios. Since it is not possible to account for all the permutations in which the intents might be expressed (e.g. combinations of different intents in a single sentence, omission of certain entities from the intents, personal preference while expressing similar intents etc.) from the survey data alone, we first homogenize the collected survey data into 12 domain-specific classes. Then, we augment this data using a large language model. Techniques like logit suppression and altering the temperature of the sampling distribution [Goodfellow et al., 2016] and zero-shot and fewshot generation [Li et al., 2023] have been used to increase the data diversity in literature. For data augmentation, we apply the approach of few-shot synthetic data generation using gpt-4-turbo-preview, where we provide humansolicited examples to the LLM and ask it to generate similar data. First, we set the following context prompt to describe the setting: "You are a pilot onboard an ultralight aircraft. You want to express that you are facing an emergency." We then introduce the generation prompt (provided in the same repository as the HDDL) using the temperature value of 1, the frequency penalty and the presence penalty of 0.19. From the generated data, the 30 most realistic utterances are then manually selected so that the intents, as well as the entities present in the utterances expressing the intents, are comparable with human data. Human-generated and synthetic data are combined to train the NLU system. Note that we prefer the OpenAI's GPT model to AviationGPT [Wang et al., 2023], an LLM fine-tuned on the aviation domain to achieve higher data diversity including utterances that a non-experienced pilot might say. We use the DIET architecture [Bunk et al., 2020] as the basis of the NLU system. We use the pre-trained BERT model weight [Devlin et al., 2018] and by using the RASA NLU pipeline, we train our NLU system for 5 epochs and achieve an intent classification accuracy of 89% and an entity F1 score of 90%.

5 Evaluation

In this section, we evaluate the performance of the proposed dialogue system. We reiterate that in this paper we are not interested in how well the system could perform in a realistic flight-mission-like setting, but rather in whether the system is at all suitable in such settings. We first evaluate the diversity of the data generated by the LLM, which we use to train our NLU system. Based on [Cox et al., 2021], we choose the Remote Clique Score (the average pairwise mean distance) and the Chamfer Distance Score (the average of minimum

General

Landing

Fire pairwise distance) to quantify the diversity of both humancollected data and LLM-generated data. The remote clique score is insensitive to highly clustered points, whereas the Chamfer distance is biased when points are clustered. We use the embedding from the pre-trained BERT model before finetuning to generate the vector representation of the utterances and then calculate the two measures. As seen in Figure 4, the synthetic data and human data are comparably diverse. We recall that this is because pilot utterances resemble preestablished communication templates in the aviation domain.

For goal-directed dialogue execution, we compare the performance of our system with that of two GPT models (gpt-3.5-turbo-0125) and (gpt-4-turbo-preview) as representatives of the LLMs. We collected data from three test persons by allowing them to interact with the dialogue system with text input in two configuration. In the first configuration, the user provide all the required information to determine the context variables at once (Config 1). In the other configuration (Config 2) the user initially provides information on partial context and iteratively provides information on the missing context variables depending on the question asked by the dialogue agent 10 . Follow-up questions are to be asked by the dialogue agent, reasoning over the currently known world state. For Config 1 and 2, we define success metrics as the number of successful executions of the dialogue to achieve user-defined context variables. If it takes more dialogue steps than the longest path required to instantiate the context variables generated by the FOND planner (See Figure 3), we assume that the dialogue execution has failed 11 . While the dialogue execution may eventually lead to a goal state in such a case, it is a reasonable assumption that the success criteria for a safety-critical goal-directed dialogue system should exhibit optimal path qualities. The rationale behind using these two different configurations is to verify whether the LLM models can ask the appropriate questions by reasoning over the world state. Prompts used for this experiment will be available in the paper repository.

Table 1 shows the results of our comparison. The scenarios are ordered according to the increasing complexity (i.e. how many context variables the dialogue system has to reason over). For a landing gear malfunction in Config 2, values of two context variables have to be determined and there are no follow-up questions, whereas for general health emergencies, values of two context variables have to be determined and based on the value, follow-up questions have to be asked. Depending on the starting point of the user and on the number of variables required to define the context without disambiguity, for the first two emergencies, two different paths are possible, while for the fire emergency, seven distinct paths are possible (note that we disregard permutations of the same path). Our analysis shows that in Config 1 (it is essentially a named entity extraction task and no reasoning is required), both GPT 3

10 In practice, the communication happens through speech signals, but we use this configuration to minimize the errors induced by ASR model for comparison.

11 Asking more questions than specifically required in an emergency scenario is not desirable.

and GPT 4 models can infer the context variables. However, in Config 2, the performance of the GPT models degrades. While GPT-4 performs better than GPT-3, the performance is still inadequate. We postulate that this degradation can be accounted for by the fact the GPT models are forced to reason over the context values to determine the follow-up question. Although the performance of the GPT models in our setting for Config 2 is poor, we acknowledge that prompting techniques can have an influential role and the performance can be improved by using different prompts. However, these techniques still do not have success guarantees.

Since the hierarchical planners can only generate corresponding plans once the context is well-defined, the results for plan generation based on the conversation are the same as Table 1. In our experiments, we notice that by setting the context of LLM with various plans as examples, followed by asking for specific plans in an end-to-end setting, the plan generation fails in cases where scenarios differ from one another only by a singular context variable. This is consistent with our findings about in-context dialogue execution using only LLMs.

6 RelatedWork

Hierarchical planners with an integrated information state approach for dialogue management [Larsson and Traum, 2000] have been used to generate technical instructions at different levels of abstraction in a Do-It-Yourself home improvement domain [Behnke et al., 2020]. Here, domain knowledge is stored in an ontology, which is used both by the planner and dialogue manager. Similarly, coupled with a natural language generation system, a hierarchical planner has been used to generate building instructions at different levels of abstraction inMinecraft [K¨ohn et al., 2020]. In the ultralight domain, the generation of strategic plans using hierarchical planners has been demonstrated in [Jamakatel and Kiam, 2024].

This paper proposes a combination of active and passive context inference for generating contextual strategic plans. Inferring the context using a dialogue with the human in the loop requires that the intents as well as the entities from human utterances can be efficiently extracted. This relates to the task of joint intent detection and slot-filling, a well-studied problem in natural language understanding. We refer to a recent survey [Weld et al., 2022] for an overview of different approaches. The BERT [Devlin et al., 2018] language model has been demonstrated to perform well in joint intent detection and slot-filling tasks [Chen et al., 2019]. Another transformer-based, widely-used language understanding model is DIET [Bunk et al., 2020]. Such models are trained on large corpora of domain-specific data, however, in certain domains, such large corpora of data are not available. Recently, the use of synthetic data has gained popularity and synthetic text generation for text classification data has been explored by various authors in literature [Li et al., 2023; Yue et al., 2022; Aggarwal et al., 2023]. An approach of diverse yet accurate text data generation with large language models and human interventions is introduced in [Chung et al., 2023]. An LLM fine-tuned with aviation data has been presented in [Wang et al., 2023]; similarly, an LLM-based Air Traffic Controller agent has been introduced in [Abdulhak et al., 2024].

Besides the task of NLU, for a dialogue system whose primary interface is through audio, a near real-time ASR system is needed. Recently, Whisper [Radford et al., 2022], a general-purpose speech recognition model, has shown stateof-the-art results, and efforts have been made to increase the inference speed of the model [Gandhi et al., 2023] with robust results. The reverse procedure, i.e. the conversion of text to speech, is known as speech synthesis and several datadriven approaches have been developed [Tatanov et al., 2021; Kim et al., 2021]. The dialogue is executed by the underlying dialogue system. End-to-end goal-directed dialogue systems are trained on large-scale datasets [Zhang et al., 2019; Xu et al., 2020]. These systems are, however, bounded by the quality of the training data and often lack transparency about their decisions, making them unsuitable for domains where transparency is crucial. Finite-state dialogue managers, e.g. [Rojas-Barahona and Giorgino, 2009], are usually hand-crafted and require higher effort for creating and updating and are thus not easily maintainable. Various solutions for dialogue agents like IBM's Watson Assistant 12 , Google's Dialogflow 13 , Amazon's Lex 14 and Microsoft's Bot Framework 15 exist. However, these systems require conversation paths to be explicitly captured in dialogue trees or graphs, making them hard to extend and maintain [Muise et al., 2019]. An approach based on FOND-Planning, where instead of creating and maintaining dialogue graphs, implicit dialogue graphs are compiled from declarative behaviour description, is introduced in [Muise et al., 2019]. Declarative syntax to describe mixed-initiative dialogue has also been presented in [Steedman and Petrick, 2007]. An open-source declarative dialogue planning framework called Plan4Dial has been introduced in [De Venezia and Muise, 2023]. A survey of a plan-based approach for dialogue management can be found in [Santos Teixeira and Dragoni, 2022].

7 Conclusion and Limitations

In this paper, we introduced a neuro-symbolic approach for goal-directed dialogue design by combining planning-based approaches for dialogue execution and strategic plan generation with a neural AI-based method for data augmentation and natural language understanding. Rather than relying solely on LLMs for generating end-to-end goal-directed dialogue, we use dialogue planning to execute verifiable conversations while exploiting the generative capabilities of LLMs to complement scarce human-generated data for the NLU system. We validate the proposed approach in a safety-critical domain of ultralight aircraft for context inference. We showed that generated data for NLU after verification from the humandomain expert can result in similar data diversity metrics. Since the dialogue agent created using this approach is guaranteed to reach the goal state by design, such a dialogue agent is comparable to rule-based dialogue systems. This approach is, however, easier to maintain and expand because of the declarative syntax used for the design. We also demonstrated that in a safety-critical domain, current LLMs are unable to deliver the required results, rendering them unsuitable. Our neuro-symbolic approach exhibits transparency on the flow of the dialogue; its decisions could be explained (if required) and the dialogue be tailored by domain experts, which is essential for human-centred AI.

12 https://www.ibm.com/cloud/watson-assistant/

13 https://cloud.google.com/dialogflow/

14 https://aws.amazon.com/lex/

15 https://dev.botframework.com/

One key limitation of this approach is that if the local conversation goal changes mid-conversation, the dialogue agent can be stuck in the loop (currently solved by resetting the conversation context variables) because the outcomes expected by the dialogue agent do not correspond to the action realization. In future works, an approach to adjust the dialogue paths to the goal mid-conversation should be implemented without requiring re-planning or restarting the dialogue. Furthermore, the natural language understanding capability of the dialogue can be augmented by using abstract meaning representation. The capability of generating the declarative dialogue syntax using LLMs should also be explored as a way to accelerate the dialogue design and the domain should be expanded to entail all possible emergency scenarios. This work focuses on the system-level design; the usability of the designed system in a realistic and continuous mission scenario still needs to be evaluated.

Ethical Statement

In safety-critical collaborative settings where intelligent systems assist humans, the provided assistance should be verifiable by domain experts. There should be guarantees concerning the system's functionality and soundness. The planningbased approach can be used to achieve these desired properties. The NLU system, which plays a central role in our dialogue system, relies on human and synthetic data. This data has to be carefully examined and verified by the dialogue agent designer so that various edge cases can be accounted for, since unaccounted edge cases could have disastrous consequences. The proposed system will pave the way for reliable, verifiable dialogue agents in safety-critical domains if proper safety measures are taken during data curation.

Acknowledgments

This work is funded by the German Federal Ministry of Economic Affairs and Climate Action (Project MOREALIS).