BTPG: A Platform and Benchmark for Behavior Tree Planning in Everyday Service Robots

1 Introduction

One of the main goals of robotics and AI is to develop intelligent robots capable of autonomously making decisions and executing behaviors to accomplish various tasks. This requires not only intelligent planning algorithms, but also a safe and robust control architecture, which is particularly crucial for everyday service robots. Behavior Trees (BTs) have become a popular control architecture for robots exactly due to their ability to ensure safety and robustness [Colledanchise and ¨Ogren, 2018; ¨Ogren and Sprague, 2022]. Because of their modular and adaptable tree structure, BTs can effectively perform various tasks and handle uncertain environments [Colledanchise et al., 2019a]. In addition, their clear and interpretable control flows enhance the reliability and predictability of robot behavior, making BT systems easy for humans to design, deploy, and scale.

However, despite the advantages of BTs, researches on automatic BT generation has been relatively slow to progress and yet far from practical application. Recently, BT planning [Cai et al., 2021; Chen et al., 2024; Cai et al., 2025b; Colledanchise et al., 2019a; Cai et al., 2025a] has emerged as a popular approach to automatically construct BTs toward specific goals (as illustrated in Figure 1). This approach is based on easily designed action models and can theoretically guarantee the success of planned BTs. Typically, when using interpretable representations of BT nodes, such as predicate logic, the goal conditions and heuristics for BT planning can be well interpreted and reasoned about using Large Language Models (LLMs) [Chen et al., 2024; Cai et al., 2025b]. These advantages make BT planning a promising approach for automatic BT generation in everyday service robots.

However, to the best of our knowledge, existing BT planning methods mainly face two significant challenges: (1) Prior BT planning research constructs experiments on various platforms, and their simulation environments are quite naive. This makes it difficult to conduct a unified evaluation of existing BT algorithms. Therefore, a standard platform for real-life environments and evaluation datasets is urgently needed for this field to progress. (2) Current performance evaluation metrics focus primarily on planning efficiency and execution efficiency. However, some important attributes of the planned BTs, such as the attraction region and robustness, are less emphasized.

To address these two issues, we propose Behavior Tree Planning Gym (BTPG), the first platform and benchmark for BT planning in everyday service robots. To establish BTPG as a standard for different BT planning methods, we focus on the following three efforts: (1) We unify the BT representation by adopting the popular predicate logic form [Chen et al., 2024] for BT nodes. To efficiently define the domain of each action and condition predicate, we group objects into categories based on their attributes. Subsequently, the BT

planning problems are formulated using STRIPS-style action models, which are general enough to model common environments in everyday service scenarios. (2) We build BT systems in four environments: RoboWaiter [Chen et al., 2024], VirtualHome [Puig et al., 2018], OmniGibson [Li et al., 2023], and RobotHow [Liao et al., 2019]. The first three environments each include a simulator with which the robot can actually interact, while the last one is used for large-scale computational tests. We design action models in each environment for BT planning respectively. (3) We design a dataset generator for each environment that can automatically generate random task goals while guaranteeing their reasonableness. Then we perform comprehensive evaluations of three state-of-the-art BT planning algorithms, as well as one proposed by us, using various common metrics. Furthermore, we design three advanced metrics - planning progress, region distance, and execution robustness - to gain deeper insights into these BT planning algorithms. Through the efforts described above, we hope BTPG can become a standard platform and benchmark that can inspire and accelerate progress in BT planning research.

2 Background

A BT is a directed rooted tree where the behavior nodes (leaves) manage the robot's perception and execution, while control nodes (internals) manage the ticking logic flow to determine which action should be executed in the current state [Colledanchise and ¨Ogren, 2018]. Current BT planning approaches focus mainly on four types of BT nodes:

• Condition. A behavior node that checks whether the specified condition holds in the current state, and returns either success or failure.
• Action. A behavior node that controls the robot to perform the specified action, and returns either success, failure, or running.
• Sequence. A control node that ticks its children from left to right and returns success when all its children succeed. Otherwise, it returns failure or running according to the first non-success status it encounters.
• Fallback. A control node with opposite return logic to the sequence node, which means it ticks its children from left to right and returns failure when all its children fail. Otherwise, it returns success or running according to the first non-failure status it encounters. The typical structure of the BT is illustrated in Figure 1.

Predicate Logic for Node Representation. In the predicate logic representation [Chen et al., 2024; Cai et al., 2025b], condition and action nodes are represented by a tuple < R,Q,O > , where R is the condition predicate set, Q is the action predicate set, and O is the object set. In this formulation, a condition node can be denoted as c = r(o1, ..., oi), r ∈ R while an action node can be denoted as a = q(o1, ..., oj), q ∈ Q.

STRIPS-style Action Model. In the STRIPS-style planning formulation [Fikes and Nilsson, 1971]. S is the finite set of environment states, A is the finite set of actions, M is the action model. Both the state s ∈ S and the condition c are represented by a set of atom conditions. c ⊆ s means condition c holds in the state s. The state transition affected by action a ∈ A can be defined as a triplet M(a) =< pre(a), add(a), del(a) >, consisting of the precondition, add effects, and delete effects of the action. After the action's execution, the subsequent state s′ = s ∪ add(a) \ del(a).

ProblemFormalization. The BT planning problem is a tuple: < S, A,M, D, s0, g >, where D : A ↦→ R+ is the cost function, s0 is the initial state, g is the goal condition. For each BT T includes a path p ∈ T , p = (a1, a2, ..., an) that achieves g from s0, BT planning aims to find the optimal BT T∗ with minimal execution cost: T∗ = arg minT ∈T D(T ) = arg minT ∈T ∑ni=1 D(ai).

3 RelatedWork

3.1 Behavior Tree Planning

Current studies on automatic BT generation generally have certain limitations. Evolutionary computing [Neupane and Goodrich, 2019; Colledanchise et al., 2019b; Lim et al., 2010], reinforcement learning [Banerjee, 2018], imitation learning [French et al., 2019], and MCTS [Scheide et al.,

2021] require implicit goals of BT, such as fitness functions or reward functions, which are difficult to translate from human instructions. Formal synthesis [Tadewos et al., 2022; Neupane and Goodrich, 2023] requires complex environment modeling and has rather high computational complexity. BT generation directly using LLMs [Izzo et al., 2024; Li et al., 2024; Lykov and Tsetserukou, 2023; Lykov et al., 2023] is adaptable to the diversity of human instructions but cannot guarantee the reliability of BTs.

In contrast, BT planning has explicit goal conditions, easily designed action models, and the theoretical guarantee of the success and reliability of planned BTs. The state-of-theart BT planning methods mainly include: (1) BT Expansion [Cai et al., 2021], a sound and complete algorithm for BT planning where planned BTs are guaranteed to be finite time successful. (2) OBTEA [Chen et al., 2024], Optimal Behavior Tree Expansion Algorithm. It is based on BT Expansion but designed to plan optimal BTs with minimal execution cost. However, the computation complexity is high when the action space becomes large. (3) HBTP [Cai et al., 2025b], Heuristic Optimal Behavior Tree Expansion Algorithm. It uses LLMs to provide action space pruning and planning heuristics through commonsense reasoning. This approach has advantages in both planning and execution efficiency, but places high demands on the task understanding and heuristics generation capabilities of LLMs.

3.2 Benchmark Environments for Robot Planning

The most common benchmarks for robot planning are designed for low-level control tasks. For path planning, there are Plannie [Rocha and Vivaldini, 2022], MRPB 1.0 [Wen et al., 2021], and PathBench [Toma et al., 2021]. For manipulation planning, there are MotionBenchMaker [Chamzas et al., 2022], the motion planning pipeline [Liu and Liu, 2022], and the grasp planning benchmark [Bekiroglu et al., 2020]. In recent years, learning methods like reinforcement learning [Bai et al., 2023; Bai et al., 2025b; Bai et al., 2025a] have become popular approaches for robot control planning, with many benchmark environments proposed, such as Safety Gymnasium [Ji et al., 2023] and RLBench [Ning et al., 2023]. For high-level behavior planning tasks, existing works are mostly in the field of embodied intelligence, and LLMs are often used to generate plans [Valmeekam et al., 2023; Singh et al., 2022; Chen et al., 2023]. However, the lack of reliability is a critical flaw when LLMs directly generate plans, not to mention the safety and robustness during the execution of these plans. BTPG fills this gap by providing a platform and benchmark for BT planning, where reliable and robust BTs for high-level behavior control can be automatically generated, without needing to access low-level control of the robots.

4 Behavior Tree Planning Gym

In this section, we first briefly introduce the object categorization that assists in modeling the BT planning problem. We then provide detailed information on the BT simulation platforms and the BT planning benchmarks. The framework of BTPG is illustrated in Figure 2.

Object Categorization. In BTPG, behavior nodes are represented using predicate logic, and action models are represented in the STRIPS style. Consequently, defining the domains of predicates and the state transitions for each action instance becomes more challenging as the number of available objects in the environment increases. Object categorization [Puig et al., 2018] is a useful technique to improve the reusability of predicate and action model templates, based on the shared attributes of objects. This approach utilizes the abstraction of the world through human common sense. One inspiration from it is that conditions and actions can also be categorized, potentially leading to the development of more efficient BT planning algorithms based on this hierarchical representation in the future.

4.1 Behavior Tree Simulation Platform

The BT systems in BTPG are all built in Python, which can communicate with external simulators such as Unity or Unreal Engine to implement low-level robot control. We design four environments in which various BT planning tasks can be tested. These environments have different themes and problem scales. Detailed information on these environments is listed in the Appendix.

RoboWaiter (RW). In this environment, the robot acts as a waiter to perform tasks according to customer needs, such as delivering coffee or cleaning tables. The planned BTs can be executed in a simulator developed based on Unreal Engine 5 by Dataa [Chen et al., 2024]. The low-level robot control for action nodes is based on scripted APIs. The task difficulty in this environment is relatively low.

VirtualHome (VH). In this environment, the robot needs to accomplish various household tasks, such as washing fruits or turning on the TV. The planned BTs can be executed in the VirtualHome [Puig et al., 2018] simulator. The low-level robot control is implemented using programmed scripts in a specific format. The task difficulty in this environment is medium.

OmniGibson (OG). In this environment, the robot executes everyday household activities in IsaacSim, a more realistic simulator developed by Nvidia. The robot and object sources are from the BEHAVIOR-1K dataset distributed with OmniGibson [Li et al., 2023] platform. The low-level robot control for action nodes is implemented using rule-based navigation and manipulation algorithms. The task difficulty in this environment is medium.

RobotHow (RH). This environment is inspired by the RobotHow knowledge base [Liao et al., 2019] which includes a number of activity programs for household robots. This environment is designed for large-scale computational tests, and the planned BTs can be executed in a headless environment without simulation deployment for now. The task difficulty in this environment is high.

In BTPG, we propose the Behavior Tree Markup Language (BTML) to represent BTs, which is more concise and readable than XML in terms of BT representation. Details about BTML can be found in Appendix.

Table 1: The detailed problem scales of four environments.

4.2 Behavior Tree Planning Benchmark

Universal Heuristic Behavior Tree Planning

We implement all three state-of-the-art BT planning algorithms in BTPG as mentioned in Section 3.1. Additionally, we further propose a new BT planning algorithm inspired by classical planning. We notice that current heuristic BT planning algorithms [Cai et al., 2025b] mainly use domain-dependent heuristics produced by LLMs. However, domain-independent heuristics have been widely studied in the field of classical planning [Hoffmann and Nebel, 2001] but are lacking in BT planning algorithms. In this paper, we simply replace the heuristics used in HBTP with domainindependent heuristics and propose Universal Heuristic Behavior Tree Planning (UHBTP).

We start with the delete relaxation used in GRAPHPLAN [Hoffmann and Nebel, 2001]. Consider a relaxed problem where, for each action, del(a) = ∅. In this case, we start from the initial state and apply all actions whose preconditions are satisfied in the current state at each step, resulting in a sequence of literal sets L1, L2, . . . , Ln, where L1 = s0 and g ⊆ Ln. We let Ln+1 = L, then we subtract these literal sets layer-by-layer to form heuristic plan layers: L′1 = L1,L′i+1 = Li+1 \ Li, i = 1, 2, ..., n. For these heuristic plan layers, we have:

The heuristic for each literal can be defined as:

The heuristic for conditions is the sum of their literals:

Although the above heuristics are simple, they are efficient in guiding BT planning by prioritizing the exploration of potentially shorter action paths, thereby accelerating the planning process. The comparison results of the performance of UHBTP with other existing BT planning algorithms are shown in the Experiment section.

Dataset Generator

Given an action model and human prior knowledge, planning tasks within the environments can be automatically generated. In BTPG, we designed a dataset generator for each environment to randomly generate batches of tasks to test the performance of planning algorithms. Typically, the dataset generator first randomly generates the environment state and goal conditions according to certain distributions to form the planning task. These distributions can be set to uniform distributions if there are no special requirements. Then, the optimal path is obtained using OBTEA as a reference for evaluating other planning algorithms. The final task goals can include a single goal condition or a combination of multiple goals, with the number of goal conditions controlling the difficulty of the planning tasks. Details of the dataset generator and the data format are provided in the Appendix.

Metrics Design

Common Metrics. Our benchmark includes the following common evaluation metrics to evaluate the performance of BTs: (1) Planning efficiency, including planning time and timeout rate; (2) Execution efficiency, including executed actions, costs, tree sizes, and the number of ticks during execution.

In addition to the above common metrics, we propose the following three advanced metrics in this paper. These metrics aim to provide deeper insights into BT planning algorithms and enable a more comprehensive evaluation.

Planning Progress (PP). For the planning efficiency of BT planning algorithms, simple metrics like planning time and the timeout rate may be insufficient. It would be helpful if we could understand the specific advance of BT planning at each exploration and estimate the overall progress of the whole planning procedure. Therefore, we propose the Planning Progress metric. This metric is based on the optimal paths provided in the datasets. Assuming we have an optimal path set P ∗ for a given task, the planning progress at time t can be calculated as follows:

where Tt is the BT planned at the current time, A(p∗ ) is the set of actions shown in path p∗ , I(p, a) is the action count that shows how many times action a appears in the path p.

With this metric, we can estimate the ability of the planning algorithm to finish the task before it completes the search. This is especially helpful when the problem scale is large and the algorithm cannot finish within the expected time limit.

However, since it is impossible to maintain all optimal paths in the datasets, this estimation could be inaccurate when multiple optimal paths are available.

Region Distance (RD). It is also helpful to know the distribution of path lengths starting from each condition within the attraction region of the planned BT, which we refer to as Region Distance. This metric provides two main insights: it allows us to understand the depths on which the search process primarily focuses, and it gives us a rough idea of which tasks the final planned BT can accomplish. Given an interval [α1, α2), we can count the number of conditions whose corresponding path lengths fall within it:

where p(T , c) is the path from condition c to goal g within the BT T , and C(T ) is the set of conditions in the attraction region of T .

With the same number of explorations, shorter path lengths within the attraction region indicate that the BT primarily explores the region near the goal, making it more likely to fail in long-horizontal tasks. Therefore, we generally expect a larger Region Distance for the planned BTs.

ExecutionRobustness (ER). Robustness is one of themost important advantages that distinguish BTs from other control architectures. However, it is rarely addressed in current BT planning literature. In this paper, we test the robustness of planned BTs by introducing noise into the environment. We first define a noise function N with a noise parameter ϵ ∈ [0, 1], which indicates the probability of each condition in the environment being disturbed. We set up disturbance functions for each environment to ensure the disturbances are reasonable and reflect real-world conditions. We have the initial noise parameter ϵ1 and the transition noise parameter ϵ2, which respectively test the BT's ability to execute tasks under conditions not targeted in the planning and its robustness under uncertainties on the environment transition. Under the effect of the noise function, the environment transitions as follows:

We measure the execution robustness of a BT by the probability that it successfully reaches the goal in a noisy environment. We assume that the planned BTs succeed in a finite time in the absence of noise, meaning that when the environment state falls in the attraction region of the BT, it can follow the predetermined path to the goal. Thus, as long as the BT has not returned a failure, it is considered to be progressing towards the goal. Execution Robustness can then be defined as follows:

where failure = T (s′i ) indicates that the BT T returns a failure under state s′i at time t.

Achieving greater robustness in the planned BTs typically involves higher planning and execution expenses. Therefore, the trade-off between efficiency and robustness is a key challenge that future BT planning algorithms need to address.

5 Experiments

To evaluate the performance of BT planning algorithms in our benchmark, for each environment, we randomly generated 100 single-goal tasks under a uniform distribution, as well as 100 multi-goal tasks where the number of goal conditions ranged from 2 to 3. We implemented the following five BT planning algorithms in BTPG: BT Expansion, OBTEA, HBTP, UHBTP and HBTP-Oracle, where in HBTP-Oracle we assume that the heuristics provided by LLMs are always accurate. All our experiments were performed on an AMD Ryzen 9 5900X 12-Core Processor (3.70 GHz).

5.1 CommonMetrics

Planning Efficiency. We set a planning time limit of 1 second for each algorithm. If the time limit is exceeded, planning is stopped and its execution efficiency is not recorded. As we can see from Table 2, BT Expansion has the lowest planning efficiency while HBTP-Oracle has the highest. The heuristics used by the HBTP algorithm significantly improved planning efficiency compared to OBTEA, but there is still a considerable gap compared to HBTP-Oracle. More surprisingly, even UHBTP, which uses only domain-independent heuristics is faster than HBTP. This indicates that there is still much room for improving the reasoning abilities of LLMs to enhance the BT planning efficiency.

Execution Efficiency. As shown in Table 2, when the timeout rate is close to 0, we can observe that algorithms using heuristics generally have higher executed actions and costs than OBTEA. However, compared to the significant improvement in planning speed they achieve, these minor sacrifices in execution efficiency are acceptable. Moreover, heuristics can lead to smaller tree sizes and lower tick counts, indicating that the generated tree structures are more concise. However, when the timeout rate increases, the assessment results for Execution Efficiency will lose statistical significance, as many tasks fail to generate an executable BT.

5.2 AdvancedMetrics

Evaluation of Planning Progress. As shown in Figure 3, we can observe a clear phenomenon where the PP in the early stage is generally positively correlated with the final planning efficiency. This indicates that our PP metric can effectively reflect the planning efficiency of an algorithm early in the planning procedure, without waiting for the entire planning to complete. This is particularly helpful in complex environments, such as RH. Furthermore, most of the planning progresses are achieved early on, and as planning continues, the increase in PP slows down significantly. This indicates that the difficulty of planning in the later stages is much higher than in the early stages. This is because the number of combinations between conditions increases dramatically as more conditions are explored. This insight suggests that improving planning efficiency in the later stages of planning might be a worthwhile research direction. Another phenomenon regarding the planning efficiency of OBTEA and BT Expansion has been observed. In the OG environment, the planning progress of BT Expansion eventually surpasses that of OBTEA, while in other environments, it is generally lower. This suggests that the planning efficiency of the algorithms may depend on the specific action model of the environment.

Evaluation of Region Distance. We conducted Region Distance experiments on the multi-goal dataset, where each algorithm was run for only 10 explorations. Despite the limited number of explorations, we can still observe differences between the algorithms. As shown in Figure 4, the Region Distance metric indicates that BT Expansion's Region Distance is generally distributed within a smaller range. The Region Distance for HBTP and UHBTP is relatively larger, which suggests that with the help of heuristics, the algorithms tend to explore states in more distant regions. Additionally, the performance of HBTP is not significantly different from its theoretical bound, HBTP-Oracle. From this experiment, we can preliminarily conclude that higher planning efficiency often implies that the algorithm explores regions at sufficiently far distances.

Evaluation of Execution Robustness. To test the robustness of BTs generated by different algorithms, we set various noise parameter settings in each environment, ran each algorithm with 5 random seeds, and calculated the average completion rate for the BTs generated by each algorithm. As shown in Table 3, when only the environment transition noise ϵ2 is present, these BTs have a relatively high success rate on the single-goal set. However, the success rate significantly drops on the multi-goal set. This is because the robot needs to complete multiple goals simultaneously, and noise interference in any single goal can lead to the failure of the entire task. Next, when comparing the second and third settings, we found that in VH, OG, and RH, the success rate with ϵ1 = 1, ϵ2 = 0 is higher than with ϵ1 = 0, ϵ2 = 0.5. However, in the RW environment, the opposite is true. This indicates that the robustness of BTs generated by planning algorithms is depends on the type of tasks in different environments. Finally, we observed that in all random perturbation environments, the robustness of the BTs generated by all algorithms is generally low. This highlights the severe deficiencies of current BT planning algorithms in terms of the robustness of the generated BTs. Improving the robustness of planned BTs is a significant challenge that needs to be addressed in future work.

6 Conclusion

This paper presents BTPG, the first platform and benchmark for Behavior Tree (BT) planning in everyday service robots. In BTPG, we adopt a predicate logic representation for behavior nodes, formulate the BT planning problem using a STRIPS-style action model, and use object categorization to aid definition. We create four environments with three simulators that encompass most everyday service activities. Additionally, we design a dataset generator for each environment. We implement and test three state-of-the-art BT planning algorithms, as well as UHBTP proposed by us, using various common metrics and three advanced metricsto gain deeper insights into these algorithms. With a standard test benchmark, we hope BTPG can inspire and accelerate progress in the field of BT planning. Future work includes continuing to improve the success rate of BT execution in the simulators, incorporating more BT planning algorithms, and providing a more comprehensive user manual and community support.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant Nos. 62032024 and 62372459) and the Independent Innovation Science Fund Project of NUDT (Grant No. 23-ZZCXKXKY-07).

Contribution Statement

Yishuai Cai and Minglong Li are corresponding authors.