Learning to Order Objects Using Haptic and Proprioceptive Exploratory Behaviors

1 INTRODUCTION

The ability to order physical objects by various properties emerges early in childhood development and is thought to be fundamental for understanding the property of numbers [Kingma and Reuvekamp, 1984]. In the Montessori method of teaching [Montessori, 1917], children solve ordering tasks using specialized sets of toys that allow the child to learn the target dimension along which the objects should be ordered [Pitamic, 2004]. Ordering tasks also appear in intelligence tests [Hagmann-von Arx et al., 2008] which suggests that ability to order objects is an important aspect of human intelligence.

In cognitive robotics, there have been relatively few works exploring how a robot can learn to order a set of objects. Instead, most related work has focused on the problem of object categorization, i.e., assigning an object to 1 or more discrete categories (see [Sanchez-Fibla et al., 2013; Orhan et al., 2013; Y¨ur¨uten et al., 2013; Sinapov et al., 2014a;

Chu et al., 2015] for a representative sample). Such methods have shown that using behaviors in conjunction with visual features allows a robot to ground the meaning of nouns and adjectives. However, many object properties cannot be represented well using discrete categories. For example, "height" is better represented by an ordering rather than a discrete clustering as the ordering captures the continuous nature of the property while the clustering does not.

To address this gap, we propose a framework for learning object ordering skills that are grounded in a robot's physical interactions with objects. In our method, the robot uses unsupervised learning to discover how objects can be ordered using multiple and different types of sensorimotor features that the robot detects during object exploration. Next, the robot undergoes a supervised learning stage during which it is trained that specific example object orders are associated with specific object properties, namely "height", "width", and "weight". More specifically, the robot learns how the three ordinal concepts relate to the object orders discovered in the previous stage. The learned model is subsequently used by the robot to recognize the property according to which a new series of objects is ordered by. Finally, the robot uses this recognition ability to correctly insert additional objects into existing object series.

2 RelatedWork

2.1 Psychology and Cognitive Science

Ordering objects emerges early in childhood - by age 2, children can compare and sort objects according to size [Graham et al., 1964]. Other studies have shown that learning to classify objects as "big" or "small" is easier when the object is directly comparable to other objects in its surroundings as opposed to using only its absolute size (see [Ebeling and Gelman, 1994; 1988]). Similarly, comparing objects to each other helps children learn to order objects according to their height [Smith et al., 1986].

In addition to supervised learning of relative concepts, some studies have also looked at whether children form order representations in an unsupervised manner. The experiments described in [Sugarman, 1981] found that the order of objects that children explore freely is influenced by how perceptually similar the objects are. For example, if the child is exploring a small toy, it is more likely to switch to a slightly bigger one as opposed to one that is much bigger. This result suggests that children can order objects in an unsupervised manner and can extract certain natural orders from interaction with objects.

Based on these findings, the framework proposed in this paper uses both unsupervised and supervised learning. First, the robot uses an unsupervised approach to detect a range of possible orders for a given set of objects. This is followed by a supervised learning stage in which the robot grounds the concepts "weight", "height", and "width" in terms of the orders discovered in the unsupervised stage.

2.2 Robotics and AI

Grounding concepts related to object ordering has received very little attention in robotics so far. In machine learning, ranking [Liu, 2009] is a related problem in which items (e.g., search results) need to be ranked according to some criteria (e.g., user's preferences and a search query). While such methods could be adapted to object ordering problems, they typically assume that there is a lot of training data and only learn one ranking relation at a time using a single flat feature vector representation. In addition, most methods focus on getting the top few results correct, i.e., the cost of mistakes depends on the position of the ranking.

In cognitive robotics, most related work focuses on the problem of grounding discrete object categories in visual features (e.g., [Gorbenko and Popov, 2012; Lai et al., 2011]) as well as non-visual sensory modalities coupled with manipulative behaviors (e.g., [Hogman et al., 2013; Orhan et al., 2013; Y¨ur¨uten et al., 2013; Nakamura et al., 2014; Sinapov et al., 2014a; Celikkanat et al., 2015; Chu et al., 2015]). Using such methods a robot can ground the meaning of nouns (e.g., "pop can", "ball", etc.) as well as adjectives (e.g., "red", "round", etc.). These methods, however, cannot be used to ground ordering concepts as they only deal with the problem of classifying an individual object into a discrete set of categories.

In a closely related study, the method described in [Sinapov et al., 2014b] relaxed the assumption that categories describe only individual objects and showed that through behavioral exploration a robot can learn pairwise object categories (e.g., drop push A is heavier than B). However, the robot stopped short of recognizing that a given series of objects is ordered by a given property. A rare example of a robotics study directly dealing with object ordering is described in [Schenck et al., 2012]. The robot in that study was able to solve the task of order completion (i.e., selecting the next object to complete a sequence) using both supervised and unsupervised approaches through the use of an objective function based on perceptual object similarity. In their work, the robot learned each ordering concept in isolation and did not explicitly ground those concepts (i.e., it could not explicitly recognize that a series of objects is ordered by weight, or height). In addition, the objects were specialized toys that varied only according to one dimension, as opposed to real-world objects.

Figure 2: The behaviors the robot used to explored the objects. From left to right and top to bottom: grasp, lift, lower, drop, press, and push. The arrows indicate the direction of motion of the end-effector for each behavior. In addition, the hold behavior (not shown) was performed after the lift behavior by simply holding the object in place for half a second.

3 Experimental Setup

3.1 Robot and Objects

The robot used in our experiment, shown in Figure 1, was a custom built mobile manipulator that uses the Segway Robotic Mobility Platform (RMP). The robot is equipped with a 6-DOF Kinova Mico Arm with a 2-fingered underactuated gripper as its end effector.

The set of objects O that the robot explored consisted of 32 common household items including cups, bottles, cans, and other containers, also shown in Figure 1. The objects varied according to their weight, height, and width. The objects' height and width was measured in millimeters while their weight was measured in grams. Some of the bottles were filled with water up to various levels. The objects were chosen such that the distributions of their weight, width, and height were roughly uniform.

3.2 Exploratory Behaviors and SensoryModalities

The robot explored the objects using 7 different behaviors: grasp, lift, hold, lower, drop, push, and press, shown in Figure 2. The behaviors were designed with the purpose of enabling the robot to obtain a behavior-grounded multi-modal object representation independent of any particular task. While some of the behaviors are likely to be irrelevant for any particular ordering task, we assume that this information is not known to the robot in advance. The behaviors were performed assuming a fixed object location on the table and were encoded using joint-space trajectories as well as timed Cartesian velocity commands (for lift, lower, push, and press).

The robot perceived the objects using the haptic (i.e., joint torques) and proprioceptive (i.e., joint angular positions) sensory modalities. During the execution of each behavior, the robot recorded the torque and position values for all 6 joints at 15 Hz resulting in two n ⇥ 6 matrices where n was the number of samples recorded. To reduce dimensionality, the temporal axis of each matrix was discretized into 10 equally spaced bins using the methodology of [Sinapov et al., 2014a]. Thus, after executing an action on an object, the robot extracted two sensorimotor feature vectors, xhaptic 2 R10⇥6 and xproprioception 2 R10⇥6 . In addition, at the end of the grasp behavior, the robot also recorded the end finger positions, i.e., xfingers 2 R2 . Each viable combination of a behavior (one of the 7) and modality (either haptic, proprioception, or fingers) constituted a sensorimotor context. Thus, the set of sensorimotor contexts, C, was of size 7⇥2+1 = 15 (the fingers modality was only available for the grasp behavior).

The robot performed each behavior on each object 5 different times, which took 7▷5 hours. Given context c 2 C and object i 2 O, the set X c i contained all 5 feature vectors observed with object i in context c. Following, we describe the learning framework which uses these observations to ground the three ordinal relations, weight, width, and height.

4 Learning Framework

4.1 Notation and ProblemFormulation

Let O be the full set of objects that the robot has explored. Let S = (OS↪GS) be an object series defined for objects OS ⇢ O using a directed cluster graph GS = (V↪ E). Each object in OS belongs to one of several clusters in V , which are connected using directed edges E. The set of edges E is constrained such that one cluster has just an outgoing edge (i.e., the start of the order), one cluster has only one incoming edge (i.e., the end of the order) while the rest have one of each. Figure 3.a) shows an example cluster graph with 5 clusters V and 10 objects OS.

Let L be the set of ordering concepts: in our case, ¶weight, width, height♦. The robot is tasked with solving the following two problems:

Order Recognition

Given an example object series S, for each concept l 2 L, the task of the robot is to learn a model Ml such that Ml(S) ! +1 if S is ordered according to l and -1 otherwise.

Object Insertion

Given a series S and object i, the task is to construct a new series S0 by inserting i at the correct position. In other words, S0 should be ordered according to the same concept l that was used to construct the order S.

To solve these problems, the robot undergoes three distinct stages described in detail below.

4.2 Interaction Stage

During this stage, the robot explores the objects in O by performing a series of exploratory behaviors on them and recording sensorimotor features capturing multiple sensory modalities. Let C be the set of sensorimotor contexts (defined in the previous section) where each context corresponds to a combination of an exploratory behavior and a sensory modality. For each context c 2 C and object i 2 O, let X c i be the set of sensorimotor features observed with object i in context c.

Let Rc 2 R♣O♣⇥♣O♣ be a matrix that specifies a pairwise object similarity relation in context c for each pair of objects, computed as follows. Given a sensorimotor feature vector xci 2 X c i detected with object i, let the function knn count(xci ↪ j↪ k) return the count of k nearest neighbors to xci detected with object j. Each entry rc ij in the matrix Rc was computed as:

In other words, each entry rc ij was set to the number of times a sensory signal xcj 2 X c j was one of the k nearest neighbors of a sensory signal xci 2 X c i . In our experiments, k was set to 25 but the results were similar for the range of 5 to 50 (k can be larger than the number of objects 32, as the matrix is computed using the raw observations, for which there are 5 per object). Following, we describe how this object representation is used by the robot to order objects in an unsupervised manner.

4.3 Unsupervised Order Discovery

In the second stage, the robot uses unsupervised learning to discover multiple possible ways to order the objects in O. To do so, for each sensory motor context c 2 C, the matrix Rc is used to fit an order Sc using the methodology described in [Kemp and Tenenbaum, 2008]. More specifically, given an input matrix Rc , the method in [Kemp and Tenenbaum, 2008] searches for the cluster graph that maximizes the posterior probability of the data given the cluster graph. This probability will be high if features in the data vary smoothly over the graph and low otherwise.

Figure 3 shows an example order fitting using a synthetically generated 10 ⇥ 10 matrix. Due to space limitations, we refer the reader to [Kemp and Tenenbaum, 2008] for further details. In our experiments, we used the source code provided with that publication with default prior likelihood parameters.

At the end of this stage, for each context c 2 C, let Sc be the order that maximizes Pr(Rc ♣Sc). In other words, at this stage the robot has discovered a set of object orders, one per sensorimotor context, which will subsequently be used to ground ordinal object relations.

4.4 Order Grounding Stage

In the third and final stage, the robot learns an order recognition model Ml for each ordering concept l 2 L in a supervised manner. Let Dl = (Sn↪ yn)Nn=1 be a set of examples such that each Sn is an object order and each yn 2 ¶-1↪ +1♦ is a class label indicating whether the objects are ordered according to l. To learn a classifier from such data, the robot represents each example according to its similarity to the orders discovered in the previous stage.

More specifically, given an example order Sn, let Scn be the order constructed by arranging the objects OSn according to the order Sc associated with context c. The series Scn is not necessarily a total order as two or more objects may belong to the same cluster node in the graph Gc. Let hS(i↪ j) 2 I be the relative difference in position between objects i and j in order S. For example, if i occupies one slot above j, then hS(i↪ j) = 1; on the other hand, if i and j occupy the same cluster node, then the value is 0. Given two orders, Sn and Scn containing the same set of objects OS, we can then define a distance function between them as:

where length(S) returns the number of clusters in the cluster graph GS of series S. Thus, a training example Sn can be represented by a feature vector xn 2 R♣C♣ such that each element xn↪c = d(Sn↪ Scn ). This feature vector encodes how similar Sn is to each of the orders that the robot discovered using unsupervised learning. Given this feature representation, for each dataset Dl = (Sn↪ yn)Nn=1 , the robot learns a classifier that can output an estimate for Pr(yn = +1♣xn).

The robot used the learned classifiers for order recognition to solve the object order insertion task as follows. Given an existing order S and an additional object i, let S 0 denote the set of orders attained by inserting i in all possible slots. The robot then inserts object i into the series S resulting in the object series S⇤ such that:

where x0 is the feature vector for the order S0 2 S 0 . In other words, the model picks the series that maximizes the output of one of the order recognition models Ml.

4.5 Evaluation

The framework is evaluated in a series of evaluation runs. During each run, for each object ordering concept l 2 L, a data set Dl = (Sn↪ yn)100 n=1 is constructed such that half the series are positive examples (i.e., ordered according to l) and the other half, negative. An object series was considered a positive example if and only if the change in the target attribute l from one position to the next exceeded a threshold ✓l (possibly set to 0.0). An object series was considered a negative example if the direction of the change along attribute l varied from position to position. The robot's order recognition models were then evaluated by performing 5-fold cross validation on Dl. The results were averaged over 100 evaluation runs with different random seeds used to generate the data sets Dl. The results are reported in terms of percentage recognition accuracy, i.e., # of correct series classifications # of total classifications ⇥100.

To evaluate the robot's ability to correctly insert an object into an existing order, for each ordering concept l 2 L, an additional set of object series D0 l was computed such that D0 l T Dl = ;. Each series in D0 l consisted of 5 objects. For each series, one object was randomly removed and the robot's model was then tasked to insert that object into the order consisting of the remaining 4 objects. Let ˆ p be the index of position for the remainder object chosen by the robot and let p be the index of the correct position for that object in the original series. The error can therefore be computed by taking the absolute difference, i.e., ♣ˆ p - p♣.

5 Results

5.1 Example Unsupervised Order Discovery

Figure 4 shows an example object order that was estimated by the robot during the unsupervised order discovery stage. In this case, the order was constructed using sensorimotor observations in the haptic sensory modality during the execution of the press behavior. Some of the objects are placed in the same position in the order, which could be due to perceptual noise (only 5 observations are available for each object) or because the differences in the height attribute for objects in the same cluster are too small to be detected by this particular behavior and sensory modality. The order reveals that this particular combination of a behavior and sensory modality induces an ordering based on the objects' height attribute. Similarly, other behaviors and modalities induced orderings that corresponded to the other two attributes considered in this study, width, and weight.

5.2 Object Order Recognition

Table 1 shows the accuracy of the robot's order recognition models using three different machine learning algorithms, kNearest Neighbor, Support Vector Machine with a quadratic kernel function, and C4.5 Decision Tree as implemented in the WEKA machine learning library [Witten and Frank, 2005]. For this test, the thresholds ✓l were set to 50 grams for weight, 15 mm for width, and 50 mm for height. In other words, an object series was considered to be ordered by height if each consecutive object was at least 50 mm taller than the previous one. All object series used for training and testing were of length 5 and were automatically generated by randomly sampling positive and negative examples from the set of all possible orders.

The C4.5 Decision Tree classifier achieves the highest accuracy and naturally performs feature selection. Since each feature is associated with a sensorimotor context, a learned tree identifies the relevant behaviors and sensory modalities for a given ordering task. For example, the tree learned for ordering objects by weight relied on features corresponding to three different behaviors: hold, lift, and lower. For approximately 75% of the test data points, the decision was made using only the top level node, corresponding to the hold behavior and the haptic sensory modality. On the other hand, the tree learned for ordering objects by width relied exclusively on the feature associated with the grasp behavior and the fingers position sensory signal. Therefore, once learned, the models could in principle be used by performing only the discovered relevant behaviors for a given ordering task instead of exhaustively performing the entire set of actions.

5.3 Sensitivity to Thresholds ✓l

In the next test, we evaluate the recognition performance with the Decision Tree classifier as the thresholds ✓l change from low to high. When these thresholds are high, series that constitute positive examples (i.e., ordered by the target property) tend to have larger differences in the target attribute from one position to the next in the series. Conversely, when these thresholds are low, the differences in the target attribute from one position to another in a given series are smaller and may even be undetectable by the specific behaviors and sensory modalities used in the experiment.

The result of this test are shown in Figure 5. As we expect, recognition accuracy is higher when the attribute increases by a larger value from one position to the next in an object series. This is largely due to perceptual limitations of the specific robot that was used in our experiments. Nevertheless, even when the threshold is set to 0 (i.e., any strictly monotonically increasing series is considered a positive example), the robot is still able to achieve recognition rates substantially better than chance. Recognizing series ordered by width turns out to be the hardest of the tasks as the only sensorimotor features relevant to this property were the end finger positions after executing the grasp behavior. The arm's fingers are underactuated and the readings are quite noisy which made this property particularly hard for the robot to learn.

5.4 Object Insertion

Finally, we evaluate the robot's ability to correctly insert an object into an existing series. Figure 6.a) shows an example insertion task in which the robot's model has to insert an object into an existing object series which is ordered by weight. The matrix visualizes the output of the 3 order recognition models when given the series constructed by inserting the object into each possible position. The output is maximized at position 3 (i.e., left of the last object in the series) for the weight ordering concept. In this example, the object in inserted into the correct slot.

For each of the three ordinal properties, the robot was tested on 500 insertion tasks. Figure 6 shows a histogram of the errors made by the robot when using the Decision Tree classifier. The difference between the correct position and the one chosen by the robot was either 1 or 0 in 88% of the tests for weight, 70% for width, and 73% for height. The task of the robot was particularly difficult as some of the example series in this test could be ordered by more than just one property due to chance. Nevertheless, this result shows that the robot can use its order recognition models to not only recognize the property that an object set is ordered by, but also to add additional objects to the existing series.

6 Conclusion and FutureWork

This paper proposed a novel framework that allows a robot to learn object ordering skills. The framework was evaluated in an experiment in which a robot learned to order common household objects using three different attributes: weight, width, and height. Based on findings in psychology and cognitive science, the framework described in this paper proposed that a robot can ground ordinal object relations in terms of object orderings discovered in an unsupervised manner. The results of our experiment showed that the framework allows a robot to learn to recognize whether a given series of objects is ordered according to a target property. In addition, the robot's recognition model was also used to successfully insert objects into an existing object series, a task that is common in human intelligence tests.

There are several direct lines for future work. A limitation of the current study is that once the objects were explored by the robot, the resulting tests were performed offline. In ongoing work, we are investigating methods that would enable the robot to efficiently explore novel objects and incrementally update the learned representations of ordinal object relations. More generally, as described in [Kemp and Tenenbaum, 2008], in addition to discrete categories (which are widely explored in robotics) and orders, object relations can induce a variety of other structural forms such as trees, rings, hierarchies, etc. Therefore, we are also pursuing a more general framework that would allow a robot to ground semantic concepts from language using multiple and heterogeneous structural forms. Finally, we are actively pursuing frameworks for grounding object-related knowledge through human-robot interaction scenarios such as the "I Spy" game described in [Thomason et al., 2016].

Acknowledgments

This work has taken place in the Learning Agents Research Group (LARG) at the Artificial Intelligence Laboratory, The University of Texas at Austin. LARG research is supported in part by grants from the National Science Foundation (CNS-1330072, CNS-1305287), ONR (21C184-01), AFRL (FA8750-14-1-0070), and AFOSR (FA9550-14-1-0087).