Towards Symbiotic Creativity: A Methodological Approach to Compare Human and AI Robotic Dance Creations

1 Introduction

Dance is an area where the potential application of Artificial Intelligence (AI) has raised interest, and humanoid robots have been successfully used thanks to their human-like aspect. Several works studied and implemented systems for robotic dance creation [Joshi and Chakrabarty, 2021], such as fine-balanced robotic dance movements with a human performer, dance motion imitation techniques, or generative techniques through visual observation. Most of these works focused on the automation of various aspects of dance creation, while only few studies have proposed evaluation metrics for robotic dance. Recent research works also aim at understanding the key differences between human and machine creation processes for different artistic disciplines, including dance [Peng et al., 2021]. Overall, there is an emerging consensus about the need of exploring the relation between robotic dance creation and its aesthetic evaluation.

∗ allegra.defilippo@unibo.it

In this perspective, our research targets the differences between human and AI artistic creation, in particular focusing on the aesthetic evaluation of robotic dance performances. We propose a methodological approach by defining a series of steps in order to carry out the analysis in a balanced initial setting that allows to fairly compare both creation processes. Our methodological approach relies on three macro-phases: 1) a common setting definition, 2) choreography creation (either by AI or human), 3) a common evaluation phase.

We tested our methodological approach on a case study based on AI- and human-created choreographies for a NAO robot 1 , in collaboration with a group dancers from the International Higher Education Academy C&C 2 led by the choreographer Carlo Massari. Firstly, we defined a common setting of constraints and positions to be satisfied by both the AI algorithm and the human choreographer. Then we performed all the choreographies with a NAO robot. The evaluation of the performances was then conducted with the help of a human audience unaware of the choreography creation processes, and using a shared evaluation scheme.

Using these results, we then investigated the general preference of the audience (human rather than AI choreographies), the importance of the music genre on the evaluation, and, more broadly, which aspects of the choreography have the largest impact on the evaluation. We also provide guidelines to replicate our analysis in the same artistic domain with different settings, and we propose future directions to further analyze the interplay between AI and human dance creation.

From our perspective, this work allows to better identify strengths and weaknesses of both human and AI creation processes. This can trigger several research directions such as the exploration of new art forms for human dance creation and symbiotic creation processes where human and AI can collaborate to design new dance performances.

2 Background

Human Dance Creation Generally, when choreographers create a dance, they start from a particular idea or stimulus from which they generate the sequence of movements [Schiphorst et al., 1990]. Thus, each path is unique and based on 1) the choreographer's experience, 2) the initial idea, and

1 https://www.aldebaran.com/en/nao

2 https://www.ceccompany.org/?lang=it

3) the interaction with dancers and other experts, who inspire modifications to the initial idea, up to the creation of the final choreography. Indeed, the creative process is both interactive and iterative [Schiphorst et al., 1990]. [Ciolfi Felice et al., 2016; Singh et al., 2011] show that, despite the creative process of each choreographer being unique and changing over time, common choreographic aspects can be defined, representing ideas at different levels of abstraction, and combining them with a set of base operations, such as rearranging elements and establishing transitions.

Formally, systems such as the Laban [Von Laban, 1975] or Benesh [Benesh and Benesh, 1977] notation have been defined. These methods are used mostly by large dance companies who can afford a full-time annotator. Contemporary choreographers rarely use these systems, that are designed to document finished work and not for annotating the first stages of exploration. More often, the initial stage of the creative process is done in the choreographers' mind, where they can explore ideas, which are then refined after a first embodiment through the dancers. According to [Heyward, 2015], choreographers often capture intermediate phases of their work with intermediate artefacts, such as inspiring images, annotated sketches, and video clips of dance fragments.

Codified forms of dances composed by precise patterns of movement exist. This is the case of Noh dance [Wolz, 1975], where each dance is composed of highly detailed "kata" (patterns of movement) joined together to form a flowing sequence. The creative process, in these cases, is aimed at creating a choreography as a sequence of kata that is capable of giving artistic expressiveness and musical coherence to the entire performance. In this work, we mainly refer to this codified form of dance.

Computational Dance Automation In recent years, many researchers proposed methods to automate partial aspects of dance, from dance notation to choreography, and from dance capture to dance generation [Sagasti, 2019; Joshi and Chakrabarty, 2021].

Especially in dance, where physical movement is a key factor, the use of robots is continually expanding thanks to their humanoid shape. Many works have studied and implemented systems for robotic dances, ranging from 1) humanoid robots performing fine-balanced dance movements with a human performer [Ramos et al., 2015; Shinozaki et al., 2007; Shinozaki et al., 2008]; 2) experiments with robot motions that automatically coordinate to the music beat using a real-time music signal over which the humanoid robot has to dance autonomously [Grunberg et al., 2010]; 3) dance motion imitation techniques for humanoid robots aimed at generating dance movements adequate to the music rhythm through visual observation [Angulo et al., 2011]. Recent works [Liu et al., 2020; Wang, 2022] are focused on the representation of choreographies as a sequence of basic and simple positions for robotic dances, according to Noh dance structure.

All these works focused on the creation of robotic dances through human-robot interaction, without taking into account a human evaluation of these artistic creations.

Dance EvaluationApproaches Defining a criterion for objective evaluation of dance performances is a complex task.

In modern dance, well-known metrics for the evaluation of qualitative aspects of dance performances such as Aesthetic Competence Evaluation (ACE) [Chatfield and Byrnes, 1990] or Performance Competence Evaluation Measure (PCEM) [Krasnow and Chatfield, 2009] have been defined as a standard in this context. In particular, they consider aspects such as technique, space, phrasing and presence, time and energy, and a focus on the physical qualities of the performer, through three different levels of judgment for each area of evaluation.

Dancing robots are getting abler and abler at performing many kinds of robotic dances [Aucouturier et al., 2008] thanks to improvements in mechanics and control. In this perspective, based on the existing measures for evaluating human dance skills, many works have proposed evaluation metrics for robotic dance. [Oliveira et al., 2012] introduced a framework to evaluate robotic dance performances based on a Likert [Likert, 1932] questionnaire. With a major focus on robotic poses, [Manfr`e et al., 2017; Peng et al., 2019] discuss aesthetic evaluation processes of robotic dance poses, for improving choreography creation. [Saffiotti et al., 2020] focuses on the combination of human and robotic performers, by proposing a collaboration model evaluated using a Likert questionnaire to assess the harmony with the music, the harmony between the two performers, and the overall judgment. A more horizontal approach for merging the two evaluation perspectives (human and robotic) can be found in [De Filippo et al., 2022b], where the evaluation questionnaire is expanded with aspects such as the overall theatricality of the robotic dance choreography, the degree of human reproducibility, and the use of the surrounding space in the dance performance.

It must be noted that all these works focus on the aesthetic evaluation of dance performances but disregard the actual creation process which, on the contrary, is an important feature to be taken into account [Hong and Curran, 2019]. Conversely, in our work we analyze how a human or a computational creation process can differently affect the aesthetic evaluation of a robotic dance performance.

3 Methodological Approach

In this work, we propose a methodological approach to analyze the differences between human- and AI-based artistic creation, in particular focusing on the aesthetic evaluation of robotic dance performances. We define and formalize the steps needed to carry out the analysis in a thorough and technically sound manner, based on a balanced initial setting to equally compare both the creation processes. Then, we experiment our approach on a case study based on AI- and humancreated choreographies for a NAO robot. Finally, we compare the obtained choreographies based on a common evaluation phase to provide useful guidelines in the analysis of the interconnections between AI and human dance creation.

As illustrated in Figure 1, our methodological approach is based on three macro-phases: 1) common setting definition, 2) choreography creation based either on AI or human creation process, 3) common evaluation phase.

In details, we proceed with the following steps for experimenting our approach on a case study based on robotic dance choreographies:

1. Common setting definition:

• 1 a) Definition of a common set of basic positions inspired by Noh dance [Wolz, 1975].
• 1 b) Definition of a common set of constraints to be satisfied in terms of (e.g.,) initial and final position and number of mandatory positions to be executed during the dance performance; a choreography is represented as sequence of positions [Liu et al., 2020; Singh et al., 2011; Wolz, 1975].
• 1 c) Definition of a common set of music tracks with fixed time duration; tracks belong to a subset of different music genres, picked for being highly recognizable based on their specific characteristics [Sturm, 2012; Scaringella et al., 2006] (i.e., folk, electronic, classical, rock and latin).

2. Choreography creation:

• 2 a) Creation of 5 robotic choreographies with a basic AI technique taking into account music and position constraints as inputs.
• 2 b) Creation of 5 robotic choreographies by a human choreographer taking into account the same constraints as the AI algorithm.

3. Common evaluation phase:

• 3 a) Creation of an ad-hoc questionnaire based on the state of the art [De Filippo et al., 2022b; Oliveira et al., 2012];
• 3 b) Execution of a between-subjects experiment where the 10 choreographies (grouped as C AI and C H, all performed by a NAO robot) are evaluated by two different groups of people with both artistic and scientific knowledge.
• 3 c) Statistical analysis of the questionnaire results in order to answer our research questions.

3.1 Common Setting Definition

The problem description, given to both our AI algorithm and the human choreographer, states an initial and a final position. We also provide a repository of robot positions 3 , with all the mandatory and intermediate positions (e.g., sit position, stand position, etc.) already implemented for the NAO robot. In particular, we provide both the code and a detailed description and video demo for each position, in order to clarify them to the human choreographer. A focus on the choreography creation phase is depicted in Figure 2 by highlighting the common initial setting.

The shared constraints are the following:

3 https://github.com/ProjectsAI/ComparativeArtisticEvaluation/tree/main/codePositions

• each choreography must start with a specific initial position and must end in a specific final position;
• the total duration must be of exactly 2 minutes;
• each choreographymust contain at least eachmandatory position;
• intermediate positions can be used to move between mandatory positions;
• each choreography must contain at least 5 different intermediate positions;
• the sequence of positions must avoid possible incompatibilities between two consecutive positions;
• each choreography must be associated with one of the 5 different genre music tracks provided.

For each creation process (AI or human), the final goal is to generate 5 different choreographies (one per genre) satisfying the given constraints. Repetitions of positions is allowed if the minimum number of intermediate and mandatory positions is satisfied. All the choreographies are available in a public repository 4 . More details can be found in Section 3.2. In Figure 3 we show an example of two choreographies (represented as sequences of positions for the same music track), one created by our AI algorithm and one by a human choreographer, respecting the same constraints.

3.2 Choreography Creation

Human Creation A group of 30 dancers from the International Higher Education C&C Academy led by the choreographer Carlo Massari 5 worked on a 3-days creative workshop to prepare the robotic choreographies. The teams of choreographer and dancers were asked to create choreographies starting from the same input of our AI algorithm. The choreographer starts from the music track to plan a sequence of positions that satisfies the required constraints. This is a well-established approach in the artistic and theatrical dance scenario, as for the case of Noh dance. In more details, we organized the creation process in 3 macro-phases: 1) definition phase: understand the problem definition, 2) preparation phase: strategize the design choices, 3) execution phase: implement the solution. In phase 1, the choreographer and the dancers were provided with a pool of positions (video and textual description), the duration (in seconds) for each position on NAO robot, the constraints, and audio tracks. Then, in phase 2, the choreographer experimented with the entire sequence of positions in collaboration with the dancers, by analyzing different position patterns with the different music tracks and by classifying them based on different stylistic choices. Finally, for phase 3, they provided us with a written sequence of positions (per music track) that satisfies all the required constraints and to be codified and then performed by a NAO robot.

4 https://github.com/ProjectsAI/ComparativeArtisticEvaluation/tree/main

5 https://www.ceccompany.org/?lang=it

A significant difference with the typical creative process lies in the fact that we are bounding the dance performance through arbitrary and somewhat unnatural constraints; this contrasts with the usual approach of human choreographers, but it allows to define a common initial setting for a more balanced evaluation of both the (AI and human-created) performances.

AI Creation The creation process is structured into three macro-phases that mirror the human creation process. The algorithm begins with defining the problem, then considers factors like dance-music synchronization and position patterns (preparation), finally creates a position sequence for a NAO robot that satisfies the required constraints (execution).

Specifically, the algorithm is based on simulated annealing [Kirkpatrick et al., 1983]. For each move in the sequence, a neighborhood is defined and used to retrieve the next move according to a value function. The value function takes into account the aesthetic constraints defined in the preparation phase. During the preparation phase, design choices are carried out in order to account for constraint satisfaction, similarly to how a choreographer would make choices to respect constraints in human choreography. These choices involve parameters used in the value function, such as:

• α: the song's amplitude, which is used to establish different intervals associated with various song-related moments (e.g., an α value above a predetermined threshold denotes an intense or explosive chorus-related moment).
• cm: the class of a move, which is defined according to factors including the robot's ability to move fluidly and

the execution time of the movement.

• move match: a fitness function, which expresses how well does a given move m ∈ cm fits in a time frame characterized by a certain amplitude α. The higher the value, the more accurately the move is associated with the amplitude, with ideal matches represented by certain pairings (α, cm).
• time match: a second fitness function, which quantifies the effectiveness of a particular move based on the transition time from the previous move. Smaller transition times are associated with higher values.

Algorithm 1 Execution

1: procedure RUN SIMULATED ANNEALING 2: while optimal solution ̸= FOUND do 3: Expand neighborhood 4: Choose a move m from the neighbourhood 5: Evaluate the chosen move through v(cm, α) 6: end while 7: return optimal solution 8: end procedure

In the execution phase (Algorithm 1), simulated annealing is repeatedly run until all the constraints of the problem described in Section 3.1 and the aesthetic constraints mentioned earlier are satisfied. On the basis of the aforementioned parameters, the algorithm specifically seeks to optimize the value function v(cm, α), which is compute by averaging the values of the two fitness functions move match and time match. Simulated annealing, by definition, makes sure that the likelihood of accepting moves that are worse for the optimization process (i.e., moves with a low score) reduces with each iteration. The optimal solution is the sequence of moves for which all the constraints of the problem described in Section 3.1 and the aesthetic constraints mentioned earlier are satisfied. Even though we opted for such a solution, the execution phase can be thought of as a black-box procedure which could be replaced by any other algorithm, provided that the same constraints are respected.

3.3 Common Evaluation

Evaluation Questionnaire The evaluation phase is conducted to investigate the reactions and perceptions of the audience of the robotic performances created. The methodological tools used for data collection are participant observation and questionnaires. Based on [De Filippo et al., 2022b] we define a survey 6 to evaluate the robotic choreographies generated by the algorithm and the choreographer. We propose two different questionnaires, i.e., one per creation process (AI or Human). Each questionnaire is composed by 5 pages (one per different choreography music), and each page is composed by a video demo of the robotic choreography and a list of questions, one for each evaluation target. Each participant anonymously vote the proposed choreographies, providing a score for all targets on a Likert scale (from 1 to 5). The targets are: (1) Storytelling; (2) Rhythm; (3) Movement Technique; (4)

6 https://forms.gle/BkPF2mpjX3QGyjfL8

Public Involvement; (5) Space Use; (6) Human Characterization; (7) Human Reproducibility. For each target, we propose a specific question to the user.

Results Collection and Audience We conduct a betweensubjects experiment for the evaluation phase; in this kind of design each user is exposed to a single subset of choreographies (AI generation vs human generation). This allows to evaluate the differences between both subsets of choreographies without any conditioning in the evaluation phase: the audience is not affected by any notion about the experimental setting [Charness et al., 2012]. We administered a questionnaire pertaining to the choreographies to different groups of subjects. To avoid decision fatigue [Pignatiello et al., 2020], questions order is randomized in each questionnaire. The audience has been selected to be equally distributed among users with scientific, artistic or both backgrounds. We collected the questionnaire responses and then analyzed them, starting with the identification of the emerging choreography features in relation with the evaluation targets.

Dataset Construction For each choreography in input, we extract 15 features. We collect them according to the evaluation targets, relying on state-of-the-art analysis [De Filippo et al., 2022b]. For each choreography, the following information are stored 7 : 1) nDifferentMovements: the number of different movements of the choreography; 2) nTotalMovements: the number of total movements (with optional repetitions) of the choreography; 3) movementDifficulty: the degree of difficulty of the moves; the allowed levels (i.e., low, medium, high) are mapped to the interval md ∈ [1, 3]; 4) acrobaticMovements: the level of acrobatic movements, am ∈ [1, 3]; 5) movementsRepetition (mr): the level of movement repetitions, mr ∈ [1, 3]; 6) humanMovements (h): the level of human movement presence, h ∈ [1, 3]; 7) balance (b): the level of balance movements, b ∈ [1, 3]; 8) speed (s): the degree of movement speed, s ∈ [1, 3]; 9) bodyPartsCombination (bc): the level of combinations involving different body parts, bc ∈ [1, 3]; 10) musicBPM (bpm): the number of Beats Per Minute; 11) headMovement (hm): the level of combinations involving the head, hm ∈ [1, 3]; 12) armsMovement (arm): the level of combinations involving the arms, arm ∈ [1, 3]; 13) handsMovement (hdm): the level of hands movement presence, hdm ∈ [1, 3]; 14) legsMovement (lm): the level of legs movement presence, lm ∈ [1, 3]; 15) feetMovement (fm): the level of feet movement presence, fm ∈ [1, 3].

The 7 evaluation targets follow the survey questions (see Section 3.3).

4 Experimental Analysis

For our experimental part, we examined user scores through three different analyses, to answer to our research questions defined in Section 3. First, we investigated score preferences for different artistic creation strategies, focusing on specific evaluation targets. Second, we examined score preferences for different music genres, again focusing on specific targets. Finally, we delved into the analysis of the choreography features that mostly influenced the audience's evaluation.

7 https://github.com/ProjectsAI/ComparativeArtisticEvaluation/tree/main/datasets

Our research questions are the following:

• RQ1: Does the audience prefer AI-created choreographies or Human-created ones?
• RQ2: Does the music genre influence the evaluation of the audience?
• RQ3: Which choreography features have the largest impact on the evaluation targets?

4.1 Experimental Setting

The audience is composed by 68 participants with a scientific and/or artistic background. Each user successfully completed only one questionnaire (AI or Human). Participants interacted with an anonymous questionnaire, as explained in Section 3.3. We conduct a between-subjects experiment with two groups composed by 34 participants. Each group voted on 5 choreographies for a total of 340 observations (considering both AI- and human-generated choreographies). In this setting, each user is exposed either to only AI-created choreographies or to only human-generated ones.

For our experimental phase, we define independent variables in order to observe and measure their effects on our dependent variables. In details, our independent variables are 1) the creation process of the choreography and 2) the music genre; the dependent variables are the evaluation targets that we want to measure, as reported in Section 3.3. To address RQ1, we compared ,for each evaluation target, the average ratings received for AI-created choreographies to the average ratings received for Human-created choreographies. To address RQ2, we further split the data based on the music genre accompanying the choreography. Finally, as regards RQ3, we used linear regression to identify the features (see Section 3.3) with the strongest influence on the evaluation target.

4.2 AI and Human Creation Process Comparison

First, we compared the average ratings received for AIcreated choreographies to the average ratings received for human-created choreographies.

We used a two-sided Mann-Whitney U rank test [Rosner and Grove, 1999] on the two independent samples of collected scores, since the requirement of normal distribution for the t-test is not met for these samples. The test results showed that the evaluation targets for which we have a statistically significant preference in term of scores are Rhythm (p = 0.008), Space (p = 0.002), and Human Reproducibility (p = 0.004). As for the Storytelling, we can observe a gap that is not statistically significant (with p = 0.184), but that deserves to be further analyzed with (e.g.,) a larger sample of observations.

This suggested that: (1) participants significantly preferred Human-created choreography in terms of rhythmic coherence with the music; (2) participants significantly preferred AIcreated choreography based on the use of space - the space is perceived to be better used in AI choreographies. This might be explained by the fact that human choreographers are used to work with real environments (the actual stage with its confined spaces, the position of the dancers and audience, etc) and thus might somewhat project these usual, implicit constraints into their choreographies, underutilizing the "artificial" space where the robot is dancing. (3) As for the Human Reproducibility target, we asked if the choreography can be easily reproduced by a human performer, and the results showed that evaluators ranked Human-created choreographies as more reproducible.

4.3 Music Genre Comparison

To address RQ2, we further split the data based on the music genre. Previous test revealed that there is statistically significant correlation between rhythmic coherence and music. We used again a two-sided Mann-Whitney U rank test on the two independent samples of collected score. All statistically significant values are highlighted with (*).

The test results (see Table 1) showed that folk is the music genre with a statistically significant preference in term of scores for almost all the targets, with significant values (p < 0.05) for Rhythm, Space, and Human Reproducibility. For this music genre, the results are in line with the trends emerged during the first experiment (Section 4.2), with a statistically significant gap for C AI related to the use of Space, and for C H related to Rhythm and Human Reproducibility, while the inversion of the results trend can be seen only for the Storytelling target that shows a significant preference for C AI.

As for the electronic genre, a significant preference (for C H) only emerged for the Human Reproducibility target, again similarly to the results of the first experiment.

As for the Latin and Rock genres, the Storytelling target shows again a significant preference for C H in line with the first experiment. This holds for the targets describing the coherence with the rhythm and the use of space as well.

Finally, for the classical genre, the significant preference (for C AI) only emerged for the use of Space target, exhibiting again the trend of the first experiment.

Moreover, this second set of experiments confirms that, in general, similar score values can be observed for two targets: Movements (related to the technique and fluidity of movements) and Public (related to the public involvement). Average preferences connected to these targets are similar for C AI and C H regardless of music genre. This can be explained by the common initial setting that limits both the movements choice and the degree of creativity allowed for public involvement.

4.4 Feature Choreography Analysis

Finally, we tackle RQ3 and we employ a linear regression model to identify the choreography features with the largest influence on the evaluation targets. We consider the set of features listed in Section 3.3 which were extracted for each choreography. We split this information based on C AI and C H in the following tables and we show only statistically significance values (p < 0.05) for at least one target.

In Table 2 and Table 3 we highlight statistically significant correlations (both positive and negative) between the features and the different evaluation targets. Many interesting correlations can be found: (1) we observe that the level of movements difficulty is negatively correlated to the evaluation targets both for C AI and C H; (2) accordingly, the total number of movements in the choreography is positively correlated with the evaluation targets; (3) music BPM is positively correlated with the evaluation targets (only) for C H.

Figure 5 shows the features with a major impact on all the evaluation targets based on the creation process (i.e., AI or human), by helping to define some common guidelines.

5 Discussion and Conclusions

In this work, we propose a methodological approach to analyze the differences between human- and AI-based artistic creation, in particular focusing on the aesthetic evaluation of robotic dance performances. We define a series of steps to carry out the analysis in a balanced initial setting that allows to fairly compare both creation processes. Our approach is based on three macro-phases: (1) definition of a common setting, (2) choreography creation based either on AI or human creation process, (3) common choreography evaluation. Related to the emerging trends of the final evaluation phase, we suggest future interdisciplinary research directions.

Firstly, related to RQ1 (Does the audience prefer AIcreated choreographies or Human-created ones?), we plan to analyze a further step in the experimental part, by providing the parallel execution of both the AI- and Humanmade choreography to the audience. The idea is to propose a within-subject analysis to a different group of participants, in order to understand if these results are confirmed also by users that are exposed to both the choreographies at the same time. We plan to make two versions of this experiment by proposing either only human or robot performers.

Related to RQ2 (Does the music genre influence the evaluation of the audience?), we have observed that the target associated to the rhythm showed a statistically significant gap in most of the experiments, so we plan to implement new choreographies with different music genres and also with different audio tracks of the same music genre, to further validate our results. Moreover, w.r.t. the feature associated to the number of different movements, we observed its positive correlation with most of the evaluation targets (RQ3). This result suggests that choreographies with a greater variety of different positions are preferred in general, but also that a longer duration of the choreography could be preferred as well ([De Filippo et al., 2022b]). In this direction, we plan to provide different common settings (e.g., time duration, rhythm, movements) for further experiments with human choreographers.

Finally, the idea is to provide more useful indications to both human choreographers and AI algorithms to create robotic choreographies based on positive/negative correlations between choreography features and evaluation targets. This can trigger the exploration of new art forms for human dance creation and symbiotic creation processes where human and AI can collaborate to design new dance performances. Moreover, this work can also provide a useful starting benchmark [De Filippo et al., 2022a] for training Machine Learning models to predict the relationship between the input features and the predicted target for new choreographies.

Acknowledgements

We thank Matteo Casari, Carlo Massari C&C Company, and the performing robots group. This work is partially supported by PNRR - M4C2 - (PE00000013) - "FAIR - Future Artificial Intelligence Research" Spoke 8 "Pervasive AI", funded by the European Commission under the NextGeneration EU programme.