Mirror-mirror on the screen am I the most aligned than I have ever been?

Recent progress in motion sensing, combined with the advanced visualization, augmented reality technologies and related movement computing research, open a great range of opportunities in realtime embodied learning applied to motion domains such as dance, sports, rehabilitation, fitness and well-being. In particular, low-end devices such as Kinect, have been used recently in a variety of domains that extend the paradigm of Augmented Mirror for dance self-training. In this paper we discuss the advantages and disadvantages of these paradigms and settings based on literature research, our previous work in WhoLoDancE project and reflection through an ongoing design process and prototyping of learning experiences related to dance. We focus on identified challenges through a user-centered and interdisciplinary lens with the belief that focusing on particular aspects of movement, guided by the practice itself can lead to more meaningful experiences for self-training. How to cite this book chapter: El Raheb, K., Stergiou, M., Katifori, A. and Ioannidis. Y. 2020. Mirror-mirror on the screen am I the most aligned than I have ever been? In: Loizides, F., Winckler, M., Chatterjee, U., Abdelnour-Nocera, J. and Parmaxi, A. (eds.) Human Computer Interaction and Emerging Technologies: Adjunct Proceedings from the INTERACT 2019 Workshops. Pp. 355–360. Cardiff: Cardiff University Press. DOI: https://doi. org/10.18573/book3.at. License: CC-BY 4.0. 356 Human Computer Interaction and Emerging Technologies


Keywords
Technology Enhanced Learning · Dance · Movement Analysis and Interaction · Human Computer Interaction · Embodied Learning 1 Making the best out of Augmented Mirror for Dance Learning In this work, we focus on the advantages and limitations of the augmented mirror setting, using depth cameras such as Kinect [ 1, 8-12, 18, 21] or regular cameras [13,19]. In addition, although the goal is different, dance learning experiences design can benefit from existing guidelines in movement based games [14]. As we describe in [4,6], designing an effective system for dance or embodied training poses a number of HCI and computational challenges. These include identifying the ideal devices and strategies for capturing movement and processing real-time data to provide appropriate feedback. We argue that there is not one solution to fit all and that the possible answers are highly related not only to the devices used but also to the movement domain, dance genre and particular learning objectives.
Comparing to a regular mirror, the setting of the Augmented Mirror presents several advantages, such as providing feedback on what can be enhanced in terms of technique or body posture, as if seen from a different spatial perspective [10,21] or different time [13]. Similar solutions have been applied for capturing and evaluating effectively the posture of piano performers [16]. In addition, the Augmented Mirror set is simple, low cost, and the mover does not have to wear any special devices. One of the risks however, is that the mover can become more focused on the screen rather than on the embodied experience [14] -A critique that is also valid when using a physical mirror in dance practice. In this work we examine the main characteristics that contribute to the optimum efficiency of the Augmented Mirror for dance learning.

Conceptual Frameworks and ontologies
The three year EU funded project WhoLoDancE, engaged a group of experts, representatives of four dance genres (Ballet, Contemporary, Greek Folk, Flamenco) [17], in co-design sessions. The question of what to measure and how for evaluating the learners performance was persistent in the design process. As a result, we proposed a conceptual framework [2,7] that focuses on different Movement Principles, i.e. aspects such as symmetry, balance, alignment, that a student might need to focus on independently of the dance genre. Camurri [3], present the different levels of features and categorises them based on how much processing or complexity they need in comparison to raw data from different sensors. This categorisation not only suggests that some aspects are harder to compute (e.g., qualitative characteristics, vs. posture or velocity), but also that not all devices are appropriate for capturing some of these features in the first place. For example, optical motion capture, depth and direct cameras cannot directly measure the pressure on the floor, and therefore evaluate effectively weight transfer on the feet. We argue that conceptual frameworks as well as ontologies about the devices [23], and/or domain knowledge of the application dance genre, can effectively guide the design of augmented mirror experiences for learning through expressing categories and rules related to movement performance and structure.

Measuring technique vs. comparing with expert dancers
The augmented mirror paradigm using Kinect for evaluating students performance can be used in dance in two ways: one is to compare the overall performance and closeness of positions and motions in relation to a stored ideal performance [1,9]. The other is to define particular rules and patterns focusing on specific aspects e.g, calculate the posture deviation through defining e.g torso misalignment or rotation to the pelvis rotation and compare with the ideal range [10,21]. Although most of the systems that use the first approach provide specific feedback on body parts, this might not be very accurate due to differences between human bodies, and learning objectives that are aligned with the dance system of teaching and practicing. Nevertheless, creating a repository of movement is expensive in cost and time and poses the constraint of capturing students, and teachers' movement with the same precision. On the other hand, one can still be correct in terms of relations and proportions according to what the technique suggests,being within this correct range even if they adopt this correctness for their own body shape and abilities. This approach might be more appropriate as body analogies can differ. Each body is different and it should be compared with its own ideal posture, not with somebody else, especially if the low end device does not allow for such precision in motion capture.

Mapping of movement practice with limitations of the set-up and hardware
Not surprisingly, most of the aforementioned efforts, target ballet [10-13, 18, 21-22,], a dance genre that requires precision of the shape and posture of the body and has a specific movement vocabulary and terminology suggesting clear known positions and transitions, and rules. It is also traditionally taught in front of the mirror. In addition, conceptual frameworks and ontologies of the movement genre as the one we have developed in our previous work [5] is extended to categorise parts of the syllabus that can benefit of similar exercises.
For example, a ballet dancer can still be performing a good developpé (slow extension of the leg) and be correct, having the spine vertical, and the pelvis aligned, even if they are not still able to extend as high as a professional dancer. In addition the posture might still be correct in terms of technique even if the mover chooses a different posture for the arms or even different directions for the leg extension.

Feedback: Focus on one aspect at a time
While early attempts use the method of alignment of the positions and motion and evaluate accuracy overall [1,9], recent research has shown that evaluating the overall similarity compared to a teachers or professionals standard might have several implications that relate to both the evaluation and comparison itself, as well as to the provision of effective feedback [20,22]. This approach, allows the user to focus on a particular aspect, without cognitive overload and frustration, focus research on particular means of feedback, and overcome the limitations of technology. With the appropriate mapping we can turn the limitations of a technology into an advantage [14]. Trajkova [22] in her evaluation on particular feedback (visual, verbal, emojis) involving 16 novices and 16 advanced ballet students, concludes that providing particular feedback on aspects e.g., focus either on one aspect of movement alignment or one body part is much more effective. Taking into account basic usability principles [15], it is important for the mover to understand what the system measures and what to improve. Knudsen [10] presents an effective system focusing on one dance genre, ballet, one exercise and one objective of learning and self-improvement, in this case alignment providing audio-visual feedback.

Conclusion
Evaluating one's movement in dance using low-end devices is a challenging task. The skilled dancer focuses on so many aspects of the shape and quality of the movement simultaneously, without thinking. Nevertheless, the limitation of not evaluating all aspects at once can become a strength from an educational perspective, especially for beginners and amateurs. Building on the idea of less-is-more and informing the design by the concepts and rules of the dance technique, low-end devices and the paradigm of the augmented mirror can create effective scenarios of learning applications. In this paper based on a) a literature survey of the relevant research that use the augmented mirror paradigm, b) the reflection on the users needs that emerged throughout the WhoLoDancE project and the development of conceptual framework, we summarize some best practices for designing and developing such applications. Currently our application, integrates a variety of modes for practicing alignment, directionality, and other aspects related to dance exercises providing feedback both in abstract manner and through score.