Applying Machine Learning to Assess Musical Development in Early Childhood Using Eye and Body Movement Data

Musical expression in early childhood includes a lot of elements of body movement. The author has conducted movement analysis using the MVN system as 3D motion capture to quantify the body movement in musical expression in early childhood from 2015year.

Recently, as well as body movement, eye movements are considered to interact with the external environment. Tobii eye tracking system was utilized to attempt to evaluate measure and score responses of children to music experience.

In this study, the author tried to apply machine learning to asses musical development in early childhood using eye and body movement data based on my previous studies. To carry out the feasibility of this study, firstly, effective feature quantities were extracted from the results of the quantitative analysis regarding body movement in musical expression in early childhood organized for the past four years. As a result, specifically, the movement of the right hand was characteristic, and a statistically significant difference was observed in the data of the right hand regarding the moving distance, the moving average velocity, the moving average acceleration and the moving smoothness compared to other measurement data. Secondly, based on eye-tracking data collected over four years in my study of early childhood children singing, the author conducted a simultaneous analysis of both eye movement and body movement in musical expression to acquire quantitative data in 2022 and 2023. Visual information is important to stabilize posture in humans as well as express body movements. Some recent studies assessed stable postural control situations with eye tracking but little research was reported to focus on music-induced movements, especially for early childhood. Thirdly, the feature quantities were extracted from the data for two years both eye movement and body movement in musical expression by simultaneous analysis, and were implemented into machine learning using several classifiers such as MLP(NN) and SVM. The author compared the discrimination accuracy between using feature quantities of both eye and body movement as a result of simultaneous analysis and using feature quantities of only body movement.

As a result, the discrimination accuracy using feature quantities of both eye and body movement by simultaneous analysis was higher than using feature quantities of only body movement. Specifically, the discrimination accuracy using MLP(NN) was higher than other several classifiers.

In this way, the author designed a methodology to include eye movement data such as gaze fixations and saccadic movements in coordinated simultaneous body motion captured kinetics data. It was verified that the author has progressed more appropriate method of machine learning using effective feature quantities based on the result of simultaneous analysis of both eye movement and body movement in musical expression.