Method for Calculating 3D Coordinates in Pose Estimation of a Stick Held by a Hand

Monocular human 3D pose estimation has become more practical with deep neural network (DNN) technology. One of the next challenges is pose estimation including stick held in the hand. In this study, we propose a method to compute the coordinates of the stick tip using a human 3D pose estimation network that measures the position where the stick is held in the hand. In particular, when the stick is orthogonal to the camera's line-of-sight to the tip point, the coordinates of the measurement point may be imaginary, and we present a solution using the method of Lagrange multiplier. Experiments confirmed the ability of the proposed method to modify the solution to a real number when it would result in an imaginary solution. The proposed method is expected to be applied to monocular motion capture of humans holding various stick-shaped objects, for example, in sword fighting techniques and baseball batting.