Publication
Exploring Human Action Recognition for Rehabilitation Game Application
| Summary: | Through computer vision algorithms motion can be computed, which can be a crucial element to be integrated with serious game environments. To evaluate the efficacy of motion detection algorithms, the information of these algorithms can be used to perform Human Action Recognition (HAR). There are several algorithms to perform HAR, although skeleton approaches can be seen as the best way to isolate human motion. To extract the human skeleton representation, the work described in this paper evaluates three distinct methods: OpenPose (2D), YOLO-Pose (2D) and BlazePose (3D). The information translated by the skeleton representations is normalized by lightweight normalization algorithms (for further real-time application). To classify the video sequence and further action identification, a Long Short Term Memory network (LSTM), was used. Using the N-UCLA dataset, the highest F1 score of 0.745 was achieved using OpenPose skeleton extraction (2D), followed by the computation of the angles in each joint, demonstrating that the OpenPose skeleton representation can be the most viable solution for computing human motion in serious games. |
|---|---|
| Main Authors: | Lopes, Júlio Castro |
| Other Authors: | Van-Deste, Isaac; Lopes, Rui Pedro |
| Subject: | Human Action Recognition Skeleton extraction OpenPose YOLO-Pose BlazePose Moving Joints Descriptor |
| Year: | 2024 |
| Country: | Portugal |
| Document type: | conference paper |
| Access type: | open access |
| Associated institution: | Instituto Politécnico de Bragança |
| Language: | English |
| Origin: | Biblioteca Digital do IPB |
| Summary: | Through computer vision algorithms motion can be computed, which can be a crucial element to be integrated with serious game environments. To evaluate the efficacy of motion detection algorithms, the information of these algorithms can be used to perform Human Action Recognition (HAR). There are several algorithms to perform HAR, although skeleton approaches can be seen as the best way to isolate human motion. To extract the human skeleton representation, the work described in this paper evaluates three distinct methods: OpenPose (2D), YOLO-Pose (2D) and BlazePose (3D). The information translated by the skeleton representations is normalized by lightweight normalization algorithms (for further real-time application). To classify the video sequence and further action identification, a Long Short Term Memory network (LSTM), was used. Using the N-UCLA dataset, the highest F1 score of 0.745 was achieved using OpenPose skeleton extraction (2D), followed by the computation of the angles in each joint, demonstrating that the OpenPose skeleton representation can be the most viable solution for computing human motion in serious games. |
|---|