Publicação
Skeleton-based action recognition in industrial settings
| Resumo: | Nowadays, with the evolution of industry, collaborative robots emerged, which are smaller, safer and able to share the workspace with human operators, however the human-robot interaction is limited. This dissertation proposes an easily integrated system in a collaborative robot capable of making the robot perceive, with low latency, which actions the human is performing. The implementation of such a system considers a vision system, which is oriented towards the human, powered with a deep learning model capable of accurately identifying, in the shortest possible time, the action performed by the human. In this dissertation a Kinect v2 camera from Microsoft was used, coupled with Openpose, which provides an estimation of the human skeleton pose in real-time. The use of a system capable of extracting information from the human skeleton in real time in conjunction with the deep learning algorithm, allows the extracting of high-level features of human movement. This discriminative informations allows a highly accurate classification, robust throughout a short period of time. Recently, human pose classification has been the subject of research and application. However, data availability is scarce, especially in industrial environment. Therefore, in the context of this proposal, it was necessary to create a dataset based on a work-cell to train the neural network. The main contributions of this dissertation are: a comprehensive list of several state-of-the-art methods for the skeleton-based human action recognition problem for offline and online detection, a dataset settled on a work-cell for action recognition in industrial environment, a 3D skeleton viewer for better understanding of the skeleton motion, and a neural network trained for action recognition that classifies the action in less than 200 milliseconds with high accuracy. |
|---|---|
| Autores principais: | Fernandes, Filipe Barbosa |
| Assunto: | Skeleton-based Action recognition Collaborative robotics Industrial environment Esqueleto humano Reconhecimento de ações Robótica colaborativa Ambiente industrial |
| Ano: | 2022 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade do Minho |
| Idioma: | inglês |
| Origem: | RepositóriUM - Universidade do Minho |
| Resumo: | Nowadays, with the evolution of industry, collaborative robots emerged, which are smaller, safer and able to share the workspace with human operators, however the human-robot interaction is limited. This dissertation proposes an easily integrated system in a collaborative robot capable of making the robot perceive, with low latency, which actions the human is performing. The implementation of such a system considers a vision system, which is oriented towards the human, powered with a deep learning model capable of accurately identifying, in the shortest possible time, the action performed by the human. In this dissertation a Kinect v2 camera from Microsoft was used, coupled with Openpose, which provides an estimation of the human skeleton pose in real-time. The use of a system capable of extracting information from the human skeleton in real time in conjunction with the deep learning algorithm, allows the extracting of high-level features of human movement. This discriminative informations allows a highly accurate classification, robust throughout a short period of time. Recently, human pose classification has been the subject of research and application. However, data availability is scarce, especially in industrial environment. Therefore, in the context of this proposal, it was necessary to create a dataset based on a work-cell to train the neural network. The main contributions of this dissertation are: a comprehensive list of several state-of-the-art methods for the skeleton-based human action recognition problem for offline and online detection, a dataset settled on a work-cell for action recognition in industrial environment, a 3D skeleton viewer for better understanding of the skeleton motion, and a neural network trained for action recognition that classifies the action in less than 200 milliseconds with high accuracy. |
|---|