Publicação
Development of vision algorithms for supply tasks by mobile manipulators
| Resumo: | Currently, companies are increasingly seeking to automate and track industrial processes. The MIAR project, in which this dissertation is integrated, aims to automate the process of delivering to and collecting boxes with Targets from the production lines. One of the fundamental elements of automating any process is the collection of information about the objects of interest in the process. Thus, the purpose of this dissertation was the development of a vision algorithm capable of detecting, recognizing and estimating the 6D pose of two types of boxes. This vision algorithm would be applied in dynamic environments and for real-time applications. In order to achieve these capabilities, the research first conducted a review of the state of the art for the mentioned tasks, with the intention of identifying the best approach. To accomplish the three tasks, YOLOv8x-pose was selected for its potential benefits in adopting a unified approach that combines object detection, recognition, and keypoints prediction. The predicted keypoints can then be utilized in a keypoint-based 6D pose estimation method. For this, several trains were made along with improvements in the training process to achieve the best YOLOv8x-pose’s performance possible. The improvements involved a modification in the keypoint loss function to enhance the accuracy and precision in the prediction of the keypoint’s location and applying the hyperparameter tuning process to utilize the best hyperparameters in the training phase of the YOLO model. With this, it was also possible to verify the positive impact hyperparameter tuning provides to the training process. Regarding the 6D pose estimation task, a post-process of the data obtained by the YOLOv8x-pose was developed, where algorithms, along with other calculations to estimate the position and orientation of the boxes, were applied. The last step focused on the implementation of ROS 2 to send the information related to the boxes visible in the image such as their type, their position, and orientation. As a result, the approach developed achieved good accuracy and precision in detection, recognition, and 6D pose estimation. |
|---|---|
| Autores principais: | Araújo, João Pedro de Sousa |
| Assunto: | 6D pose estimation Keypoint Object detection and recognition Performance YOLO Model Desempenho Deteção e reconhecimento de objetos Estimação de pose 6D Modelo YOLO Ponto-chave Engenharia e Tecnologia::Engenharia Mecânica |
| Ano: | 2025 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade do Minho |
| Idioma: | inglês |
| Origem: | RepositóriUM - Universidade do Minho |
| Resumo: | Currently, companies are increasingly seeking to automate and track industrial processes. The MIAR project, in which this dissertation is integrated, aims to automate the process of delivering to and collecting boxes with Targets from the production lines. One of the fundamental elements of automating any process is the collection of information about the objects of interest in the process. Thus, the purpose of this dissertation was the development of a vision algorithm capable of detecting, recognizing and estimating the 6D pose of two types of boxes. This vision algorithm would be applied in dynamic environments and for real-time applications. In order to achieve these capabilities, the research first conducted a review of the state of the art for the mentioned tasks, with the intention of identifying the best approach. To accomplish the three tasks, YOLOv8x-pose was selected for its potential benefits in adopting a unified approach that combines object detection, recognition, and keypoints prediction. The predicted keypoints can then be utilized in a keypoint-based 6D pose estimation method. For this, several trains were made along with improvements in the training process to achieve the best YOLOv8x-pose’s performance possible. The improvements involved a modification in the keypoint loss function to enhance the accuracy and precision in the prediction of the keypoint’s location and applying the hyperparameter tuning process to utilize the best hyperparameters in the training phase of the YOLO model. With this, it was also possible to verify the positive impact hyperparameter tuning provides to the training process. Regarding the 6D pose estimation task, a post-process of the data obtained by the YOLOv8x-pose was developed, where algorithms, along with other calculations to estimate the position and orientation of the boxes, were applied. The last step focused on the implementation of ROS 2 to send the information related to the boxes visible in the image such as their type, their position, and orientation. As a result, the approach developed achieved good accuracy and precision in detection, recognition, and 6D pose estimation. |
|---|