Publicação
Deep Learning for Multi-Animal Tracking
| Resumo: | In collective behaviour studies, the use of multi-animal tracking systems is extremely valuable. To be able to identify and track each individual in a group helps in the study and understanding of their behaviour in the collective. For this, researchers can use tracking systems, which can use sensors to detect the individuals; or they can use image-based tracking, with or without the need to mark the individuals. idtracker.ai is a state-of-the-art multi-animal image-based tracking system that uses convolutional neural networks to identify each of the individuals in a video. In videos with a higher density of individuals, idtracker.ai cannot extract enough frames of the single individuals (few frames ≈ 30) and the training of the identification network is slower. With the idea of decreasing this training time, here we propose to test three different machine learning methods. The first method is to use Transfer Learning models, with the expectation that the training can be done with few data. The second method is to use the ensemble method to join the results of various models of the idtracker.ai identification network, and thus decrease the variability of classification. Finally, the third method is to use not only multi-class labels but also pairwise-labels to increase the amount of information the network has available for training. The three methods are compared to the idtracker.ai model in terms of image classification accuracy and training time. Transfer learning and ensemble improved the accuracy of classification, but failed to reduce the time of training of the identification network. The pairwise method increased accuracy and time of training was comparable to the one of idtracker.ai. More specifically, by training the identification network with multi-class labeled images and pairwise-labeled images, the information the network can have from few images leads to an average classification accuracy of 94% (for 3000 images per class with 30 multi-labeled images per class). This is comparable to idtracker.ai trained with 3000 multi-labeled images (per class) - 98% accuracy, and is better than when idtracker.ai is trained with 30 multi-labeled images - 56% accuracy. |
|---|---|
| Autores principais: | Valente, Madalena de Oliveira Santos Lourenço |
| Assunto: | tracking baseado em imagens transfer learning aprendizagem em ensemble identificação de pares Teses de mestrado - 2021 |
| Ano: | 2021 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade de Lisboa |
| Idioma: | inglês |
| Origem: | Repositório da Universidade de Lisboa |
| Resumo: | In collective behaviour studies, the use of multi-animal tracking systems is extremely valuable. To be able to identify and track each individual in a group helps in the study and understanding of their behaviour in the collective. For this, researchers can use tracking systems, which can use sensors to detect the individuals; or they can use image-based tracking, with or without the need to mark the individuals. idtracker.ai is a state-of-the-art multi-animal image-based tracking system that uses convolutional neural networks to identify each of the individuals in a video. In videos with a higher density of individuals, idtracker.ai cannot extract enough frames of the single individuals (few frames ≈ 30) and the training of the identification network is slower. With the idea of decreasing this training time, here we propose to test three different machine learning methods. The first method is to use Transfer Learning models, with the expectation that the training can be done with few data. The second method is to use the ensemble method to join the results of various models of the idtracker.ai identification network, and thus decrease the variability of classification. Finally, the third method is to use not only multi-class labels but also pairwise-labels to increase the amount of information the network has available for training. The three methods are compared to the idtracker.ai model in terms of image classification accuracy and training time. Transfer learning and ensemble improved the accuracy of classification, but failed to reduce the time of training of the identification network. The pairwise method increased accuracy and time of training was comparable to the one of idtracker.ai. More specifically, by training the identification network with multi-class labeled images and pairwise-labeled images, the information the network can have from few images leads to an average classification accuracy of 94% (for 3000 images per class with 30 multi-labeled images per class). This is comparable to idtracker.ai trained with 3000 multi-labeled images (per class) - 98% accuracy, and is better than when idtracker.ai is trained with 30 multi-labeled images - 56% accuracy. |
|---|