Publicação
Multimodal perception for robotics in forestry operations
| Resumo: | Forest fires constitute not only an environmental problem with natural resources destruction and ecological degradation, but also a societal problem where human lives can be at stake. The integration of robotic solutions and advanced perception systems can facilitate the optimal management of forest resources, enhance the productivity of forestry tasks, and increase the profitability of forest stands. Nevertheless, perception at stand-level in forests is a domain with a great potential to grow, being underdeveloped compared to airborne or spaceborne remote sensing. This thesis was developed within this scope with the goal of achieving a multimodal and multipurpose perception system for robotics in forestry operations at standlevel. A multipurpose and multimodal system called ForestMP (Figure 1) was designed to handle multiple sensors of different natures and fuse the information into meaningful perceptions that can be used in robotic forestry operations. The system at hand can intervene at least in four distinct forestry operations: forest soil preparation before plantation (soil ripping and fertilisation), forest environment monitoring (tree trunk monitoring and mapping), forest species growth control (eucalyptus selective thinning), and forest areas maintenance (line vegetation clearing). During the soil ripping and fertilisation operation, the system maps the ripping depth while it applies fertiliser at a variable rate based on a prescription map. The operation of forest tree monitoring consists in detecting and mapping the trees using multispectral imagery. With respect to selective thinning, the system runs an algorithm capable of automatically selecting eucalyptus stems to keep based on their robustness and uprightness. Regarding line vegetation clearing, the system is able to identify objects to clean or not based on an OAK-D device, and subsequently control a forest clearing implement to perform the operation. Furthermore, a LiDAR odometry method was employed in order to conduct semantic and multimodal mapping experiments on the objects related to this operation. ForestMP is composed by a mobile phone or tablet, running an Android application that integrates functions for visualisation of sensing data and operation settings, computer for sensor interface and data processing, hardware boards for actuators control, and acquisition and transmission of sensor data, and some sensors – LiDARs, cameras, GNSS receiver, and IMU. The proposed system was validated on robots and forestry machines with implements and was then successfully tested and evaluated in three of the four mentioned operations, leaving only the line vegetation clearing operation to be evaluated. |
|---|---|
| Autores principais: | Silva, Daniel Queirós da |
| Assunto: | Artificial Intelligence Forest maintenance Forest management Forestry robotics Line vegetation clearing Multimodal perception Multipurpose system Selective thinning Soil ripping Soil fertilisation Tree trunks monitoring |
| Ano: | 2024 |
| País: | Portugal |
| Tipo de documento: | tese de doutoramento |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade de Trás-os-Montes e Alto Douro |
| Idioma: | inglês |
| Origem: | Repositório da UTAD |
| _version_ | 1863854796954402816 |
|---|---|
| author | Silva, Daniel Queirós da |
| author_facet | Silva, Daniel Queirós da |
| author_role | author |
| contributor.other.fl_str_mv | Filipe, Vítor |
| contributor_name_str_mv | Filipe, Vítor Santos, Filipe Baptista Neves dos Sousa, Armando Jorge Miranda de Repositório Institucional da UTAD |
| country_str | PT |
| creators_json_str | [{\"Person.name\":\"Silva, Daniel Queirós da\"}] |
| datacite.contributors.contributor.contributorName.fl_str_mv | Filipe, Vítor Santos, Filipe Baptista Neves dos Sousa, Armando Jorge Miranda de Repositório Institucional da UTAD |
| datacite.creators.creator.creatorName.fl_str_mv | Silva, Daniel Queirós da |
| datacite.date.Accepted.fl_str_mv | 2024-07-04T00:00:00Z |
| datacite.date.available.fl_str_mv | 2024-09-16T09:43:12Z |
| datacite.date.embargoed.fl_str_mv | 2024-09-16T09:43:12Z |
| datacite.rights.fl_str_mv | http://purl.org/coar/access_right/c_abf2 |
| datacite.subjects.subject.fl_str_mv | Artificial Intelligence Forest maintenance Forest management Forestry robotics Line vegetation clearing Multimodal perception Multipurpose system Selective thinning Soil ripping Soil fertilisation Tree trunks monitoring |
| datacite.titles.title.fl_str_mv | Multimodal perception for robotics in forestry operations |
| dc.contributor.none.fl_str_mv | Filipe, Vítor Santos, Filipe Baptista Neves dos Sousa, Armando Jorge Miranda de Repositório Institucional da UTAD |
| dc.creator.none.fl_str_mv | Silva, Daniel Queirós da |
| dc.date.Accepted.fl_str_mv | 2024-07-04T00:00:00Z |
| dc.date.available.fl_str_mv | 2024-09-16T09:43:12Z |
| dc.date.embargoed.fl_str_mv | 2024-09-16T09:43:12Z |
| dc.format.none.fl_str_mv | application/pdf application/pdf application/pdf |
| dc.identifier.none.fl_str_mv | https://hdl.handle.net/10348/12848 |
| dc.language.none.fl_str_mv | eng |
| dc.rights.none.fl_str_mv | http://purl.org/coar/access_right/c_abf2 |
| dc.subject.none.fl_str_mv | Artificial Intelligence Forest maintenance Forest management Forestry robotics Line vegetation clearing Multimodal perception Multipurpose system Selective thinning Soil ripping Soil fertilisation Tree trunks monitoring |
| dc.title.fl_str_mv | Multimodal perception for robotics in forestry operations |
| dc.type.none.fl_str_mv | http://purl.org/coar/resource_type/c_db06 |
| description | Forest fires constitute not only an environmental problem with natural resources destruction and ecological degradation, but also a societal problem where human lives can be at stake. The integration of robotic solutions and advanced perception systems can facilitate the optimal management of forest resources, enhance the productivity of forestry tasks, and increase the profitability of forest stands. Nevertheless, perception at stand-level in forests is a domain with a great potential to grow, being underdeveloped compared to airborne or spaceborne remote sensing. This thesis was developed within this scope with the goal of achieving a multimodal and multipurpose perception system for robotics in forestry operations at standlevel. A multipurpose and multimodal system called ForestMP (Figure 1) was designed to handle multiple sensors of different natures and fuse the information into meaningful perceptions that can be used in robotic forestry operations. The system at hand can intervene at least in four distinct forestry operations: forest soil preparation before plantation (soil ripping and fertilisation), forest environment monitoring (tree trunk monitoring and mapping), forest species growth control (eucalyptus selective thinning), and forest areas maintenance (line vegetation clearing). During the soil ripping and fertilisation operation, the system maps the ripping depth while it applies fertiliser at a variable rate based on a prescription map. The operation of forest tree monitoring consists in detecting and mapping the trees using multispectral imagery. With respect to selective thinning, the system runs an algorithm capable of automatically selecting eucalyptus stems to keep based on their robustness and uprightness. Regarding line vegetation clearing, the system is able to identify objects to clean or not based on an OAK-D device, and subsequently control a forest clearing implement to perform the operation. Furthermore, a LiDAR odometry method was employed in order to conduct semantic and multimodal mapping experiments on the objects related to this operation. ForestMP is composed by a mobile phone or tablet, running an Android application that integrates functions for visualisation of sensing data and operation settings, computer for sensor interface and data processing, hardware boards for actuators control, and acquisition and transmission of sensor data, and some sensors – LiDARs, cameras, GNSS receiver, and IMU. The proposed system was validated on robots and forestry machines with implements and was then successfully tested and evaluated in three of the four mentioned operations, leaving only the line vegetation clearing operation to be evaluated. |
| dirty | 0 |
| eu_rights_str_mv | openAccess |
| format | doctoralThesis |
| fulltext.url.fl_str_mv | https://repositorio.utad.pt/bitstreams/178e7303-7838-44ea-a8ad-8590f98373d6/download |
| id | utad_8650d39d00a93cbf15df5260f53ac8b2 |
| identifier.url.fl_str_mv | https://hdl.handle.net/10348/12848 |
| instacron_str | utad |
| institution | Universidade de Trás-os-Montes e Alto Douro |
| instname_str | Universidade de Trás-os-Montes e Alto Douro |
| language | eng |
| network_acronym_str | utad |
| network_name_str | Repositório da UTAD |
| oai_identifier_str | oai:repositorio.utad.pt:10348/12848 |
| organization_str_mv | urn:organizationAcronym:utad |
| person_str_mv | Silva, Daniel Queirós da Filipe, Vítor Filipe, Vítor http://orcid.org/0000-0002-3747-6577 0000-0002-3747-6577 |
| publishDate | 2024 |
| reponame_str | Repositório da UTAD |
| repository_id_str | urn:repositoryAcronym:utad |
| service_str_mv | urn:repositoryAcronym:utad |
| spelling | engForest fires constitute not only an environmental problem with natural resources destruction and ecological degradation, but also a societal problem where human lives can be at stake. The integration of robotic solutions and advanced perception systems can facilitate the optimal management of forest resources, enhance the productivity of forestry tasks, and increase the profitability of forest stands. Nevertheless, perception at stand-level in forests is a domain with a great potential to grow, being underdeveloped compared to airborne or spaceborne remote sensing. This thesis was developed within this scope with the goal of achieving a multimodal and multipurpose perception system for robotics in forestry operations at standlevel. A multipurpose and multimodal system called ForestMP (Figure 1) was designed to handle multiple sensors of different natures and fuse the information into meaningful perceptions that can be used in robotic forestry operations. The system at hand can intervene at least in four distinct forestry operations: forest soil preparation before plantation (soil ripping and fertilisation), forest environment monitoring (tree trunk monitoring and mapping), forest species growth control (eucalyptus selective thinning), and forest areas maintenance (line vegetation clearing). During the soil ripping and fertilisation operation, the system maps the ripping depth while it applies fertiliser at a variable rate based on a prescription map. The operation of forest tree monitoring consists in detecting and mapping the trees using multispectral imagery. With respect to selective thinning, the system runs an algorithm capable of automatically selecting eucalyptus stems to keep based on their robustness and uprightness. Regarding line vegetation clearing, the system is able to identify objects to clean or not based on an OAK-D device, and subsequently control a forest clearing implement to perform the operation. Furthermore, a LiDAR odometry method was employed in order to conduct semantic and multimodal mapping experiments on the objects related to this operation. ForestMP is composed by a mobile phone or tablet, running an Android application that integrates functions for visualisation of sensing data and operation settings, computer for sensor interface and data processing, hardware boards for actuators control, and acquisition and transmission of sensor data, and some sensors – LiDARs, cameras, GNSS receiver, and IMU. The proposed system was validated on robots and forestry machines with implements and was then successfully tested and evaluated in three of the four mentioned operations, leaving only the line vegetation clearing operation to be evaluated.application/pdfapplication/pdfapplication/pdfMultimodal perception for robotics in forestry operationsSilva, Daniel Queirós daOtherPersonalFilipe, VítorVítor Manuel de JesusFilipeORCIDhttp://orcid.org0000-0002-3747-6577Santos, Filipe Baptista Neves dosSousa, Armando Jorge Miranda deHostingInstitutionOrganizationalRepositório Institucional da UTADe-mailmailto:jborges@utad.ptjborges@utad.pt2024-09-16T09:43:12Z2024-07-042024-07-102024-07-04T00:00:00ZHandlehttps://hdl.handle.net/10348/12848http://purl.org/coar/access_right/c_abf2open accessArtificial IntelligenceForest maintenanceForest managementForestry roboticsLine vegetation clearingMultimodal perceptionMultipurpose systemSelective thinningSoil rippingSoil fertilisationTree trunks monitoring133920678 bytes610392 bytes641971 bytesliteraturehttp://purl.org/coar/resource_type/c_db06doctoral thesishttp://purl.org/coar/access_right/c_abf2application/pdffulltexthttps://repositorio.utad.pt/bitstreams/178e7303-7838-44ea-a8ad-8590f98373d6/downloadhttp://purl.org/coar/access_right/c_abf2application/pdffulltexthttps://repositorio.utad.pt/bitstreams/e7c75b9c-8ef7-40e1-8170-cf39f7ef7d42/downloadhttp://purl.org/coar/access_right/c_abf2application/pdffulltexthttps://repositorio.utad.pt/bitstreams/c100b8c8-a018-4695-8250-ed548daafb89/download |
| spellingShingle | Multimodal perception for robotics in forestry operations Silva, Daniel Queirós da Artificial Intelligence Forest maintenance Forest management Forestry robotics Line vegetation clearing Multimodal perception Multipurpose system Selective thinning Soil ripping Soil fertilisation Tree trunks monitoring |
| subject.fl_str_mv | Artificial Intelligence Forest maintenance Forest management Forestry robotics Line vegetation clearing Multimodal perception Multipurpose system Selective thinning Soil ripping Soil fertilisation Tree trunks monitoring |
| title | Multimodal perception for robotics in forestry operations |
| title_full | Multimodal perception for robotics in forestry operations |
| title_fullStr | Multimodal perception for robotics in forestry operations |
| title_full_unstemmed | Multimodal perception for robotics in forestry operations |
| title_short | Multimodal perception for robotics in forestry operations |
| title_sort | Multimodal perception for robotics in forestry operations |
| topic | Artificial Intelligence Forest maintenance Forest management Forestry robotics Line vegetation clearing Multimodal perception Multipurpose system Selective thinning Soil ripping Soil fertilisation Tree trunks monitoring |
| topic_facet | Artificial Intelligence Forest maintenance Forest management Forestry robotics Line vegetation clearing Multimodal perception Multipurpose system Selective thinning Soil ripping Soil fertilisation Tree trunks monitoring |
| url | https://hdl.handle.net/10348/12848 |
| visible | 1 |