Publicação
Deep reinforcement learning applied to a robotic pick-and-place application
| Resumo: | Industrial robot manipulators are widely used for repetitive applications that require high precision, like pick-and-place. In many cases, the movements of industrial robot manipulators are hard-coded or manually defined, and need to be adjusted if the objects being manipulated change position. To increase flexibility, an industrial robot should be able to adjust its configuration in order to grasp objects in variable/ unknown positions. This can be achieved by off-the-shelf visionbased solutions, but most require prior knowledge about each object to be manipulated. To address this issue, this work presents a ROS-based deep reinforcement learning solution to robotic grasping for a Collaborative Robot (Cobot) using a depth camera. The solution uses deep Q-learning to process the color and depth images and generate a ϵ- greedy policy used to define the robot action. The Q-values are estimated using Convolutional Neural Network (CNN) based on pre-trained models for feature extraction. Experiments were carried out in a simulated environment to compare the performance of four different pretrained CNN models (RexNext, MobileNet, MNASNet and DenseNet). Results show that the best performance in our application was reached by MobileNet, with an average of 84 % accuracy after training in simulated environment. |
|---|---|
| Autores principais: | Gomes, Natanael Magno |
| Outros Autores: | Martins, Felipe N.; Lima, José; Wörtche, Heinrich |
| Assunto: | Cobots Reinforcement learning Computer vision Pick-and-place Grasping |
| Ano: | 2021 |
| País: | Portugal |
| Tipo de documento: | comunicação em conferência |
| Tipo de acesso: | acesso restrito |
| Instituição associada: | Instituto Politécnico de Bragança |
| Idioma: | inglês |
| Origem: | Biblioteca Digital do IPB |
| _version_ | 1867172952607293440 |
|---|---|
| author | Gomes, Natanael Magno |
| author2 | Martins, Felipe N. Lima, José Wörtche, Heinrich |
| author2_role | author author author |
| author_facet | Gomes, Natanael Magno Martins, Felipe N. Lima, José Wörtche, Heinrich |
| author_role | author |
| contributor_name_str_mv | Biblioteca Digital do IPB |
| country_str | PT |
| creators_json_txt | [{\"Person.name\":\"Gomes, Natanael Magno\"},{\"Person.name\":\"Martins, Felipe N.\"},{\"Person.name\":\"Lima, José\",\"Person.identifier.orcid\":\"0000-0001-7902-1207\"},{\"Person.name\":\"Wörtche, Heinrich\"}] |
| datacite.contributors.contributor.contributorName.fl_str_mv | Biblioteca Digital do IPB |
| datacite.creators.creator.creatorName.fl_str_mv | Gomes, Natanael Magno Martins, Felipe N. Lima, José Wörtche, Heinrich |
| datacite.date.Accepted.fl_str_mv | 2021-01-01T00:00:00Z |
| datacite.date.available.fl_str_mv | 2022-04-05T14:21:52Z |
| datacite.date.embargoed.fl_str_mv | 2022-04-05T14:21:52Z |
| datacite.rights.fl_str_mv | http://purl.org/coar/access_right/c_16ec |
| datacite.subjects.subject.fl_str_mv | Cobots Reinforcement learning Computer vision Pick-and-place Grasping |
| datacite.titles.title.fl_str_mv | Deep reinforcement learning applied to a robotic pick-and-place application |
| dc.contributor.none.fl_str_mv | Biblioteca Digital do IPB |
| dc.creator.none.fl_str_mv | Gomes, Natanael Magno Martins, Felipe N. Lima, José Wörtche, Heinrich |
| dc.date.Accepted.fl_str_mv | 2021-01-01T00:00:00Z |
| dc.date.available.fl_str_mv | 2022-04-05T14:21:52Z |
| dc.date.embargoed.fl_str_mv | 2022-04-05T14:21:52Z |
| dc.format.none.fl_str_mv | application/pdf |
| dc.identifier.none.fl_str_mv | http://hdl.handle.net/10198/25357 |
| dc.language.none.fl_str_mv | eng |
| dc.publisher.none.fl_str_mv | Springer Nature |
| dc.rights.cclincense.fl_str_mv | http://creativecommons.org/licenses/by/4.0/ |
| dc.rights.none.fl_str_mv | http://purl.org/coar/access_right/c_16ec |
| dc.subject.none.fl_str_mv | Cobots Reinforcement learning Computer vision Pick-and-place Grasping |
| dc.title.fl_str_mv | Deep reinforcement learning applied to a robotic pick-and-place application |
| dc.type.none.fl_str_mv | http://purl.org/coar/resource_type/c_5794 |
| description | Industrial robot manipulators are widely used for repetitive applications that require high precision, like pick-and-place. In many cases, the movements of industrial robot manipulators are hard-coded or manually defined, and need to be adjusted if the objects being manipulated change position. To increase flexibility, an industrial robot should be able to adjust its configuration in order to grasp objects in variable/ unknown positions. This can be achieved by off-the-shelf visionbased solutions, but most require prior knowledge about each object to be manipulated. To address this issue, this work presents a ROS-based deep reinforcement learning solution to robotic grasping for a Collaborative Robot (Cobot) using a depth camera. The solution uses deep Q-learning to process the color and depth images and generate a ϵ- greedy policy used to define the robot action. The Q-values are estimated using Convolutional Neural Network (CNN) based on pre-trained models for feature extraction. Experiments were carried out in a simulated environment to compare the performance of four different pretrained CNN models (RexNext, MobileNet, MNASNet and DenseNet). Results show that the best performance in our application was reached by MobileNet, with an average of 84 % accuracy after training in simulated environment. |
| dirty | 0 |
| eu_rights_str_mv | restrictedAccess |
| format | conferencePaper |
| fulltext.url.fl_str_mv | https://bibliotecadigital.ipb.pt/bitstreams/cf06c6b5-54d1-4d55-89aa-ba7887db5757/download |
| funding.funder.alternateName_str_mv | FCT |
| funding.funder.identifier_str_mv | http://doi.org/10.13039/501100001871 |
| funding.funder.name_str_mv | Fundação para a Ciência e a Tecnologia |
| funding.name_str_mv | 6817 - DCRRNI ID |
| id | ipb_76d4cd577d095faf06bb5f7674e436e8 |
| identifier.url.fl_str_mv | http://hdl.handle.net/10198/25357 |
| instacron_str | ipb |
| institution | Instituto Politécnico de Bragança |
| instname_str | Instituto Politécnico de Bragança |
| language | eng |
| network_acronym_str | ipb |
| network_name_str | Biblioteca Digital do IPB |
| oai_identifier_str | oai:bibliotecadigital.ipb.pt:10198/25357 |
| organization_str_mv | urn:organizationAcronym:ipb |
| person_str_mv | Gomes, Natanael Magno Martins, Felipe N. Lima, José Lima, José https://www.ciencia-id.pt/6016-C902-86A9 6016-C902-86A9 http://orcid.org/0000-0001-7902-1207 0000-0001-7902-1207 Wörtche, Heinrich |
| publishDate | 2021 |
| publisher.none.fl_str_mv | Springer Nature |
| reponame_str | Biblioteca Digital do IPB |
| repository_id_str | urn:repositoryAcronym:ipb |
| service_str_mv | urn:repositoryAcronym:ipb |
| spelling | engSpringer Naturept_PTIndustrial robot manipulators are widely used for repetitive applications that require high precision, like pick-and-place. In many cases, the movements of industrial robot manipulators are hard-coded or manually defined, and need to be adjusted if the objects being manipulated change position. To increase flexibility, an industrial robot should be able to adjust its configuration in order to grasp objects in variable/ unknown positions. This can be achieved by off-the-shelf visionbased solutions, but most require prior knowledge about each object to be manipulated. To address this issue, this work presents a ROS-based deep reinforcement learning solution to robotic grasping for a Collaborative Robot (Cobot) using a depth camera. The solution uses deep Q-learning to process the color and depth images and generate a ϵ- greedy policy used to define the robot action. The Q-values are estimated using Convolutional Neural Network (CNN) based on pre-trained models for feature extraction. Experiments were carried out in a simulated environment to compare the performance of four different pretrained CNN models (RexNext, MobileNet, MNASNet and DenseNet). Results show that the best performance in our application was reached by MobileNet, with an average of 84 % accuracy after training in simulated environment.application/pdfpt_PTDeep reinforcement learning applied to a robotic pick-and-place applicationGomes, Natanael MagnoMartins, Felipe N.PersonalLima, JoséDSpacehttp://dspace.org/items/d88c2b2a-efc2-48ef-b1fd-1145475e0055DSpacehttp://dspace.org/items/d88c2b2a-efc2-48ef-b1fd-1145475e0055LimaJoséCiência IDhttps://www.ciencia-id.pt6016-C902-86A9ORCIDhttp://orcid.org0000-0001-7902-1207Researcher IDhttps://www.researcherid.comL-3370-2014Scopus Author IDhttps://www.scopus.com55851941311Wörtche, HeinrichHostingInstitutionOrganizationalBiblioteca Digital do IPBe-mailmailto:dspace@ipb.ptdspace@ipb.ptISBNIsPartOf978-3-030-91884-2DOIIsPartOf10.1007/978-3-030-91885-9_182022-04-05T14:21:52Z20212021-01-01T00:00:00ZHandlehttp://hdl.handle.net/10198/25357http://purl.org/coar/access_right/c_16ecrestricted accessCobotsReinforcement learningComputer visionPick-and-placeGrasping3586227 bytesFundação para a Ciência e a TecnologiaResearch Centre in Digitalization and Intelligent Robotics6817 - DCRRNI IDCrossref Funder IDhttp://doi.org/10.13039/501100001871other research producthttp://purl.org/coar/resource_type/c_5794conference paper2021http://creativecommons.org/licenses/by/4.0/http://purl.org/coar/access_right/c_16ecapplication/pdffulltexthttps://bibliotecadigital.ipb.pt/bitstreams/cf06c6b5-54d1-4d55-89aa-ba7887db5757/downloadOptimization, learning algorithms and applications: first International Conference, OL2A 20211488251265Bragança |
| spellingShingle | Deep reinforcement learning applied to a robotic pick-and-place application Gomes, Natanael Magno Cobots Reinforcement learning Computer vision Pick-and-place Grasping |
| status | SINGLETON |
| subject.fl_str_mv | Cobots Reinforcement learning Computer vision Pick-and-place Grasping |
| title | Deep reinforcement learning applied to a robotic pick-and-place application |
| title_full | Deep reinforcement learning applied to a robotic pick-and-place application |
| title_fullStr | Deep reinforcement learning applied to a robotic pick-and-place application |
| title_full_unstemmed | Deep reinforcement learning applied to a robotic pick-and-place application |
| title_short | Deep reinforcement learning applied to a robotic pick-and-place application |
| title_sort | Deep reinforcement learning applied to a robotic pick-and-place application |
| topic | Cobots Reinforcement learning Computer vision Pick-and-place Grasping |
| topic_facet | Cobots Reinforcement learning Computer vision Pick-and-place Grasping |
| url | http://hdl.handle.net/10198/25357 |
| visible | 1 |