Publicação
Hands-free Interaction Framework for Virtual Reality Applications
| Resumo: | Virtual Reality (VR) has significantly transformed digital experiences and Immersive VR (iVR) systems in particular have evolved to improve users’ sense of presence and support realistic experiences, enabling users to experience a Virtual Environment (VE) through devices like an head-mounted display (HMD). While traditional iVR interactions rely heavily on tracked handheld controllers, there is a shift in the industry towards the use of Natural User Interfaces (NUI) that allow users to directly interact with virtual objects by using their hands as they would in reality. However, hand gestures can be cumbersome and limit the immersive experience, particularly in selection tasks and system control tasks with Graphical User Interfaces (GUI) that are based on a point-and-click metaphor and in scenarios where hands are otherwise occupied. As such, this thesis explores the use of hands-free interaction methods as a capable alternative to hand-based interactions, focusing on GUI selection tasks within iVR environments. To address this thesis, three research questions were set: (1) what hands-free methods can be used to perform selections with GUI elements, (2) do those methods provide similar usability to the traditional controllers, and (3) within hands-free interactions which methods provide better usability results. To answer these questions a modular interaction framework and a testbed application were developed facilitating the evaluation of interaction methods in iVR, and allowing future integration of new methods and evaluation tests. The first question was addressed by conducting an extensive systematic review of the literature which demonstrated that the main hands-free methods are voice commands, eye gaze, and head gaze. Novel approaches (e.g., brain-computer interfaces, electromyography) were less used given their apparatus requirements. As selections can be interpreted as a point-confirm process, any method that allows users to intently choose a direction can be used to perform the pointing phase, and any method that detects a user action can generally be used for confirmation. The remaining questions were answered through empirical studies, which assessed the usability of the methods in relevant metrics, namely iVR Experience (response to the use of iVR), System Usability (specific to the interaction method), User Satisfaction (while using the interaction methods), and Effectiveness and Efficiency (of the methods completing the interaction tasks). The methods were evaluated when the selections were both the primary and background tasks. The findings indicate that hands-free methods can perform point-and-click interactions efficiently and effectively, without detracting from the immersive experience. The thesis contributes to the field by providing a set of interaction guidelines for developers, interaction designers, and researchers, aimed at enhancing the usability of hands-free interaction and supporting the broader adoption of hands-free methods. |
|---|---|
| Autores principais: | Monteiro, Pedro Miguel Ermida |
| Assunto: | Immersive Virtual Reality Human-Computer Interaction Hands-free Interaction Selection GUI Usability Evaluation Guidelines |
| Ano: | 2025 |
| País: | Portugal |
| Tipo de documento: | tese de doutoramento |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade de Trás-os-Montes e Alto Douro |
| Idioma: | inglês |
| Origem: | Repositório da UTAD |
| _version_ | 1865920869859065856 |
|---|---|
| author | Monteiro, Pedro Miguel Ermida |
| author_facet | Monteiro, Pedro Miguel Ermida Monteiro, Pedro Miguel Ermida |
| author_role | author |
| contributor.other.fl_str_mv | Bessa, Maximino Melo, Miguel |
| contributor_name_str_mv | Bessa, Maximino Melo, Miguel Repositório Institucional da UTAD |
| country_str | PT |
| creators_json_str | [{\"Person.name\":\"Monteiro, Pedro Miguel Ermida\"}] |
| datacite.contributors.contributor.contributorName.fl_str_mv | Bessa, Maximino Melo, Miguel Repositório Institucional da UTAD |
| datacite.creators.creator.creatorName.fl_str_mv | Monteiro, Pedro Miguel Ermida |
| datacite.date.Accepted.fl_str_mv | 2025-01-31T00:00:00Z |
| datacite.date.available.fl_str_mv | 2026-03-19T14:41:53Z |
| datacite.date.embargoed.fl_str_mv | 2026-03-19T14:41:53Z |
| datacite.rights.fl_str_mv | http://purl.org/coar/access_right/c_abf2 |
| datacite.subjects.subject.fl_str_mv | Immersive Virtual Reality Human-Computer Interaction Hands-free Interaction Selection GUI Usability Evaluation Guidelines |
| datacite.titles.title.fl_str_mv | Hands-free Interaction Framework for Virtual Reality Applications |
| dc.contributor.none.fl_str_mv | Bessa, Maximino Melo, Miguel Repositório Institucional da UTAD |
| dc.creator.none.fl_str_mv | Monteiro, Pedro Miguel Ermida |
| dc.date.Accepted.fl_str_mv | 2025-01-31T00:00:00Z |
| dc.date.available.fl_str_mv | 2026-03-19T14:41:53Z |
| dc.date.embargoed.fl_str_mv | 2026-03-19T14:41:53Z |
| dc.format.none.fl_str_mv | application/pdf application/pdf application/pdf |
| dc.identifier.none.fl_str_mv | https://hdl.handle.net/10348/14006 |
| dc.language.none.fl_str_mv | eng |
| dc.rights.none.fl_str_mv | http://purl.org/coar/access_right/c_abf2 |
| dc.subject.none.fl_str_mv | Immersive Virtual Reality Human-Computer Interaction Hands-free Interaction Selection GUI Usability Evaluation Guidelines |
| dc.title.fl_str_mv | Hands-free Interaction Framework for Virtual Reality Applications |
| dc.type.none.fl_str_mv | http://purl.org/coar/resource_type/c_db06 |
| description | Virtual Reality (VR) has significantly transformed digital experiences and Immersive VR (iVR) systems in particular have evolved to improve users’ sense of presence and support realistic experiences, enabling users to experience a Virtual Environment (VE) through devices like an head-mounted display (HMD). While traditional iVR interactions rely heavily on tracked handheld controllers, there is a shift in the industry towards the use of Natural User Interfaces (NUI) that allow users to directly interact with virtual objects by using their hands as they would in reality. However, hand gestures can be cumbersome and limit the immersive experience, particularly in selection tasks and system control tasks with Graphical User Interfaces (GUI) that are based on a point-and-click metaphor and in scenarios where hands are otherwise occupied. As such, this thesis explores the use of hands-free interaction methods as a capable alternative to hand-based interactions, focusing on GUI selection tasks within iVR environments. To address this thesis, three research questions were set: (1) what hands-free methods can be used to perform selections with GUI elements, (2) do those methods provide similar usability to the traditional controllers, and (3) within hands-free interactions which methods provide better usability results. To answer these questions a modular interaction framework and a testbed application were developed facilitating the evaluation of interaction methods in iVR, and allowing future integration of new methods and evaluation tests. The first question was addressed by conducting an extensive systematic review of the literature which demonstrated that the main hands-free methods are voice commands, eye gaze, and head gaze. Novel approaches (e.g., brain-computer interfaces, electromyography) were less used given their apparatus requirements. As selections can be interpreted as a point-confirm process, any method that allows users to intently choose a direction can be used to perform the pointing phase, and any method that detects a user action can generally be used for confirmation. The remaining questions were answered through empirical studies, which assessed the usability of the methods in relevant metrics, namely iVR Experience (response to the use of iVR), System Usability (specific to the interaction method), User Satisfaction (while using the interaction methods), and Effectiveness and Efficiency (of the methods completing the interaction tasks). The methods were evaluated when the selections were both the primary and background tasks. The findings indicate that hands-free methods can perform point-and-click interactions efficiently and effectively, without detracting from the immersive experience. The thesis contributes to the field by providing a set of interaction guidelines for developers, interaction designers, and researchers, aimed at enhancing the usability of hands-free interaction and supporting the broader adoption of hands-free methods. |
| dirty | 0 |
| eu_rights_str_mv | openAccess |
| format | doctoralThesis |
| fulltext.url.fl_str_mv | https://repositorio.utad.pt/bitstreams/874cda76-caca-443b-b085-7713ef505206/download |
| id | utad_bd1d7dbf76fb21f5c60c780cfdcaa06a |
| identifier.url.fl_str_mv | https://hdl.handle.net/10348/14006 |
| instacron_str | utad |
| institution | Universidade de Trás-os-Montes e Alto Douro |
| instname_str | Universidade de Trás-os-Montes e Alto Douro |
| language | eng |
| network_acronym_str | utad |
| network_name_str | Repositório da UTAD |
| oai_identifier_str | oai:repositorio.utad.pt:10348/14006 |
| organization_str_mv | urn:organizationAcronym:utad |
| person_str_mv | Monteiro, Pedro Miguel Ermida Bessa, Maximino Bessa, Maximino https://www.ciencia-id.pt/CA19-AF37-DA35 CA19-AF37-DA35 http://orcid.org/0000-0002-3002-704X 0000-0002-3002-704X Melo, Miguel Melo, Miguel https://www.ciencia-id.pt/A315-E9CE-0FD6 A315-E9CE-0FD6 http://orcid.org/0000-0003-4050-3473 0000-0003-4050-3473 |
| publishDate | 2025 |
| reponame_str | Repositório da UTAD |
| repository_id_str | urn:repositoryAcronym:utad |
| service_str_mv | urn:repositoryAcronym:utad |
| spelling | engVirtual Reality (VR) has significantly transformed digital experiences and Immersive VR (iVR) systems in particular have evolved to improve users’ sense of presence and support realistic experiences, enabling users to experience a Virtual Environment (VE) through devices like an head-mounted display (HMD). While traditional iVR interactions rely heavily on tracked handheld controllers, there is a shift in the industry towards the use of Natural User Interfaces (NUI) that allow users to directly interact with virtual objects by using their hands as they would in reality. However, hand gestures can be cumbersome and limit the immersive experience, particularly in selection tasks and system control tasks with Graphical User Interfaces (GUI) that are based on a point-and-click metaphor and in scenarios where hands are otherwise occupied. As such, this thesis explores the use of hands-free interaction methods as a capable alternative to hand-based interactions, focusing on GUI selection tasks within iVR environments. To address this thesis, three research questions were set: (1) what hands-free methods can be used to perform selections with GUI elements, (2) do those methods provide similar usability to the traditional controllers, and (3) within hands-free interactions which methods provide better usability results. To answer these questions a modular interaction framework and a testbed application were developed facilitating the evaluation of interaction methods in iVR, and allowing future integration of new methods and evaluation tests. The first question was addressed by conducting an extensive systematic review of the literature which demonstrated that the main hands-free methods are voice commands, eye gaze, and head gaze. Novel approaches (e.g., brain-computer interfaces, electromyography) were less used given their apparatus requirements. As selections can be interpreted as a point-confirm process, any method that allows users to intently choose a direction can be used to perform the pointing phase, and any method that detects a user action can generally be used for confirmation. The remaining questions were answered through empirical studies, which assessed the usability of the methods in relevant metrics, namely iVR Experience (response to the use of iVR), System Usability (specific to the interaction method), User Satisfaction (while using the interaction methods), and Effectiveness and Efficiency (of the methods completing the interaction tasks). The methods were evaluated when the selections were both the primary and background tasks. The findings indicate that hands-free methods can perform point-and-click interactions efficiently and effectively, without detracting from the immersive experience. The thesis contributes to the field by providing a set of interaction guidelines for developers, interaction designers, and researchers, aimed at enhancing the usability of hands-free interaction and supporting the broader adoption of hands-free methods.application/pdfapplication/pdfapplication/pdfHands-free Interaction Framework for Virtual Reality ApplicationsMonteiro, Pedro Miguel ErmidaOtherPersonalBessa, MaximinoMaximino Esteves CorreiaBessaUTAD-DENGScopus Author IDhttps://www.scopus.com14031038800Ciência IDhttps://www.ciencia-id.ptCA19-AF37-DA35ORCIDhttp://orcid.org0000-0002-3002-704XOtherPersonalMelo, MiguelMiguel Ângelo Correia deMeloUTAD-DENGScopus Author IDhttps://www.scopus.com7102354924Ciência IDhttps://www.ciencia-id.ptA315-E9CE-0FD6ORCIDhttp://orcid.org0000-0003-4050-3473HostingInstitutionOrganizationalRepositório Institucional da UTADe-mailmailto:jborges@utad.ptjborges@utad.pt2026-03-19T14:41:53Z2025-01-312025-02-132025-01-31T00:00:00ZHandlehttps://hdl.handle.net/10348/14006http://purl.org/coar/access_right/c_abf2open accessImmersive Virtual RealityHuman-Computer InteractionHands-free InteractionSelectionGUIUsability EvaluationGuidelines9742809 bytes2349196 bytes1619332 bytesliteraturehttp://purl.org/coar/resource_type/c_db06doctoral thesishttp://purl.org/coar/access_right/c_abf2application/pdffulltexthttps://repositorio.utad.pt/bitstreams/874cda76-caca-443b-b085-7713ef505206/downloadhttp://purl.org/coar/access_right/c_abf2application/pdffulltexthttps://repositorio.utad.pt/bitstreams/289136b5-6665-408e-aed6-36b68a2c3a72/downloadhttp://purl.org/coar/access_right/c_abf2application/pdffulltexthttps://repositorio.utad.pt/bitstreams/92bc1560-d513-4729-b237-0ac676ce3cf4/download |
| spellingShingle | Hands-free Interaction Framework for Virtual Reality Applications Hands-free Interaction Framework for Virtual Reality Applications Monteiro, Pedro Miguel Ermida Immersive Virtual Reality Human-Computer Interaction Hands-free Interaction Selection GUI Usability Evaluation Guidelines Monteiro, Pedro Miguel Ermida Immersive Virtual Reality Human-Computer Interaction Hands-free Interaction Selection GUI Usability Evaluation Guidelines |
| status | NEW |
| subject.fl_str_mv | Immersive Virtual Reality Human-Computer Interaction Hands-free Interaction Selection GUI Usability Evaluation Guidelines |
| title | Hands-free Interaction Framework for Virtual Reality Applications |
| title_full | Hands-free Interaction Framework for Virtual Reality Applications |
| title_fullStr | Hands-free Interaction Framework for Virtual Reality Applications Hands-free Interaction Framework for Virtual Reality Applications |
| title_full_unstemmed | Hands-free Interaction Framework for Virtual Reality Applications Hands-free Interaction Framework for Virtual Reality Applications |
| title_short | Hands-free Interaction Framework for Virtual Reality Applications |
| title_sort | Hands-free Interaction Framework for Virtual Reality Applications |
| topic | Immersive Virtual Reality Human-Computer Interaction Hands-free Interaction Selection GUI Usability Evaluation Guidelines |
| topic_facet | Immersive Virtual Reality Human-Computer Interaction Hands-free Interaction Selection GUI Usability Evaluation Guidelines |
| url | https://hdl.handle.net/10348/14006 |
| visible | 1 |