Publicação
Hands-free Interaction Framework for Virtual Reality Applications
| Resumo: | Virtual Reality (VR) has significantly transformed digital experiences and Immersive VR (iVR) systems in particular have evolved to improve users’ sense of presence and support realistic experiences, enabling users to experience a Virtual Environment (VE) through devices like an head-mounted display (HMD). While traditional iVR interactions rely heavily on tracked handheld controllers, there is a shift in the industry towards the use of Natural User Interfaces (NUI) that allow users to directly interact with virtual objects by using their hands as they would in reality. However, hand gestures can be cumbersome and limit the immersive experience, particularly in selection tasks and system control tasks with Graphical User Interfaces (GUI) that are based on a point-and-click metaphor and in scenarios where hands are otherwise occupied. As such, this thesis explores the use of hands-free interaction methods as a capable alternative to hand-based interactions, focusing on GUI selection tasks within iVR environments. To address this thesis, three research questions were set: (1) what hands-free methods can be used to perform selections with GUI elements, (2) do those methods provide similar usability to the traditional controllers, and (3) within hands-free interactions which methods provide better usability results. To answer these questions a modular interaction framework and a testbed application were developed facilitating the evaluation of interaction methods in iVR, and allowing future integration of new methods and evaluation tests. The first question was addressed by conducting an extensive systematic review of the literature which demonstrated that the main hands-free methods are voice commands, eye gaze, and head gaze. Novel approaches (e.g., brain-computer interfaces, electromyography) were less used given their apparatus requirements. As selections can be interpreted as a point-confirm process, any method that allows users to intently choose a direction can be used to perform the pointing phase, and any method that detects a user action can generally be used for confirmation. The remaining questions were answered through empirical studies, which assessed the usability of the methods in relevant metrics, namely iVR Experience (response to the use of iVR), System Usability (specific to the interaction method), User Satisfaction (while using the interaction methods), and Effectiveness and Efficiency (of the methods completing the interaction tasks). The methods were evaluated when the selections were both the primary and background tasks. The findings indicate that hands-free methods can perform point-and-click interactions efficiently and effectively, without detracting from the immersive experience. The thesis contributes to the field by providing a set of interaction guidelines for developers, interaction designers, and researchers, aimed at enhancing the usability of hands-free interaction and supporting the broader adoption of hands-free methods. |
|---|---|
| Autores principais: | Monteiro, Pedro Miguel Ermida |
| Assunto: | Immersive Virtual Reality Human-Computer Interaction Hands-free Interaction Selection GUI Usability Evaluation Guidelines |
| Ano: | 2025 |
| País: | Portugal |
| Tipo de documento: | tese de doutoramento |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade de Trás-os-Montes e Alto Douro |
| Idioma: | inglês |
| Origem: | Repositório da UTAD |
| Resumo: | Virtual Reality (VR) has significantly transformed digital experiences and Immersive VR (iVR) systems in particular have evolved to improve users’ sense of presence and support realistic experiences, enabling users to experience a Virtual Environment (VE) through devices like an head-mounted display (HMD). While traditional iVR interactions rely heavily on tracked handheld controllers, there is a shift in the industry towards the use of Natural User Interfaces (NUI) that allow users to directly interact with virtual objects by using their hands as they would in reality. However, hand gestures can be cumbersome and limit the immersive experience, particularly in selection tasks and system control tasks with Graphical User Interfaces (GUI) that are based on a point-and-click metaphor and in scenarios where hands are otherwise occupied. As such, this thesis explores the use of hands-free interaction methods as a capable alternative to hand-based interactions, focusing on GUI selection tasks within iVR environments. To address this thesis, three research questions were set: (1) what hands-free methods can be used to perform selections with GUI elements, (2) do those methods provide similar usability to the traditional controllers, and (3) within hands-free interactions which methods provide better usability results. To answer these questions a modular interaction framework and a testbed application were developed facilitating the evaluation of interaction methods in iVR, and allowing future integration of new methods and evaluation tests. The first question was addressed by conducting an extensive systematic review of the literature which demonstrated that the main hands-free methods are voice commands, eye gaze, and head gaze. Novel approaches (e.g., brain-computer interfaces, electromyography) were less used given their apparatus requirements. As selections can be interpreted as a point-confirm process, any method that allows users to intently choose a direction can be used to perform the pointing phase, and any method that detects a user action can generally be used for confirmation. The remaining questions were answered through empirical studies, which assessed the usability of the methods in relevant metrics, namely iVR Experience (response to the use of iVR), System Usability (specific to the interaction method), User Satisfaction (while using the interaction methods), and Effectiveness and Efficiency (of the methods completing the interaction tasks). The methods were evaluated when the selections were both the primary and background tasks. The findings indicate that hands-free methods can perform point-and-click interactions efficiently and effectively, without detracting from the immersive experience. The thesis contributes to the field by providing a set of interaction guidelines for developers, interaction designers, and researchers, aimed at enhancing the usability of hands-free interaction and supporting the broader adoption of hands-free methods. |
|---|