Publicação
Gestures speak louder than words: radar-based gesture interaction with smart environments
| Resumo: | The environments we live in are becoming increasingly smarter, creating the need of finding suitable ways to interact with them. While interaction modes such as speech, text, and touch remain prevalent, there are associated challenges related to background noise interference, accessibility issues, and reliance on devices (e.g., smartphones, wearables). Gesture-based interaction emerges as a promising alternative or complement to those modes. In this context, radars represent a good option for enabling gesture recognition, since they rely on radio waves to detect moving targets, not presenting the disadvantages of other sensors such as cameras and wearables, which can be considered too intrusive by the users. The main objective of this work is to develop a solution to interact with smart environments, such as smart homes, through the use of gestures detected by non-intrusive sensors, more specifically radars. As a proof of concept, in the context of interaction with displays installed in the smart home, a prototype was implemented that includes a gesture input modality, as well as output modalities that are updated according to the detected gesture. The gesture input modality is capable of recognizing five different gestures performed by the user at different distances between the user and the radar, based on the data acquired by this sensor. Gesture recognition relies on a model trained using transfer learning and features corresponding to images obtained from radar data. To evaluate this method, a main study was conducted involving twelve subjects. The best results were obtained for a solution dependent on both the user and the distance between the user and the radar (mean accuracy and F1 score between 85% and 95%). Solutions independent of the user or distance proved to be more challenging. However, it proved to be sufficient for practical use when considering several consecutive windows. All results obtained were very useful for implementing the gesture modality prototype and also inform future work related to gesture recognition and interaction with smart environments. |
|---|---|
| Autores principais: | Aguiar, Gonçalo Soares Teixeira de |
| Assunto: | Radar Gesture recognition Human machine interaction Smart homes Transfer learning Non intrusive sensors Distance |
| Ano: | 2024 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade de Aveiro |
| Idioma: | inglês |
| Origem: | RIA - Repositório Institucional da Universidade de Aveiro |
| Resumo: | The environments we live in are becoming increasingly smarter, creating the need of finding suitable ways to interact with them. While interaction modes such as speech, text, and touch remain prevalent, there are associated challenges related to background noise interference, accessibility issues, and reliance on devices (e.g., smartphones, wearables). Gesture-based interaction emerges as a promising alternative or complement to those modes. In this context, radars represent a good option for enabling gesture recognition, since they rely on radio waves to detect moving targets, not presenting the disadvantages of other sensors such as cameras and wearables, which can be considered too intrusive by the users. The main objective of this work is to develop a solution to interact with smart environments, such as smart homes, through the use of gestures detected by non-intrusive sensors, more specifically radars. As a proof of concept, in the context of interaction with displays installed in the smart home, a prototype was implemented that includes a gesture input modality, as well as output modalities that are updated according to the detected gesture. The gesture input modality is capable of recognizing five different gestures performed by the user at different distances between the user and the radar, based on the data acquired by this sensor. Gesture recognition relies on a model trained using transfer learning and features corresponding to images obtained from radar data. To evaluate this method, a main study was conducted involving twelve subjects. The best results were obtained for a solution dependent on both the user and the distance between the user and the radar (mean accuracy and F1 score between 85% and 95%). Solutions independent of the user or distance proved to be more challenging. However, it proved to be sufficient for practical use when considering several consecutive windows. All results obtained were very useful for implementing the gesture modality prototype and also inform future work related to gesture recognition and interaction with smart environments. |
|---|