Publication
IoT home automation with mobile devices vision
| Summary: | In recent years, the number of Internet of Things devices has grown rapidly, in particular devices used in a smart home context. The home automation market has been hampered by an endless number of protocols, often difficult to integrate, especially when new devices are attached to the system. Also, the integration of mobile devices is incomplete, namely when considering the integration and use of sensor information, in its camera, which can be collected from the device. To make this integration more effective, we developed a solution where it’s possible, through a mobile device, to add and control new Internet of Things devices. This solution, supported by the OpenHAB open-source platform, will use the camera of the user's mobile device as an input to identify which object the user is trying to control. For this, the work developed uses artificial intelligence, specifically a Siamese network, which has the advantage of sharing the same weights between the two towers, making it a neural network, easy to train and lightweight, suitable for the possibility of operating the model on a mobile device. This system obtained good results when trying to identify objects using either already present images or new images, for those objects. It was also able to correctly adapt to a new class being added to the dataset, without the need to retrain the whole network. Thus, it was proved that it is possible to improve the integration of mobile devices in home automation, innovatively using the camera sensor. |
|---|---|
| Main Authors: | Guerra, Carlos Daniel Simões Jorge |
| Subject: | Internet of things Machine learning Object recognition Smart home Home automation Reconhecimento de objetos Casa inteligente Domótica |
| Year: | 2022 |
| Country: | Portugal |
| Document type: | master thesis |
| Access type: | open access |
| Associated institution: | ISCTE |
| Language: | English |
| Origin: | Repositório ISCTE |
| Summary: | In recent years, the number of Internet of Things devices has grown rapidly, in particular devices used in a smart home context. The home automation market has been hampered by an endless number of protocols, often difficult to integrate, especially when new devices are attached to the system. Also, the integration of mobile devices is incomplete, namely when considering the integration and use of sensor information, in its camera, which can be collected from the device. To make this integration more effective, we developed a solution where it’s possible, through a mobile device, to add and control new Internet of Things devices. This solution, supported by the OpenHAB open-source platform, will use the camera of the user's mobile device as an input to identify which object the user is trying to control. For this, the work developed uses artificial intelligence, specifically a Siamese network, which has the advantage of sharing the same weights between the two towers, making it a neural network, easy to train and lightweight, suitable for the possibility of operating the model on a mobile device. This system obtained good results when trying to identify objects using either already present images or new images, for those objects. It was also able to correctly adapt to a new class being added to the dataset, without the need to retrain the whole network. Thus, it was proved that it is possible to improve the integration of mobile devices in home automation, innovatively using the camera sensor. |
|---|