Publicação
Conversão de esboços de páginas Web para HTML usando aprendizagem automática
| Resumo: | In the last decades, there has been an exponential development in the area of computing, which includes Artificial Intelligence (AI). The development of AI translates into the emergence of programs that replicate the ability to make decisions, perceive and solve problems in a similar way to humans. Today, artificial intelligence is already part of various areas of society, such as security, health, or virtual assistants. This dissertation aimed to develop a Web application that converts graphical interface sketches, elaborated with the Balsamiq Mockups application, into HTML, CSS and Bootstrap code. Converting a Web page sketch into code is a task that developers typically perform. Due to the time consuming of this task, it becomes impossible to devote more time to the application logic. On the other hand, it is a repetitive and tedious task. Two deep neural network models were built, divided into two distinct approaches. The first approach, presenting poor results, uses a convolutional network and two recurring networks, according to an encoder-decoder architecture, similar to image captioning. It also uses a DSL language and a compiler that transforms DSL into code. The second approach is completely different and it is more focused on the spatial component of the addressed task. It uses YOLO model and a layout algorithm that converts the output of YOLO into code. In the same test set, the first approach achieves 71.30% accuracy, while in the second approach it yields 88.28% accuracy. The Web application, which allows the user to upload images and automatically generate HTML, CSS and Bootstrap code, is supported by the YOLO based model as it gives better results. |
|---|---|
| Autores principais: | Bouças, Tiago André Alves |
| Assunto: | Artificial intelligence Neural network Deep Convolutional network Recurring network YOLO Web application Inteligência artificial Rede neuronal Profunda Convolucional Recorrentes Aplicação web |
| Ano: | 2020 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade do Minho |
| Idioma: | português |
| Origem: | RepositóriUM - Universidade do Minho |
| Resumo: | In the last decades, there has been an exponential development in the area of computing, which includes Artificial Intelligence (AI). The development of AI translates into the emergence of programs that replicate the ability to make decisions, perceive and solve problems in a similar way to humans. Today, artificial intelligence is already part of various areas of society, such as security, health, or virtual assistants. This dissertation aimed to develop a Web application that converts graphical interface sketches, elaborated with the Balsamiq Mockups application, into HTML, CSS and Bootstrap code. Converting a Web page sketch into code is a task that developers typically perform. Due to the time consuming of this task, it becomes impossible to devote more time to the application logic. On the other hand, it is a repetitive and tedious task. Two deep neural network models were built, divided into two distinct approaches. The first approach, presenting poor results, uses a convolutional network and two recurring networks, according to an encoder-decoder architecture, similar to image captioning. It also uses a DSL language and a compiler that transforms DSL into code. The second approach is completely different and it is more focused on the spatial component of the addressed task. It uses YOLO model and a layout algorithm that converts the output of YOLO into code. In the same test set, the first approach achieves 71.30% accuracy, while in the second approach it yields 88.28% accuracy. The Web application, which allows the user to upload images and automatically generate HTML, CSS and Bootstrap code, is supported by the YOLO based model as it gives better results. |
|---|