Publicação
Automated data analysis in quantitative research: Prototyping an automation software for obtaining fast and reliable insights in quantitative research settings
| Resumo: | project results report introduces a work project in which a prototype was built as a “proof of concept” for a data analysis tool that automates data preparation, exploration and modeling tasks within empirical research settings. Empirical research currently relies on manual processing and analysis of data. In order to enhance research efficiency and to support users, this work can be automated. In particular, tasks such as preprocessing, exploring and analyzing data with statistical methods can be simplified using an automated workflow. The comprehensive tool envisioned should react flexibly to a variety of data input and incorporate a wide range of conventional analyses. It should produce a structured and formulated research report of Word (.docx) format to facilitate further user manipulation. It should communicate with the user through an intuitive web tool. Exploring the possibility of such a tool, which is the scope of this work project, included the programming of a less comprehensive “proof of concept” tool. This prototype performs, upon the exploration and preprocessing of the data, linear regression with OLS estimation on a cross-sectional data set. The prototype was successfully developed and tested on multiple data sets. It is ready to support users without programming skills or access to proprietary software in preprocessing and exploring their data and finding relationships between different variables. Every step of the analysis is explained to the user in the comprehensive automatically generated output report. The prototype provides a basis on which the functionality can be extended towards other use cases such as time series tasks as well as towards offering access to the application of more complex algorithms. |
|---|---|
| Autores principais: | Wahrenburg, Gero |
| Assunto: | Research automation Automation AutoML Research process Automated data mining |
| Ano: | 2023 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade Nova de Lisboa |
| Idioma: | inglês |
| Origem: | Repositório Institucional da UNL |
| Resumo: | project results report introduces a work project in which a prototype was built as a “proof of concept” for a data analysis tool that automates data preparation, exploration and modeling tasks within empirical research settings. Empirical research currently relies on manual processing and analysis of data. In order to enhance research efficiency and to support users, this work can be automated. In particular, tasks such as preprocessing, exploring and analyzing data with statistical methods can be simplified using an automated workflow. The comprehensive tool envisioned should react flexibly to a variety of data input and incorporate a wide range of conventional analyses. It should produce a structured and formulated research report of Word (.docx) format to facilitate further user manipulation. It should communicate with the user through an intuitive web tool. Exploring the possibility of such a tool, which is the scope of this work project, included the programming of a less comprehensive “proof of concept” tool. This prototype performs, upon the exploration and preprocessing of the data, linear regression with OLS estimation on a cross-sectional data set. The prototype was successfully developed and tested on multiple data sets. It is ready to support users without programming skills or access to proprietary software in preprocessing and exploring their data and finding relationships between different variables. Every step of the analysis is explained to the user in the comprehensive automatically generated output report. The prototype provides a basis on which the functionality can be extended towards other use cases such as time series tasks as well as towards offering access to the application of more complex algorithms. |
|---|