Publicação
Automation of machine learning pipelines for anomaly detection challenges
| Resumo: | Machine Learning (ML) and Data Science can solve different real-world problems. Businesses are becoming increasingly interested in these approaches, and as technology evolves, new challenges can be identified, mostly regarding the ML models development, deployment cycle and data cleansing, which can significantly decrease the accuracy and viability of ML software systems. Development and Operations (DevOps) practices have become popular in operating software systems at scale successfully, but they need to be adapted to deliver the best results when applied to ML systems. This led to the emergence of Machine Learning and Operations (MLOps), a development culture specific for ML systems, derived from DevOps principles. What MLOps attempts to address is the unification of the development cycle of ML based software systems while striving for automation and monitoring, in order to allow continuous integration and delivery. With this thesis, the goal is to study different available frameworks and methods for ML systems, in order to develop an automated ML pipeline to ingest and manipulate high volumes of data. A sensorial system, which simulates the interior of a vehicle, gathers enough data to feed the pipeline. Alongside the development of the ML system, a visual interface which allows control over the overall system and its data is created. |
|---|---|
| Autores principais: | Martins, Ricardo Rodrigues |
| Assunto: | Software engineering Machine learning MLOps Model Automation Engenharia de software Aprendizagem automática Modelo Automação |
| Ano: | 2023 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade do Minho |
| Idioma: | inglês |
| Origem: | RepositóriUM - Universidade do Minho |
Registos relacionados
school Automation of machine learning models benchmarking
por: Sá, João Pedro Barros
Publicado em: (2022)
por: Sá, João Pedro Barros
Publicado em: (2022)
article Automating the software verification test process for SIL logic solvers for subsea oil & gas applications
por: Marqués, Ricardo
Publicado em: (2016)
por: Marqués, Ricardo
Publicado em: (2016)
article Machine learning on insurance premium prediction
por: Jesus, Rodrigo M.
Publicado em: (2023)
por: Jesus, Rodrigo M.
Publicado em: (2023)
school Orchestrator selection process for cloud-native machine learning experimentation
por: Sousa, Afonso Rafael Carvalho
Publicado em: (2022)
por: Sousa, Afonso Rafael Carvalho
Publicado em: (2022)
school Accountability in marketing : the impact of marketing automation processes in the measurement of marketing activity performance
por: Fernandes, Mariana Amaral
Publicado em: (2019)
por: Fernandes, Mariana Amaral
Publicado em: (2019)
school Towards MLOps in a Startup Company
por: Ene, Emil Henricsson
Publicado em: (2023)
por: Ene, Emil Henricsson
Publicado em: (2023)
school Automação de testes de um software da área da saúde
por: Silva, Ana Catarina Araújo
Publicado em: (2024)
por: Silva, Ana Catarina Araújo
Publicado em: (2024)
school Conceção e desenvolvimento de uma plataforma para a democratização de APIs
por: Coutinho, André Rodrigues
Publicado em: (2022)
por: Coutinho, André Rodrigues
Publicado em: (2022)
school Automação de testes
por: Ramalho, André Tiago Gonçalves
Publicado em: (2022)
por: Ramalho, André Tiago Gonçalves
Publicado em: (2022)
school Static Analysis for Detection of Defects in Machine Learning Pipelines
por: Silva, Pedro Miguel Alcântara da
Publicado em: (2024)
por: Silva, Pedro Miguel Alcântara da
Publicado em: (2024)
school Bitcoin Anomaly Detection (BAD) - use of machine learning for fraudulent transaction detection
por: Gomes, Nuno Gonçalo Rodrigues Cabral
Publicado em: (2025)
por: Gomes, Nuno Gonçalo Rodrigues Cabral
Publicado em: (2025)
school Developing an automated machine and deep learning framework for protein classification
por: Sousa, Guilherme Lobo
Publicado em: (2024)
por: Sousa, Guilherme Lobo
Publicado em: (2024)
article A Comparison of AutoML Tools for Machine Learning, Deep Learning and XGBoost
por: Ferreira, Luís
Publicado em: (2021)
por: Ferreira, Luís
Publicado em: (2021)
school Predictive modeling, production scheduling, and process automation in a textile company
por: Coelho Lima, Maria Costa Afonso
Publicado em: (2025)
por: Coelho Lima, Maria Costa Afonso
Publicado em: (2025)
school Development of a web clinical management application
por: Cerqueira, Rúben Correia
Publicado em: (2023)
por: Cerqueira, Rúben Correia
Publicado em: (2023)
article Understanding the motivations, challenges, and practices of software rejuvenation
por: Lucas, Walter
Publicado em: (2023)
por: Lucas, Walter
Publicado em: (2023)
school A Hybrid Machine Learning System for Vulnerability Detection in Web Applications
por: Oliveira, Miguel César de Albuquerque
Publicado em: (2024)
por: Oliveira, Miguel César de Albuquerque
Publicado em: (2024)
article An automated and distributed machine learning framework for telecommunications risk management
por: Ferreira, Luís
Publicado em: (2020)
por: Ferreira, Luís
Publicado em: (2020)
school Safety verification for ROS software
por: Santos, André Filipe Faria
Publicado em: (2021)
por: Santos, André Filipe Faria
Publicado em: (2021)
school A generalized pipeline infraestructure for developing multiple ML algorithms
por: Vieira, Catarina Pais
Publicado em: (2024)
por: Vieira, Catarina Pais
Publicado em: (2024)
article Prediction of maintenance equipment failures using automated machine learning
por: Ferreira, Luís
Publicado em: (2021)
por: Ferreira, Luís
Publicado em: (2021)
article A multi-agent system for automated machine learning
por: Fernandes, B.
Publicado em: (2022)
por: Fernandes, B.
Publicado em: (2022)
article Automated cleansing and harmonization of international trade data
por: Oliveira, Sandra
Publicado em: (2021)
por: Oliveira, Sandra
Publicado em: (2021)
article A scalable and automated machine learning framework to support risk management
por: Ferreira, Luís
Publicado em: (2021)
por: Ferreira, Luís
Publicado em: (2021)
school Supervised machine learning techniques in high energy physics
por: Oliveira, António Carlos Pinto
Publicado em: (2022)
por: Oliveira, António Carlos Pinto
Publicado em: (2022)
article Production time prediction for contract manufacturing industries using automated machine learning
por: Sousa, Afonso
Publicado em: (2022)
por: Sousa, Afonso
Publicado em: (2022)
school Generic productization of ML Models
por: Costeira, Gonçalo Nogueira
Publicado em: (2024)
por: Costeira, Gonçalo Nogueira
Publicado em: (2024)
school Empirical software engineering in educational context
por: Alves, Luís Manuel
Publicado em: (2023)
por: Alves, Luís Manuel
Publicado em: (2023)
school Back to programming from Galois connections
por: Pereira, Paulo Ricardo Antunes
Publicado em: (2024)
por: Pereira, Paulo Ricardo Antunes
Publicado em: (2024)
article AI4CITY - An automated machine learning platform for smart cities
por: Pereira, Pedro José
Publicado em: (2023)
por: Pereira, Pedro José
Publicado em: (2023)
school Automating digital marketing processes using Artificial Intelligence and prompt engineering
por: Vieira, Vasco Cerqueira
Publicado em: (2025)
por: Vieira, Vasco Cerqueira
Publicado em: (2025)
school Software de controlo remoto de robô Desktop para aplicação de doseamento de fluidos
por: Gomes, Sérgio Alexandre Barros Ferreira
Publicado em: (2018)
por: Gomes, Sérgio Alexandre Barros Ferreira
Publicado em: (2018)
school Development of a machine learning-based application to predict the toxicity of chemical compounds
por: Rebelo, Diogo da Silva
Publicado em: (2025)
por: Rebelo, Diogo da Silva
Publicado em: (2025)
article A comparison of automated machine learning tools for predicting energy building consumption in smart cities
por: Soares, Daniela
Publicado em: (2023)
por: Soares, Daniela
Publicado em: (2023)
school Robotic process automation e a auditoria financeira : modern framework
por: Calçada, Leonardo Ismael Silva
Publicado em: (2020)
por: Calçada, Leonardo Ismael Silva
Publicado em: (2020)
school EvoSPL: An evolutionary approach for adopting software product lines in the automotive industry
por: Ignaim, Karam
Publicado em: (2021)
por: Ignaim, Karam
Publicado em: (2021)
school Analysis of the impact of remote work on portuguese software professionals during the COVID-19 pandemic
por: Almeida, Ana João Dias de
Publicado em: (2022)
por: Almeida, Ana João Dias de
Publicado em: (2022)
article An automated model based approach to mobile UI specification and development
por: Ribeiro, António Nestor
Publicado em: (2016)
por: Ribeiro, António Nestor
Publicado em: (2016)
school The impact of robotic process automation on On-Demand Grocery Delivery companies
por: Brunner, Alexander Miguel
Publicado em: (2022)
por: Brunner, Alexander Miguel
Publicado em: (2022)
school Robotic process automation : impact and best practices in portuguese banks
por: Carmo, Gonçalo Pires de Carvalho Mota
Publicado em: (2020)
por: Carmo, Gonçalo Pires de Carvalho Mota
Publicado em: (2020)
Registos relacionados
-
school Automation of machine learning models benchmarking
por: Sá, João Pedro Barros
Publicado em: (2022) -
article Automating the software verification test process for SIL logic solvers for subsea oil & gas applications
por: Marqués, Ricardo
Publicado em: (2016) -
article Machine learning on insurance premium prediction
por: Jesus, Rodrigo M.
Publicado em: (2023) -
school Orchestrator selection process for cloud-native machine learning experimentation
por: Sousa, Afonso Rafael Carvalho
Publicado em: (2022) -
school Accountability in marketing : the impact of marketing automation processes in the measurement of marketing activity performance
por: Fernandes, Mariana Amaral
Publicado em: (2019)