Publicação
Configuring and executing ETL tasks on GRID environments - requirements and specificities
| Resumo: | Data Warehouses store integrated and consistent data in a subject-oriented data repository dedicated especially to support business intelligence processes. Nevertheless, in order to maintain a data warehouse up-to-date, data intensive tasks retrieve regularly specialized information from specific preselected information sources, transforming and conforming it accordingly to some specific business requirements provided by decision-makers. Such tasks, commonly named as Extract-Transform-Load (ETL) processes, have a limited time frame window to be executed over an ever increasing amount of data with extremely complex operations. The common approach to deal with the need of more computational power is the acquisition of new and more powerful hardware. This expensive approach disregards the unused computational resources available in desktop computers already present at most enterprises’ computational environments. This paper intends to define a different approach to deal with ETL processes, taking advantage of parallel processing over a GRID environment using XML data as an effective support to data storage and communication, demonstrating that GRID environments could be a real alternative for the implementation of low cost data warehouses. |
|---|---|
| Autores principais: | Santos, Vasco |
| Outros Autores: | Oliveira, Bruno; Silva, Rui; Belo, O. |
| Assunto: | Data Warehousing ETL GRID Parallel processing Relational algebra |
| Ano: | 2011 |
| País: | Portugal |
| Tipo de documento: | comunicação em conferência |
| Tipo de acesso: | acesso restrito |
| Instituição associada: | Universidade do Minho |
| Idioma: | inglês |
| Origem: | RepositóriUM - Universidade do Minho |
Registos relacionados
article Using relational algebra on the specification of real world ETL processes
por: Santos, Vasco
Publicado em: (2015)
por: Santos, Vasco
Publicado em: (2015)
article Using relational algebra on the specification of real world ETL processes
por: Santos, Vasco
Publicado em: (2015)
por: Santos, Vasco
Publicado em: (2015)
article Modeling ETL data quality enforcement tasks using relational algebra operators
por: Santos, Vasco
Publicado em: (2013)
por: Santos, Vasco
Publicado em: (2013)
article Distribuição de tarefas ETL em ambientes GRID
por: Santos, Vasco
Publicado em: (2011)
por: Santos, Vasco
Publicado em: (2011)
article Modelling ETL conciliation tasks using relational algebra operators
por: Santos, Vasco
Publicado em: (2014)
por: Santos, Vasco
Publicado em: (2014)
article Towards a low cost ETL system
por: Santos, Vasco
Publicado em: (2014)
por: Santos, Vasco
Publicado em: (2014)
school A relational algebra approach to ETL modeling
por: Santos, Vasco Nuno Caio dos
Publicado em: (2015)
por: Santos, Vasco Nuno Caio dos
Publicado em: (2015)
article An ETL pattern for log configuration and analysis
por: Oliveira, Bruno
Publicado em: (2019)
por: Oliveira, Bruno
Publicado em: (2019)
article ETL patterns on YAWL: towards to the specification of platform-independent data warehousing populating processes
por: Oliveira, Bruno
Publicado em: (2014)
por: Oliveira, Bruno
Publicado em: (2014)
article A process mining approach for discovering ETL black points
por: Belo, Orlando
Publicado em: (2017)
por: Belo, Orlando
Publicado em: (2017)
article Colored Petri nets in the simulation of ETL standard tasks: the surrogate key pipelining case
por: Silva, Diogo
Publicado em: (2012)
por: Silva, Diogo
Publicado em: (2012)
article ETL logs under a pattern-oriented approach
por: Oliveira, Bruno
Publicado em: (2021)
por: Oliveira, Bruno
Publicado em: (2021)
article Approaching ETL processes specification using a pattern-based ontology
por: Oliveira, Bruno
Publicado em: (2017)
por: Oliveira, Bruno
Publicado em: (2017)
article Task clustering on ETL systems – A pattern-oriented approach
por: Oliveira, Bruno
Publicado em: (2015)
por: Oliveira, Bruno
Publicado em: (2015)
article A domain-specific language for ETL patterns specification in Data Warehousing Systems
por: Oliveira, Bruno
Publicado em: (2015)
por: Oliveira, Bruno
Publicado em: (2015)
article Slowly changing dimensions specification: a relational algebra approach
por: Santos, Vasco
Publicado em: (2009)
por: Santos, Vasco
Publicado em: (2009)
article Approaching ETL conceptual modelling and validation using BPMN and BPEL
por: Oliveira, Bruno
Publicado em: (2013)
por: Oliveira, Bruno
Publicado em: (2013)
article ETL development using patterns: A service-oriented approach
por: Oliveira, Bruno
Publicado em: (2019)
por: Oliveira, Bruno
Publicado em: (2019)
article Using BPMN for ETL conceptual modelling: A case study
por: Oliveira, Bruno
Publicado em: (2021)
por: Oliveira, Bruno
Publicado em: (2021)
article Especificação do comportamento de padrões ETL
por: Gomes, Cláudia
Publicado em: (2014)
por: Gomes, Cláudia
Publicado em: (2014)
article An ontology for describing ETL patterns behavior
por: Oliveira, Bruno
Publicado em: (2016)
por: Oliveira, Bruno
Publicado em: (2016)
article Using relational algebra on the specification of slowly changing dimensions: a first step
por: Santos, Vasco
Publicado em: (2011)
por: Santos, Vasco
Publicado em: (2011)
article Using a domain-specific language to enrich ETL schemas
por: Belo, O.
Publicado em: (2015)
por: Belo, O.
Publicado em: (2015)
article Modeling agents working on ETL processes
por: Gomes, Nuno André Barbosa
Publicado em: (2016)
por: Gomes, Nuno André Barbosa
Publicado em: (2016)
school Transformação de especificações ETL em YAWL para processos Kettle
por: Freitas, Janine Marlene Duarte Silva
Publicado em: (2019)
por: Freitas, Janine Marlene Duarte Silva
Publicado em: (2019)
article Using reo on ETL conceptual modelling: a first approach
por: Oliveira, Bruno
Publicado em: (2013)
por: Oliveira, Bruno
Publicado em: (2013)
article Allocating data cubes on a grid environment: endorsing an alternative approach
por: Pinto, Miguel
Publicado em: (2013)
por: Pinto, Miguel
Publicado em: (2013)
school Especificação e validação de processos ETL em Alloy
por: Capelo, Mariana Almeida Brandão
Publicado em: (2018)
por: Capelo, Mariana Almeida Brandão
Publicado em: (2018)
article BPMN patterns for ETL conceptual modelling and validation
por: Oliveira, Bruno
Publicado em: (2012)
por: Oliveira, Bruno
Publicado em: (2012)
article Modeling an opportunistic ETL agent based system using YAWL
por: Gomes, Nuno André Barbosa
Publicado em: (2016)
por: Gomes, Nuno André Barbosa
Publicado em: (2016)
article Pattern-based ETL conceptual modelling
por: Oliveira, Bruno
Publicado em: (2013)
por: Oliveira, Bruno
Publicado em: (2013)
article Determining the quality of execution of a data warehousing populating system
por: Dias, Nuno
Publicado em: (2017)
por: Dias, Nuno
Publicado em: (2017)
article Designing safe and reliable ETL systems using alloy
por: Capelo, Mariana
Publicado em: (2018)
por: Capelo, Mariana
Publicado em: (2018)
article Automatic generation of ETL physical systems from BPMN conceptual models
por: Belo, O.
Publicado em: (2015)
por: Belo, O.
Publicado em: (2015)
article On the specification of extract, transform, and load patterns behavior: A domain-specific language approach
por: Oliveira, Bruno
Publicado em: (2017)
por: Oliveira, Bruno
Publicado em: (2017)
article On the conceptualization of ETL patterns: a Reo approach
por: Oliveira, Bruno
Publicado em: (2014)
por: Oliveira, Bruno
Publicado em: (2014)
article ETL standard processes modelling: a novel BPMN approach
por: Oliveira, Bruno
Publicado em: (2013)
por: Oliveira, Bruno
Publicado em: (2013)
school Mapper: An Efficient Data Transformation Operator
por: Carreira, Paulo J.F.
Publicado em: (2008)
por: Carreira, Paulo J.F.
Publicado em: (2008)
article No need to type slowly changing dimensions
por: Santos, Vasco
Publicado em: (2011)
por: Santos, Vasco
Publicado em: (2011)
article Geração automática de esqueletos para sistemas ETL
por: Guimarães, Miguel
Publicado em: (2014)
por: Guimarães, Miguel
Publicado em: (2014)
Registos relacionados
-
article Using relational algebra on the specification of real world ETL processes
por: Santos, Vasco
Publicado em: (2015) -
article Using relational algebra on the specification of real world ETL processes
por: Santos, Vasco
Publicado em: (2015) -
article Modeling ETL data quality enforcement tasks using relational algebra operators
por: Santos, Vasco
Publicado em: (2013) -
article Distribuição de tarefas ETL em ambientes GRID
por: Santos, Vasco
Publicado em: (2011) -
article Modelling ETL conciliation tasks using relational algebra operators
por: Santos, Vasco
Publicado em: (2014)