Publicação
Automated cleansing and harmonization of international trade data
| Resumo: | Large volumes of data are becoming increasingly available and can be very valuable for the analysis of different phenomena. These data can originate from multiple sources and be recorded in diverse formats, requiring preliminary scrutiny in order to be further used in scientific analyses. This first crucial phase of filtering and cleansing data is usually a cumbersome and time-consuming task, but automated routines can be developed to help researchers. A routine created with the R language is here presented, to screen, harmonize and aggregate international trade data, representing the trade flows between countries for specific products, in a timeframe that covers monthly flows for at least 15 years for most countries. The R script implementing these routines is provided, being easily adapted to other datasets with similar issues. • A step-by-step procedure for cleansing and harmonizing international trade data, using R programming language, is presented • Automated routines are very effective in obtaining robust and filtered data inputs to integrate in scientific models • Spatial and temporal patterns of worldwide trade relations can be explored to enhance our understanding of various associated phenomena |
|---|---|
| Autores principais: | Oliveira, Sandra |
| Outros Autores: | Capinha, César; Rocha, Jorge |
| Assunto: | Automated screening Data harmonization Time-series analysis R software |
| Ano: | 2021 |
| País: | Portugal |
| Tipo de documento: | artigo |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade de Lisboa |
| Idioma: | inglês |
| Origem: | Repositório da Universidade de Lisboa |
Registos relacionados
article A comparison of automated time series forecasting tools for smart cities
por: Pereira, Pedro José
Publicado em: (2022)
por: Pereira, Pedro José
Publicado em: (2022)
article A Benchmark of Automated Multivariate Time Series Forecasting Tools for Smart Cities
por: Pereira, Pedro José
Publicado em: (2025)
por: Pereira, Pedro José
Publicado em: (2025)
assignment A tutorial on using the rminer R package for data mining tasks
por: Cortez, Paulo
Publicado em: (2015)
por: Cortez, Paulo
Publicado em: (2015)
article Single-phase series active conditioner for the compensation of voltage harmonics, sags, swell and flicker
por: Carneiro, H.
Publicado em: (2011)
por: Carneiro, H.
Publicado em: (2011)
article An automated machine learning approach for predicting chemical laboratory material consumption
por: Silva, António João
Publicado em: (2021)
por: Silva, António João
Publicado em: (2021)
school Automation of machine learning models benchmarking
por: Sá, João Pedro Barros
Publicado em: (2022)
por: Sá, João Pedro Barros
Publicado em: (2022)
article Complex Sounds Analysis. An Experimental Approach to Implement in Secondary Level Teaching
por: Cunha, S. M.
Publicado em: (2017)
por: Cunha, S. M.
Publicado em: (2017)
article Evidence of the Long-term Influence of Local Regulations as a Challenge to the International Harmonization of Financial Reporting
por: Albuquerque, Fábio
Publicado em: (2022)
por: Albuquerque, Fábio
Publicado em: (2022)
book Nationalism versus globalization: public sector accounting international harmonization and national resistance
por: Jorge, Susana
Publicado em: (2020)
por: Jorge, Susana
Publicado em: (2020)
article Big Data as an emerging paradigm in organisations' management: a bibliometric analysis
por: Gonçalves, Sidalina
Publicado em: (2024)
por: Gonçalves, Sidalina
Publicado em: (2024)
article Data supporting the role of enzymes and polysaccharides during cassava postharvest physiological deterioration
por: Uarrota, Virgílio Gavicho
Publicado em: (2016)
por: Uarrota, Virgílio Gavicho
Publicado em: (2016)
article The need of tax harmonization within the wealth taxation in the European Union
por: Silva, Hugo Manuel Flores
Publicado em: (2016)
por: Silva, Hugo Manuel Flores
Publicado em: (2016)
school Integração de métodos preditivos em sistemas de informação na Quidgest
por: Tomás, Daniel Filipe Monteiro
Publicado em: (2018)
por: Tomás, Daniel Filipe Monteiro
Publicado em: (2018)
article Data cleansing for indoor positioning Wi-Fi fingerprinting datasets
por: Quezada-Gaibor, Darwin
Publicado em: (2022)
por: Quezada-Gaibor, Darwin
Publicado em: (2022)
article Harmonized classification of forest types in the Iberian Peninsula based on National Forest Inventories
por: Nunes, Leónia
Publicado em: (2020)
por: Nunes, Leónia
Publicado em: (2020)
article Comparison of manual and automated methods of liquid-based cytology. A morphologic study
por: Alves, Venancio Avancini Ferreira
Publicado em: (2004)
por: Alves, Venancio Avancini Ferreira
Publicado em: (2004)
article Spatial-temporal modellization of the NO2 concentration data through geostatistical tools
por: Menezes, Raquel
Publicado em: (2016)
por: Menezes, Raquel
Publicado em: (2016)
school Automation of machine learning pipelines for anomaly detection challenges
por: Martins, Ricardo Rodrigues
Publicado em: (2023)
por: Martins, Ricardo Rodrigues
Publicado em: (2023)
school Forecasting na previsão da incidência de pneumonia em Portugal Continental
por: Veloso, Sara Isabel Ferreira
Publicado em: (2017)
por: Veloso, Sara Isabel Ferreira
Publicado em: (2017)
book Time series analysis: recent advances, new perspectives and applications
por: Rocha, Jorge
Publicado em: (2024)
por: Rocha, Jorge
Publicado em: (2024)
school Pairs trading : cointegration-based methods : applied to the cryptocurrency market
por: Carvalho, Daniel da Silva
Publicado em: (2021)
por: Carvalho, Daniel da Silva
Publicado em: (2021)
article Combining Genetic Algorithms, Neural Networks and Data Filtering for Time Series Forecasting
por: Neves, José
Publicado em: (1998)
por: Neves, José
Publicado em: (1998)
book Introductory chapter: time series analysis
por: Viana, Cláudia
Publicado em: (2024)
por: Viana, Cláudia
Publicado em: (2024)
article Automating the software verification test process for SIL logic solvers for subsea oil & gas applications
por: Marqués, Ricardo
Publicado em: (2016)
por: Marqués, Ricardo
Publicado em: (2016)
article Strong enhancement of second harmonic generation in 2-methyl-4-nitroaniline nanofibers
por: Isakov, D. V.
Publicado em: (2012)
por: Isakov, D. V.
Publicado em: (2012)
article Meteorological time series: an exploratory statistical and critical analysis
por: Gonçalves, A. Manuela
Publicado em: (2023)
por: Gonçalves, A. Manuela
Publicado em: (2023)
school Sobre a Definição de Outlier no Domínio Específico dos Modelos Lineares e Séries Temporais
por: Jorge, Ana Maria Nabais
Publicado em: (1999)
por: Jorge, Ana Maria Nabais
Publicado em: (1999)
article Improving cities sustainability through the use of data mining in a context of big city data
por: Carlos Costa
Publicado em: (2015)
por: Carlos Costa
Publicado em: (2015)
article Predicting waiting time in customer queuing systems
por: Carvalho, André
Publicado em: (2016)
por: Carvalho, André
Publicado em: (2016)
school Is global regulatory harmonization possible for cosmetics?
por: Ferreira, Mariana Gonçalves
Publicado em: (2022)
por: Ferreira, Mariana Gonçalves
Publicado em: (2022)
article Time series forecasting using Holt-Winters exponential smoothing: an application to economic data
por: Lima, Susana
Publicado em: (2019)
por: Lima, Susana
Publicado em: (2019)
draft Robust filtering with quantile regression
por: Assunção, João Borges
Publicado em: (2022)
por: Assunção, João Borges
Publicado em: (2022)
book Regional agricultural production statistics for 160 years using the geographic information system and the spatial analytical technique
por: Viana, Cláudia M.
Publicado em: (2019)
por: Viana, Cláudia M.
Publicado em: (2019)
school Improvement of data quality through outlier detection in time series
por: Alves, Bruno Miguel Pinheiro
Publicado em: (2024)
por: Alves, Bruno Miguel Pinheiro
Publicado em: (2024)
article Arylthienyl-vinyl-benzothiazoles as efficient second harmonic generators (SHG) for nonlinear optics
por: Batista, Rosa Maria Ferreira
Publicado em: (2018)
por: Batista, Rosa Maria Ferreira
Publicado em: (2018)
category Arylthienyl-vinyl-benzothiazoles as efficient second harmonic generators (SHG) for nonlinear optics
por: Batista, Rosa Maria Ferreira
Publicado em: (2018)
por: Batista, Rosa Maria Ferreira
Publicado em: (2018)
school Accountability in marketing : the impact of marketing automation processes in the measurement of marketing activity performance
por: Fernandes, Mariana Amaral
Publicado em: (2019)
por: Fernandes, Mariana Amaral
Publicado em: (2019)
article Genetic and evolutionary algorithms for time series forecasting
por: Cortez, Paulo
Publicado em: (2001)
por: Cortez, Paulo
Publicado em: (2001)
article Time series analysis by state space models applied to a water quality data in Portugal
por: Gonçalves, A. Manuela
Publicado em: (2018)
por: Gonçalves, A. Manuela
Publicado em: (2018)
article Deep learning for supervised classification of temporal data in ecology
por: Capinha, César
Publicado em: (2021)
por: Capinha, César
Publicado em: (2021)
Registos relacionados
-
article A comparison of automated time series forecasting tools for smart cities
por: Pereira, Pedro José
Publicado em: (2022) -
article A Benchmark of Automated Multivariate Time Series Forecasting Tools for Smart Cities
por: Pereira, Pedro José
Publicado em: (2025) -
assignment A tutorial on using the rminer R package for data mining tasks
por: Cortez, Paulo
Publicado em: (2015) -
article Single-phase series active conditioner for the compensation of voltage harmonics, sags, swell and flicker
por: Carneiro, H.
Publicado em: (2011) -
article An automated machine learning approach for predicting chemical laboratory material consumption
por: Silva, António João
Publicado em: (2021)