Publicação

Synthetic data generation from JSON/XML schemas

Ver documento

Detalhes bibliográficos
Resumo:The objective of this dissertation is the development of an application capable of automatically generating synthetic datasets that are representative and, possibly, very large, directly from JSON and XML schemas, in order to facilitate the testing of software applications and scientific endeavors in areas such as Data Science or Application Development. For this purpose, it is intended to develop a new version of DataGen, an online open-source application that allows the quick prototyping of datasets through its own Domain Specific Language (DSL) of specification of data models. DataGen is able to parse these models and generate synthetic datasets according to the structural and semantic restrictions stipulated, automating the whole process of data generation with spontaneous values created in runtime and/or from a library of support datasets. The objective of this new product, DataGen From Schemas, is to expand DataGen’s use cases and raise the datasets specification’s abstraction level, making it possible to generate synthetic datasets directly from schemas. This new platform builds upon its prior version and acts as its complement, operating jointly and sharing the same data layer, in order to assure the compatibility of both platforms and the portability of the created DSL models between them. Its purpose is to parse schema files and generate corresponding DSL models, effectively translating the JSON or XML specification to a DataGen model, then using the original application as a middleware to generate the final datasets. The present dissertation details the entire creative process behind the development of this application: firstly, it frames the topic of study and its initial phase of investigation, debating relevant technologies and existing related work; then, the ideation phase of the product is addressed, projecting an adequate arquitecture and the reasons behind its design choices, as well as surveying technical requirements for DataGen From Schemas, while taking into account the conclusions reached through prior research; afterwards, the development phase is covered, carefully explaining the elaborated components, their properties and the data flow between them, for both the JSON and XML modules; finally, the reader is presented with conclusions taken from this project’s development and possible future work to implement, in order to improve the current solution.
Autores principais:Cardoso, Hugo André Coelho
Assunto:Schemas JSON XML Data generation Synthetic data DataGen DSL Dataset Grammar Randomization Open source Data science REST API PEG.js Geração de Dados Dados sintéticos Gramática Aleatoriedade Ciência de dados
Ano:2022
País:Portugal
Tipo de documento:dissertação de mestrado
Tipo de acesso:acesso aberto
Instituição associada:Universidade do Minho
Idioma:inglês
Origem:RepositóriUM - Universidade do Minho

Registos relacionados

Registos relacionados