Publicação

Open-Source Language Models for News Classification: Implementing Small Models in Low-Resource Environments

Ver documento

Detalhes bibliográficos
Resumo:Pre-Trained Models (PTMs) or Large Language Models (LLMs) are deep neural networks trained on vast amounts of text data, enabling them to make predictions based on learned knowledge. Google has played a significant role in this field, particularly through popularizing the Transformers Architecture. However, the landscape evolved dramatically with the release of ChatGPT by OpenAI in November 2022, marking the advent of the universal artificial intelligence era. This event sparked significant interest and efforts in studying LLMs, prompting industries to adapt their operations, software providers to refine their skills, and society to contemplate ethical implications. This research delves into the use of Open-Source LLMs, focusing particularly on text classification — a critical task in Natural Language Processing (NLP). The study employs techniques such as fine-tuning and model quantization, which are essential for leveraging LLMs effectively in practical applications. Key questions addressed include evaluating the comparability of open-source models with established benchmarks across different text classification approaches. The research aims to identify primary challenges and limitations associated with running modern open-source LLMs in lowresource environments. By exploring these topics, the research aims to contribute insights into optimizing the deployment of open-source LLMs, enhancing their accessibility, and addressing practical constraints that affect their widespread adoption across various sectors and applications in NLP.
Autores principais:Figueiredo, Fernando Niglio de
Assunto:Large Language Models (LLM) Pre-trained models (PTM) Text classification News classification Fine-tuning Open Source LoRA QLoRA low-resource environment SDG 8 - Decent work and economic growth
Ano:2024
País:Portugal
Tipo de documento:dissertação de mestrado
Tipo de acesso:acesso aberto
Instituição associada:Universidade Nova de Lisboa
Idioma:inglês
Origem:Repositório Institucional da UNL

Registos relacionados

Registos relacionados