Publicação
Exploration and refinement of language models for applications in the food industry
| Resumo: | This thesis explores the utilization of Large Language Models (LLMs) in the food industry to enhance decision-making processes and improve product safety and quality. Leveraging advancements in Natural Language Processing (NLP), this work aims to develop a Retrieval Augmented Generation (RAG) system. This system interacts intelligently with domain experts by retrieving and generating responses based on curated corpora of scientific literature and regulatory documents. Data sources include PubMed, Code of Federal Regulations (CFR) Title 21 regulations, and other authoritative materials, which were processed and stored in a vector index to enable efficient and contextually relevant retrieval. To assess the performance of the system, a custom dataset, ”LLM Eval”, was created to simulate real-world food engineering queries, allowing for both human and automatic evaluation of model responses. This work also explores the use of rerankers and corrective mechanisms to improve answer relevance and factual accuracy. The result is an intelligent conversational agent capable of supporting industry professionals with informed, fact-based recommendations, aiding in compliance, safety improvements, and sustainable food production practices. |
|---|---|
| Autores principais: | Magalhães, José Pedro Martins |
| Assunto: | RAG AI LLM NLP Transformers |
| Ano: | 2024 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade do Minho |
| Idioma: | inglês |
| Origem: | RepositóriUM - Universidade do Minho |
| Resumo: | This thesis explores the utilization of Large Language Models (LLMs) in the food industry to enhance decision-making processes and improve product safety and quality. Leveraging advancements in Natural Language Processing (NLP), this work aims to develop a Retrieval Augmented Generation (RAG) system. This system interacts intelligently with domain experts by retrieving and generating responses based on curated corpora of scientific literature and regulatory documents. Data sources include PubMed, Code of Federal Regulations (CFR) Title 21 regulations, and other authoritative materials, which were processed and stored in a vector index to enable efficient and contextually relevant retrieval. To assess the performance of the system, a custom dataset, ”LLM Eval”, was created to simulate real-world food engineering queries, allowing for both human and automatic evaluation of model responses. This work also explores the use of rerankers and corrective mechanisms to improve answer relevance and factual accuracy. The result is an intelligent conversational agent capable of supporting industry professionals with informed, fact-based recommendations, aiding in compliance, safety improvements, and sustainable food production practices. |
|---|