Publicação
Retrieval Augmented Generation for Enhanced Enterprise Information Availability: Nova IMS Case Study
| Resumo: | This research aims to improve information retrieval and accessibility in academic institutions by utilizing the power of Large Language Models (LLMs) within the Azure AI Foundry platform to create an innovative Retrieval Augmented Generation (RAG) artifact. The chatbot is designed to support students and Academic Services. It successfully processes natural language queries, retrieves relevant documents, and generate accurate responses grounded in verifiable sources from the university website with URL references. Integrating a retrieval mechanism with generative AI, the RAG approach significantly enhances the accuracy, relevance and transparency of the chatbot’s responses, mitigating the risk of AI hallucinations. The study outcomes demonstrate the artifact’s potential to simplify navigation through the web sources, generate accurate answers, and boost productivity in academic environment. Quantitative assessment demonstrated that the chatbot can provide answers with a high level of similarity, generating responses that are comparable to those provided on the website. Qualitative assessment made by Academic Services highlights the intuitive design of the chatbot and its potential to reduce academic services workload by minimizing unnecessary first line contact between students and Academic Services. However, the study’s finding is tempered by limitations, including reliance on potentially outdated web sources and the use of an older GPT model due to cost constraints. Future improvement should include enabling Academic Services to supplement the knowledge base, implementing incremental data refreshes, and exploring alternative or more advanced LLMs. Addressing these limitations the chatbot’s overall effectiveness and accuracy can be significantly enhanced, offering a more reliable and helpful resource for Nova IMS. |
|---|---|
| Autores principais: | Nosorowski, Jan Jerzy |
| Assunto: | Generative Artificial Intelligence Retrieval Augmented Generation Academic Environment Question Answering Natural Language Processing Azure AI Foundry SDG 4 - Quality education SDG 8 - Decent work and economic growth SDG 9 - Industry, innovation and infrastructure |
| Ano: | 2025 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade Nova de Lisboa |
| Idioma: | inglês |
| Origem: | Repositório Institucional da UNL |
| Resumo: | This research aims to improve information retrieval and accessibility in academic institutions by utilizing the power of Large Language Models (LLMs) within the Azure AI Foundry platform to create an innovative Retrieval Augmented Generation (RAG) artifact. The chatbot is designed to support students and Academic Services. It successfully processes natural language queries, retrieves relevant documents, and generate accurate responses grounded in verifiable sources from the university website with URL references. Integrating a retrieval mechanism with generative AI, the RAG approach significantly enhances the accuracy, relevance and transparency of the chatbot’s responses, mitigating the risk of AI hallucinations. The study outcomes demonstrate the artifact’s potential to simplify navigation through the web sources, generate accurate answers, and boost productivity in academic environment. Quantitative assessment demonstrated that the chatbot can provide answers with a high level of similarity, generating responses that are comparable to those provided on the website. Qualitative assessment made by Academic Services highlights the intuitive design of the chatbot and its potential to reduce academic services workload by minimizing unnecessary first line contact between students and Academic Services. However, the study’s finding is tempered by limitations, including reliance on potentially outdated web sources and the use of an older GPT model due to cost constraints. Future improvement should include enabling Academic Services to supplement the knowledge base, implementing incremental data refreshes, and exploring alternative or more advanced LLMs. Addressing these limitations the chatbot’s overall effectiveness and accuracy can be significantly enhanced, offering a more reliable and helpful resource for Nova IMS. |
|---|