Publicação
NOVA IMS Assistant: Enhancing Information Access and Campus Engagement through an Intelligent Chatbot
| Resumo: | Chatbots have revolutionized human-technology interactions with their remarkable capabilities in various applications, offering intuitive and efficient communication solutions for diverse environments, including academic contexts. This work focuses on leveraging the advancements of natural language processing models and chatbots to develop a GPT-3.5- based chatbot enhanced with Retrieval-Augmented Generation tailored for the NOVA IMS community. The chatbot was built using LangChain for construction and Chroma for vector storage, enabling the chatbot to provide accurate and contextually relevant responses. Two custom datasets were created to conduct the evaluation of multiple aspects of the chatbot's performance, including similarity measure for the Retriever, chunking strategies, and prompt templates, which included both manual review and RAGAS. Overall, the chatbot performs well, providing accurate and relevant replies within the Nova IMS settings. Despite this, qualitative analysis revealed areas for improvement, such as incomplete answers and irrelevant information. |
|---|---|
| Autores principais: | Sousa, Joana Pardelha Marcelo |
| Assunto: | Chatbot RAG GPT Natural Language Processing Artificial Intelligence SDG 8 - Decent work and economic growth |
| Ano: | 2024 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade Nova de Lisboa |
| Idioma: | inglês |
| Origem: | Repositório Institucional da UNL |
| Resumo: | Chatbots have revolutionized human-technology interactions with their remarkable capabilities in various applications, offering intuitive and efficient communication solutions for diverse environments, including academic contexts. This work focuses on leveraging the advancements of natural language processing models and chatbots to develop a GPT-3.5- based chatbot enhanced with Retrieval-Augmented Generation tailored for the NOVA IMS community. The chatbot was built using LangChain for construction and Chroma for vector storage, enabling the chatbot to provide accurate and contextually relevant responses. Two custom datasets were created to conduct the evaluation of multiple aspects of the chatbot's performance, including similarity measure for the Retriever, chunking strategies, and prompt templates, which included both manual review and RAGAS. Overall, the chatbot performs well, providing accurate and relevant replies within the Nova IMS settings. Despite this, qualitative analysis revealed areas for improvement, such as incomplete answers and irrelevant information. |
|---|