A Context-Aware Retrieval-Augmented Generation Framework for Enterprise Product Discovery Question Answering

Abstract

Product discovery in e-commerce relies on efficient retrieval algorithms to locate pertinent goods and provide responses to user queries that are contextually relevant. Our research introduces a Context-Aware Retrieval-Augmented Generation (CA-RAG) method for answering business questions and discovering products using the Home Depot Product Search Relevance dataset. The framework combines domain-specific named entity recognition (NER), Sentence-BERT embeddings, FAISS-based semantic retrieval, BM25 keyword search, and entity-based ranking with Reciprocal Rank Fusion (RRF) for enhanced product matching. SmolLM2-1.7B-Instruct and TinyLlama-1.1B-Chat-v1.0 are comparatively assessed in terms of answer generation and factuality. The results demonstrate that SmolLM2 has a retrieval quality score of 0.9328, a generation quality score of 0.7319, and an overall score of 0.7342 when tested on 35 queries, outperforming TinyLlama on these reported scores. SmolLM2 also demonstrates greater faithfulness (0.5870) and a lower hallucination rate (0.4130) than TinyLlama. However, statistical analysis reveals no significant differences between the models in overall system scores or faithfulness. These results show the potential of hybrid retrieval and context-aware language generation techniques to assist in product discovery and personalised product recommendations.