Design of a CNN–NLP Based Visual Chatbot for e-Commerce Systems

Abstract

E-commerce platforms have limitations to assist interactive and highly personalized experiences using conventional text or voice-based chatbots, particularly in context-aware areas where product discovery and visual identification are necessary. Hence, we propose to integrate an image as a visual-based chatbot. We present a conceptual framework that combines Convolutional Neural Networks (CNN) for image recognition with Natural Language Processing (NLP) for dynamic, context-aware search in a chatbot. The CNN model enables the recognition of products from user-uploaded images, while the NLP component processes and generates appropriate responses to enhance the shopping experience by providing suggestions. This hybrid system will be developed using the Python programming language, which uses libraries such as Keras, OpenCV, and Flask. By allowing users to interact with a chatbot that uses visual inputs, this system aims to create a more intuitive, personalized e-commerce experience that could lead to high engagement and customer satisfaction. The evaluation metrics are measured in terms of the usage of this system by three users in real-world e-commerce applications. This small-scale test may offer insights into how image-based communication can revolutionize the online shopping experience by analysing usability, interaction efficiency, user satisfaction, engagement behaviour, and system responsiveness in practical scenarios during testing.