PublishedICDSA 2024 · Springer Nature·2024

Ascend.AI — Building Confidence Through Technology: A Technical Exploration of Facial Expression, Tone, and Pitch Analysis with Chatbot Guidance

Archit RathodGargi SatheSiddh ShahKumkum Saxena
abstract

The proposed approach embarks on an intricate and comprehensive exploration of advanced and innovative technologies to enhance interview skills. Leveraging the power of OpenCV and Xception, the paper delves into the nuances of facial expression analysis, unraveling the intricacies of emotion recognition. The system analyzes tone and pitch with the aid of tools like LIBROSA to extract vocal features in order to understand the intensity of emotions, and has developed a 1D-CNN model for classification using RAVDESS, TESS, SAVEE, and CREMA-D datasets. The system includes a chatbot using vector database Qdrant and an open-source LLM Mixtral 8x7b, offering personalized interview guidance derived from scraping 30 diverse websites for interview-related questions. This technical exploration extends from conventional interview preparation to introducing an innovative framework that intertwines machine learning models with real-time analysis.

keywords
Interview PreparationFacial Expression AnalysisTone AnalysisPitch AnalysisOpenCVXceptionLIBROSA1D-CNNLLMQdrantChatbot