TO-DO

Research papers for Research Project

Summary of previous paper

Overall Description

The project aims to develop a real-time, multimodal stress detection system using physiological signals, voice data, and questionnaire inputs, with a focus on practical deployment and explainability. Leveraging the WESAD dataset (excluding respiration), the system processes signals such as EDA, ECG, temperature, and accelerometer data, alongside voice features extracted from a separate speech emotion dataset and responses from psychological questionnaires. Feature extraction is performed using Python libraries like numpy ,pandas, scipy, and librosa, with preprocessing pipelines designed for modularity and reuse. Each modality is processed independently using machine learning models, specifically, Random Forest and XGBoost for physiological and questionnaire data, and potentially convolutional or recurrent neural networks for voice inputs. The outputs of these models are then combined using a late fusion strategy, such as weighted softmax averaging or a meta-classifier, to produce the final stress classification. To ensure transparency, explainable AI (XAI) techniques like SHAP and LIME are applied post-fusion to attribute the final prediction to contributing modalities and input features. The solution will be integrated with custom hardware comprising sensors (for EDA, ECG, and TEMP), a microphone, and an Arduino microcontroller, enabling real-time data collection and testing. This integrated system will also serve as a functional demonstrator during the final presentation, offering both performance and interpretability.

Parameters upon which we will be working upon

Parameter How it will be measured Dataset used
Voice laptop mic RAVDESS and IEMOCAP
Questionnaire google form/website integrated form WESAD/something else
ACC(Accelerometer) MPU6050 WESAD
EMG(Electromyography) EMG sensor WESAD
Temperature MLX90614 WESAD
EDA(Electrodermal Activity) GSR sensor WESAD
ECG(Electrocardiogram) MAX30102 WESAD

Models to be used