Research papers for Research Project
The project aims to develop a real-time, multimodal stress detection system using physiological signals, voice data, and questionnaire inputs, with a focus on practical deployment and explainability. Leveraging the WESAD dataset (excluding respiration), the system processes signals such as EDA, ECG, temperature, and accelerometer data, alongside voice features extracted from a separate speech emotion dataset and responses from psychological questionnaires. Feature extraction is performed using Python libraries like numpy
,pandas
, scipy
, and librosa
, with preprocessing pipelines designed for modularity and reuse. Each modality is processed independently using machine learning models, specifically, Random Forest and XGBoost for physiological and questionnaire data, and potentially convolutional or recurrent neural networks for voice inputs. The outputs of these models are then combined using a late fusion strategy, such as weighted softmax averaging or a meta-classifier, to produce the final stress classification. To ensure transparency, explainable AI (XAI) techniques like SHAP and LIME are applied post-fusion to attribute the final prediction to contributing modalities and input features. The solution will be integrated with custom hardware comprising sensors (for EDA, ECG, and TEMP), a microphone, and an Arduino microcontroller, enabling real-time data collection and testing. This integrated system will also serve as a functional demonstrator during the final presentation, offering both performance and interpretability.
Parameter | How it will be measured | Dataset used |
---|---|---|
Voice | laptop mic | RAVDESS and IEMOCAP |
Questionnaire | google form/website integrated form | WESAD/something else |
ACC(Accelerometer) | MPU6050 | WESAD |
EMG(Electromyography) | EMG sensor | WESAD |
Temperature | MLX90614 | WESAD |
EDA(Electrodermal Activity) | GSR sensor | WESAD |
ECG(Electrocardiogram) | MAX30102 | WESAD |