NSF CISE CRII CPS Project

CRII CPS Publications            Media Attention          Software         Education & Outreach          Undergraduate Research

CRII: CPS: Wearable-Machine Interface Architectures

Acknowledgment: This material is based upon work supported by the National Science Foundation under Grant Number 1755780
Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Project Overview

Video Summary of the CRII CPS Project and Results

Wrist-worn wearable devices provide rich sets of pulsatile physiological data under various modalities and circumstances. An unexploited capability is that the pulsatile physiological time series collected by wrist-worn wearable devices can be used for recovering internal brain dynamics. The goal of this project is to present wearable machine-interface (WMI) architectures related to mental stress and their potential applications for tracking fatigue and arousal states. Decoding brain states using wrist-worn wearables will transform how mental-stress-related diseases are diagnosed and treated. This proposal presents two design classes of WMI architectures related to mental stress: (1) A decoder that using skin conductance data recovers undesired stimuli triggered brain activity from wearables and a controller that delivers intermittent stimulation to reverse the adverse effect of stimuli. (2) A decoder that using cortisol data recovers undesired stimuli-triggered inhibition of cortisol secretion from wearables and a controller that delivers intermittent stimulation to generate the desired cortisol profile. The proposed methods will be validated by analyzing electrodermal activity as well as concurrent cortisol and adrenocorticotropic hormone pulsatile data in the context of mental-stress-related arousal and fatigue.

 

 

CRII CPS Publications (22 Publications)

Journal Papers [10 (7 published/accepted, 3 submitted)], Full-Length Conference Papers [14 (13 published/accepted, 1 submitted)], and Invited One-Page Conference Papers [2].

Journal Articles:

 Amin M.R., and Faghih R.T., “Sparse Deconvolution of Electrodermal Activity via Continuous-Time System Identification,” IEEE Transactions on Biomedical Engineering (2018).

This paper proposes a convex optimization scheme for deconvolution to overcome the non-convexity challenge in system parameter estimation of electrodermal activity. The method is proposed based on the continuous-system identification with modulating function technique with a novel adaptive frequency band selection method. (supplementary_info)

 

Wickramasuriya D.S., and Faghih R.T., “A Bayesian Filtering Approach for Tracking Arousal from Binary and Continuous Skin Conductance Features,” IEEE Transactions on Biomedical Engineering, 2019.  (supplementary info)

 

 

Wickramasuriya DS, Amin MR, and Faghih RT (2019). “Skin Conductance as a Viable Alternative for Closing the Deep Brain Stimulation Loop in Neuropsychiatric Disorders,” Frontiers in Neuroscience 13:780. DOI: 10.3389/fnins.2019.00780

The article demonstrates the viability of using Skin Conductance as a viable option for closing the loop feedback in deep brain stimulation for treating neuropsychiatric patients. The study demonstrates that arousal estimation is possible using the inferred sympathetic nervous activity from the skin conductance signal.

 


Taghvafard, H., Cao, M., Kawano, Y. and Faghih, RT “Design of Intermittent Control for Cortisol Secretion under Time-Varying Demand and Holding Cost Constraints” IEEE Transactions on Biomedical Engineering, 2019.

 

 


Amin M.R., and Faghih R.T., “Robust Inference of Autonomic Nervous System Activation using Skin Conductance Measurements: A Multi-Channel Sparse System Identification Approach,” IEEE Access, 2019.

 


Fekri Azgomi H, and Faghih R.T., “Closed-Loop Cognitive Stress Regulation using Fuzzy Control in Wearable-Machine Interface Architectures,” Submitted.

 

 


Pednekar D, Amin M.R., Fekri Azgomi H, Aschbacher K,  J. Crofford L, and Faghih R.T., “Characterization of Cortisol Dysregulation in Fibromyalgia and Chronic Fatigue Syndromes: A State-Space Approach” IEEE Transactions on Biomedical Engineering, 2020. (supplementary info)

 


Wickramasuriya, D.S., and Faghih R.T., “A Mixed Filter Algorithm for Sympathetic Arousal Tracking from Skin Conductance and Heart Rate Measurements in Pavlovian Fear Conditioning,” Submitted.

 


Genty J. X., Amin M.R., Shaw N. D., Klerman E. B., and Faghih R.T., “Sparse Deconvolution of Pulsatile Growth Hormone Secretion in Children,” Submitted.

 

 

 

Wickramasuriya D.S., and Faghih R.T., “A Marked Point Process Filtering Approach for Tracking Sympathetic Arousal from Skin Conductance” IEEE Access, 2020. Accepted.

 


 

Conference Proceedings:

Amin M.R., and Faghih R.T., “Inferring Autonomic Nervous System Stimulation from Hand and Foot Skin Conductance Measurements,”  52nd Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, 2018.

This paper proposes a state-space model relating skin conductance signals from different locations of the body with a single neural stimuli input from the autonomic nervous system. It also provides a concurrent deconvolution algorithm to recover the neural stimuli from the concurrent recordings of skin conductance data from hand and foot. The deconvolution results can be used as observations to decode mental stress.

 

Wickramasuriya D.S., Qi C., and Faghih R.T., “A State-Space Approach for Detecting Stress from Electrodermal Activity,” the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Honolulu, HI, 2018.  [Video]

This paper relates mental stress to the probability that a skin conductance response appears and decodes the mental stress state. For example, using data from an experiment for studying driver stress, the stress state was decoded during relaxation as well as driving on different road conditions. This could, for instance, be applied to reduce driver stress by automatically playing calming music when stress is detected.

 

Parshi S, Amin M.R., Fekri Azgomi H, and Faghih R.T., “Mental Workload Classification via Hierarchical Latent Dictionary Learning: A Functional Near Infrared Spectroscopy Study,” IEEE-EMBS International Conference on Biomedical and Health Informatics, Chicago, IL, 2019.

 

 

Amin M.R., and Faghih R.T., “Tonic and Phasic Decomposition of Skin Conductance Data: A Generalized-Cross-Validation-Based Block Coordinate Descent Approach,” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany, 2019.

 

 

Wickramasuriya D.S., and Faghih R.T., “A Novel Filter for Tracking Real-World Cognitive Stress using Multi-Time-Scale Point Process Observations,” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany, 2019.

 

 

Fekri Azgomi H, Wickramasuriya D.S., and Faghih R.T., “State-Space Modeling and Fuzzy Feedback Control of Cognitive Stress,” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany, 2019.

 

 

Pednekar D, Amin M.R., Fekri Azgomi H, Aschbacher K,  J. Crofford L, and Faghih R.T., “A System Theoretic Investigation of Cortisol Dysregulation in Fibromyalgia Patients with Chronic  Fatigue,” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany, 2019. 

 

 

 

Wickramasuriya D.S., and Faghih R.T., “A Cortisol-Based Energy Decoder for Investigation of Fatigue in Hypercortisolism,” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany, 2019.

 

 


Fekri Azgomi H, and Faghih R.T., “A Wearable Brain Machine Interface Architecture for Regulation of Energy in Hypercortisolism,” 53rd Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, 2019.

 

 
Badri Ahmadi M, Craik A, Fekri Azgomi H,  Francis J. T.Contreras-Vidal J.L., and Faghih R.T., “Real-Time Seizure State Tracking using Two Channels: A Mixed-Filter Approach,” IEEE Asilomar Conference on Signals, Systems and Computers 2019.

 


Yadav T, Atique Md M, Fekri Azgomi H,  Francis J. T., and Faghih R.T., “Emotional Valence Tracking and Classification via State-Space Analysis of Facial Electromyography,” IEEE Asilomar Conference on Signals, Systems and Computers 2019.

 

Ravindran A.S., Nakagome S., Wickramasuriya D.S., Contreras-Vidal J.L., and Faghih R.T., “Emotion Recognition by Point Process Characterization of Heartbeat Dynamics,” the IEEE-NIH Special Topics Conference on Healthcare Innovations and Point-of-Care Technologies, Bethesda, MD, 2019.

 

 

Wickramasuriya D.S., Tessmer M.K., and Faghih R.T., “Facial Expression-Based Emotion Classification using Electrocardiogram and Respiration Signals,” the IEEE-EMB Special Topics Conference on Healthcare Innovations and Point-of-Care Technologies, Bethesda, MD, 2019.

 

 


Seet M. S., Amin M. R., Abbasi N. I., Hamano J., Bezerianos A., Faghih R.T. , Thakor N. V., Dragomir A., “Olfactory-induced Positive Affect and Autonomous Response as a Function of Hedonic and Intensity Attributes of Fragrances,” Submitted.

 

 

Invited One-Page Conference Papers:

Wickramasuriya D.S. and Faghih R.T., “A Mixed-Filter Algorithm for Arousal Tracking from Galvanic Skin Response and Heart Rate Measurement,” IEEE-EMBS International Conference on Biomedical and Health Informatics, Chicago, IL, 2019.

a


Amin M.R., Wickramasuriya D.S., and Faghih R.T., “Wearable-Machine Interface Architectures for Mental Stress,” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany, 2019. 

 

Selected Media Attention

Brain Stimulation for PTSD Patients, University of Houston Press Release, 2019

Brain stimulation for PTSD patients, Science Daily, 2019

Sweat-Triggered Brain Stimulation Proposed for PTSD, Other Disorders, Psych Central, 2019

Brain stimulation for PTSD patients, EurekaAlert, 2019

Closed-loop brain stimulator based on sweat response for PTSD patients, News Medical Life Sciences, 2019

Brain Stimulation for PTSD Patients, Biospace, 2019

Brain stimulation for PTSD patients, Neuroscience News, 2019

Mood monitor, IEEE Signal Processing Magazine, 2018

Wearable Technology to Track Brain, Predict Illness, University of Houston Press Release, 2018.

Decoding The Brain: A Wrist-worn Wearable Designed To Fight Stress In The Age Of AnxietyFashNerd, a Digital Magazine, 2018.

Wearable technology to track brain, predict illness, EurekAlert, Science News, 2018.

Wearable Devices Explored to Detect Emotional States, Science & Enterprise, 2018.

Texas Researcher Granted Funds to Research Wearable Tech Applications in Mental HealthFyxes.com, Startup and Tech Stories, 2018.

Wearable Technology to Track Brain, Predict Illness, HealthNewsDigest, 2018.

Can I check your watch? I want to see what’s on your mind, MobiHealthNews, 2018.

 

Provisional Patent

R. T. Faghih. Wearable Machine Interface System for Tracking and Controlling Mental Stress Related Brain Dynamics using Wrist-Worn Wearable Devices, 2018. US Patent App. 62/614,070. [Tech Summary]

Software

Deconvolution

Estimation

Closed-Loop Control

Data and Results

Educational and Outreach Activities

Girls Engineering the Future Outreach Event  (2019) – (Video Link)

Girl’s Engineering the Future Outreach Event (2018) – (Video Link)

Educational Videos

Point Processes Applications in Gamma-Ray Bursts and Pulsar Astronomy – Video by Srinidhi Parshi (Spring 2019)

Mind Control Drone Project Video for ECE 5335 – Control Systems Lab II Course by Bach Nguyen, Samantha Havens, Travis Tran, Anujan Murugesu (Fall 2018)

Modal Analysis and Canonical Forms – Video by Xinyue Zhang (Fall 2018))

Linear Quadratic Gaussian Control – Video by Rajarsi Roy (Fall 2018)

Matrix Algebra and Linear Equations – Video by Prasanna Kumar Reddy Gade (Fall 2018)

Video on Stability by Nitin Bhabsar (Fall 2018)

Lyapunov’s Indirect Method and Linking Lyapunov Function to LQR –  Video by Harshitha Kethyreddy (Fall 2018)

Gilbert Realization and Pole Zero Cancellation – Video by Divesh Pednekar

Reference Tracking and McMillan Degree –  Video by Aruj Khandelwal

Undergraduate Research Videos

Cortisol Sensor Video, Research Experience for Undergraduates (REU) Student Video By Mikayla Tessmer (Summer 2018).

Skin Conductance Video, Undergraduate Research Video by Huy Hua and Zanne Solitz (Summer 2018).

Undergraduate Research Posters

Zoch S. R., Amin M.R., and Faghih R.T., “Electromyography Analysis in Emotion Detection and Recognition”. NSF REU Poster Session, University of Houston, Summer 2019.

Tadrous M., Wickramasuriya D.S., and Faghih R.T., “Exploration of Electrocardiography in Emotion Recognition”. SF REU Poster Session, University of Houston, Summer 2019.

Tessmer M.K., Wickramasuriya D.S., and Faghih R.T., “Emotional Valence Classification at High Arousal Using Heartrate Variability and Respiration”. NSF REU Poster Session, University of Houston, Summer 2018.

Hua H., Amin M.R., and Faghih R.T., “A State-space Investigation of Cortisol Alterations in Chronic Fatigue Syndrome”. Undergraduate Research D
ay, University of Houston, Fall 2018.

Undergraduate Senior Design Project (Spring’18 – Fall’18):  Wearable Device to Recognize Attentiveness in the Workplace

Skin Conductance based Attentiveness Monitor Video

Senior Design Project Members:

Nicholas Hermann, Senior Design Project Computer Engineering Undergraduate Student (senior), University of Houston (January-Decmber 2018)

 

 

Huy Hua, Senior Design Project Electrical Engineering Undergraduate Student (senior), University of Houston (January-December 2018)

 

 

 

Zanne Soliz, Senior Design Project Electrical Engineering Undergraduate Student (senior), University of Houston (January-Decmber 2018)

 

 

Christopher Theriot, Senior Design Project Computer Engineering Undergraduate Student (senior), University of Houston (January-Decmber 2018)

Undergraduate Senior Design Project (Spring’19 – Fall’19):  Wearable Cardiac Monitor

 Senior Design Project Members:

Antonio Ruiz, Senior Design Project (January-Decmber 2019), Electrical Engineering Undergraduate Student, University of Houston

 

 

 

Aathira Karippurathu Vijayakumar,  Senior Design Project (January-Decmber 2019), Electrical Engineering Undergraduate Student, University of Houston

 

 

 

Harshitha Gandra, Senior Design Project (January-Decmber 2019), Electrical Engineering Undergraduate Student, University of Houston

 

 

 

Tahsin As Sami, Senior Design Project (January-Decmber 2019), Electrical Engineering Undergraduate Student, University of Houston

Undergraduate Senior Design Project (Spring’19 – Fall’19):  Emotion Recognition and Regulation System

 Senior Design Project Members:

Jin Lu, Senior Design Project (January-Decmber 2019), Electrical Engineering Undergraduate Student, University of Houston

 

 

 

Yuehua Hu, Senior Design Project (January-Decmber 2019), Electrical Engineering Undergraduate Student, University of Houston

 

 

 

 

Yuzhu Zhang, Senior Design Project (January-Decmber 2019), Electrical Engineering Undergraduate Student, University of Houston

 

 

 

 

Zhaohan Liu, Senior Design Project (January-Decmber 2019), Electrical Engineering Undergraduate Student, University of Houston

 

 

 

WCE Internal Business Plan Competition Team: 

 

 

Annie Ngo, Intellectual Property Project (May-December 2019), Wolff Center for Entrepreneurship Undergraduate Student, University of Houston

 

 

 

Cory Gaddis, Intellectual Property Project (May-December 2019), Wolff Center for Entrepreneurship Undergraduate Student, University of Houston

 

 

 

Denah Rooks, Intellectual Property Project (May-December 2019), Wolff Center for Entrepreneurship Undergraduate Student, University of Houston

 

 

 

Kyle Li, Intellectual Property Project (May-December 2019), Wolff Center for Entrepreneurship Undergraduate Student, University of Houston

 

 

 

David Lai, Intellectual Property Project (May-December 2019), Wolff Center for Entrepreneurship Undergraduate Student, University of Houston

 

 

 

Milan Nguyen, Intellectual Property Project (May-December 2019), Valenti School of Communication Undergraduate Student, University of Houston

 

 

Sam Matte, Intellectual Property Project (May-December 2019), Valenti School of Communication Undergraduate Student, University of Houston

 

 

 

Alexandria Dave, Intellectual Property Project (May-December 2019), Valenti School of Communication Undergraduate Student, University of Houston

 

 

Web Outreach

Computational Medicine Lab’s Youtube Channel

K-12 and College Level Mentor on Engineer Girl, a website by the National Academy of Engineering.

Web Analytics