ARNE: In-Vehicle Affect Recognition and Regulation

In recent years, there has been a massive increase in the number of people affected by mental health issues. Over the last decade, major depressive disorders rapidly increased by 18.4% to 320 million cases per year worldwide and anxiety disorders by 14.9% to 264 million cases. Beyond the loss of quality of life for the individual, the resulting economic burden of these diseases is enormous because society is confronted with productivity losses and high treatment costs. Given that less than half of all mental disorders are detected and treated, the World Health Organization has described the situation as alarming. Hence, prevention of mental illnesses has received increased attention in recent years. The demand for large-scale preventive interventions exceeds the available capacity of existing forms of treatments in psychotherapy, which usually involve personal interaction. Novel interventions, however, ubiquitously carried out via the internet or on mobile devices, may provide supplementary, cost-effective forms of support and offer the advantage of being independent of time and place. Many of them aim to reach the masses to prevent diseases before they arise and before longer-lasting harm materializes, for example by strengthening resilience factors such as well-being. The World Health Organization strongly emphasizes the relevance of prevention. As a matter of fact, they define health not only as the absence of disease, but also as “a state of complete physical, social,and emotional well-being“. And indeed, well-being is recognized as a cornerstone to reduce the vulnerability of suffering from mental disorders.

In light of the depicted challenges, psychotherapy extends its reach by bringing interventions into everyday life situations (e.g., by app-based solutions). In this context, existing research has recognized the potential of in-vehicle interventions by exploring novel types of interventions that can be applied in the car without compromising driver safety, such as guided breathing exercises or personalized voice assistants. Commuters in particular spend considerable time in the car that might be used for such interventions. In Germany, for example, approximately 60% of employees commute by car every day. Among these, over 20% spend at least one hour per day in their car. Even higher figures apply for the US, where 57 million employees (38% of the workforce) commute at least one hour per workday. Yet, most existing research in the domain still focuses on laboratory studies or, at most, on real-world settings under highly controlled conditions (i.e., short periods of time or predefined routes). This differs substantially from our research, as we inspect the impact of mental health and well-being interventions in the real world, i.e., on public roads and over a longer period of time. Therefore, we also explore whether AI in vehicles can be leveraged for analyzing people’s health status by monitoring and evaluating their driving behaviors.


Koch, K., Mishra, V., Liu, S., Berger, T., Fleisch, E., Kotz, D.F., Wortmann, F., When Do Drivers Interact with In-Vehicle Well-being Interventions? An Exploratory Analysis of a Longitudinal Study on Public Roads, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 1, Article 19, 30 pages. [PDF]

Liu, S., Koch, K., Zhou, Z., Föll, S., He, X., Menke, T., Fleisch, E., Wortmann, F., The Empathetic Car: Exploring Emotion Inference via Driver Behaviour and Traffic Context, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 5, No. 3, Article 117. Publication date: September 2021. [PDF]

Koch, K., Tiefenbeck, V., Liu, S., Berger, T., Fleisch, E., Wortmann, F., Taking Mental Health & Well-Being to the Streets: An Exploratory Evaluation of In-Vehicle Interventions in the Wild , In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21). Article 539, 1–15. [PDF]

Koch, K., Liu, S., Berger, T., Wortmann, F., Towards the Healing Car: Investigating the Potential of Psychotherapeutic In-Vehicle Interventions, European Conference on Information Systems – Research in Progress (ECIS 2020). [PDF]

Liu, S., Koch, K., Zhou, Z., Maritsch, M., He, X., Fleisch, E., Wortmann, F., Towards Non-Intrusive Camera-Based Heart Rate Variability Estimation in the Car under Naturalistic Condition, IEEE Internet of Things Journal, 10.1109/JIOT.2021.3131742. [PDF]

Share this post

Get in Touch

Shu Liu
Shu LiuResearcher
Kevin Koch
Kevin KochResearcher