A Multi-Sensor Approach to Automatically Recognize Breaks and Work Activities of Knowledge Workers in Academia

Personal informatics systems for the work environment can help improving workers' well-being and productivity. Using both self-reported data logged manually by the users and information automatically inferred from sensor measurements, such systems may track users' activities at work and help them reflect on their work habits through insightful data visualizations. They can further support interventions like, e.g., blocking distractions during work activities or suggest the user to take a break. The ability to automatically recognize when the user is engaged in a work activity or taking a break is thus a fundamental primitive such systems need to implement. In this paper, we explore the use of data collected from personal devices -- smartwatches, laptops, and smartphones -- to automatically recognize when users are working or taking breaks. We collect a data set of of continuous streams of sensor data captured from personal devices along with labels indicating whether a user is working or taking a break. We use multiple instruments to facilitate the collection of users' self-reported labels and discuss our experience with this approach. We analyse the available data -- 449 labelled activities of nine knowledge workers collected during a typical work week -- using machine learning techniques and show that user-independent models can achieve a (F1 score) of 94% for the identification of work activities and of 69% for breaks, outperforming baseline methods by 5-10 and 12-54 percentage points, respectively.