The structured selection interview is a valid predictor of job performance, often featuring so-called past-behavior questions, which invite applicants to tell a story about a past work situation. Applicants’ storytelling responses are often suboptimal. Moreover, it is unclear what they reflect: Mastery of a specific competency, or a more general trait (e.g., charisma). Recently, asynchronous digital interviews have emerged, where applicant behaviour (typically nonverbal) is videorecorded and automatically analysed via machine learning algorithms. Applicants perform worse in these interviews than face-to-face. The cause of this difference is poorly understood; moreover, the criterion validity of digital interviews for predicting job performance is unknown. Analysis of verbal behavior is less advanced, being limited by automatic speech recognition (ASR) and standards for defining content for natural language processing (NLP) analyses. Storytelling in response to past-behavior questions provides a yardstick for “good” content, and could be identified by building state-of-the-art ASR and NLP approaches allowing integration of verbal and nonverbal behavior.
This project is an interdisciplinary collaboration between psychologists and computer scientists with three goals: (1) Understanding how storytelling performance reflects the mastery of a specific competency versus more generic abilities (charisma), (2) understanding processes behind performance differences in technology-mediated versus face-to-face interviews, and (3) improving the verbal pipeline for AI-powered digital interviews by improving ASR and NLP. We start by analyzing existing data on how applicants respond to past-behavior questions in digital versus face-to-face interviews (Studies 1 and 2). In the main study (Study 3), participants complete a work sample to measure task performance, and later are asked to tell what they did in a simulated interview, in either a face-to-face or a digital condition. Participants’ behavior in the interview and work sample will be transcribed and coded to compute criterion validity. Responses in face-to-face and digital conditions will be compared. The data will be used to build an ASR and to extract storytelling using end-to-end machine learning. The planned research will make key contributions to personnel selection research and computer science, in terms of scientific publications, software development and training and education.