Funded by ARL
Award Proposal Number GRANT13840121
BAA/FOA Number: W911NF- 19-S-0001
The rapid rise of machine intelligence offers unprecedented opportunities to create collaborative, better-together teams that combine the strengths of humans and machines. This research will investigate two angles of this problem: from the human perspective, how do people orient toward intelligent machine teammates? And from the machine perspective, how can technologies be designed to better anticipate, interact, and respond to their human teammates? This study is situated in one of the most important task domains where humans and machines must work synergistically: problem-solving and creative thinking. Teams often need to problem-solve and ideate due to process losses when members struggle to share, discover, and integrate disparate information openly and experience production blocking on creative tasks. This project addresses RQ-B.2: What characteristics of humans facilitate successful hybrid human-machine intelligence? from the call for proposals. Focusing on hybrid team decision-making, we will explore the individual and team characteristics that enable people to work effectively in hybrid teams.
Building on our STRONG Cycle 1 project, where we conducted Wizard-of-Oz human-AI team experiments, we now use that data to train an AI teammate using ChatGPT who interacts intelligently and adaptively with teams performing a problem-solving and creative thinking task. In these experiments, we manipulate two factors: task type and team function. We manipulate the task type (2 levels) by having each team perform two tasks with the AI: creative thinking and problem-solving. Next, we manipulate team function (3 levels) as teamwork.
In the teamwork condition, the AI will engage in social facilitation processes designed to increase motivation, regulate affect, and promote member contributions to the team. In the taskwork condition, the AI will provide ideas on the problem or creative task but not engage in team regulation. In the combined teamwork and taskwork conditions, the AI will provide both functions.
The AI teammate will be based on a pre-trained Large Language Model (LLM) (e.g., OpenAI Davinci, GPT-3.5, or GPT-4). Large language models are trained on several tasks using extremely large natural language datasets but can be fine-tuned to adapt them to specific tasks using custom data (Dodge et al., 2020; Howard & Ruder, 2018). We will use this feature of LLMs to adapt a model to our specific task, an AI chatbot embedded in a human-AI team, using custom transcripts data from Vero, the “Wizard-of-Oz” human impersonating an AI, that we collected in Cycle 1 STRONG.