T.E.A.M.S. (Teamwork for Enhanced Adaptive Machine Synergies)

The rapid rise of machine intelligence offers unprecedented opportunities to create collaborative, better-together teams that combine the strengths of humans and machines. This research will investigate two angles of this problem: from the human perspective, how do people orient toward intelligent machine teammates? And from the machine perspective, how can technologies be designed to better anticipate, interact, and respond to their human teammates? This study is situated in one of the most important task domains where humans and machines must work synergistically: problem-solving and creative thinking. Teams often need to problem-solve and ideate due to process losses when members struggle to share, discover, and integrate disparate information openly and experience production blocking on creative tasks. This project addresses RQ-B.2: What characteristics of humans facilitate successful hybrid human-machine intelligence? from the call for proposals. Focusing on hybrid team decision-making, we will explore the individual and team characteristics that enable people to work effectively in hybrid teams.

Building on our STRONG Cycle 1 project, where we conducted Wizard-of-Oz human-AI team experiments, we now use that data to train an AI teammate using ChatGPT who interacts intelligently and adaptively with teams performing a problem-solving and creative thinking task. In these experiments, we manipulate two factors: task type and team function. We manipulate the task type (2 levels) by having each team perform two tasks with the AI: creative thinking and problem-solving. Next, we manipulate team function (3 levels) as teamwork.

In the teamwork condition, the AI will engage in social facilitation processes designed to increase motivation, regulate affect, and promote member contributions to the team. In the taskwork condition, the AI will provide ideas on the problem or creative task but not engage in team regulation. In the combined teamwork and taskwork conditions, the AI will provide both functions. 

The AI teammate will be based on a pre-trained Large Language Model (LLM) (e.g., OpenAI Davinci, GPT-3.5, or GPT-4). Large language models are trained on several tasks using extremely large natural language datasets but can be fine-tuned to adapt them to specific tasks using custom data (Dodge et al., 2020; Howard & Ruder, 2018). We will use this feature of LLMs to adapt a model to our specific task, an AI chatbot embedded in a human-AI team, using custom transcripts data from Vero, the “Wizard-of-Oz” human impersonating an AI, that we collected in Cycle 1 STRONG.

Investigators: 

• Noshir S. Contractor, PI, Northwestern, social network analysis, Senior Investigator
• Leslie A. DeChurch, co-I, Northwestern, team effectiveness, Senior Investigator
• Anoop A. Javalagi, Post-Doctoral Scholar, Northwestern, team effectiveness, Junior Investigator
• Vsevolod Suschevskiy, PhD student, Northwestern, network analysis & HCI, Junior Investigator
• Javier Garcia, collaborator, Army Research Laboratory, neuroscience, Senior Investigator
• Sean Fitzhugh, collaborator, Army Research Laboratory, network analysis, Senior Investigator

Vsevolod Suschevskiy, Anoop Javalagi, Sean Fitzhugh, Fumika Hoshi, Xamantha Laos Cueva, Javier Garcia, Leslie A. DeChurch, Noshir Contractor (2023, November 10–11). Approaches to Investigating Human-AI Teams: Transition from the Wizard of Oz to LLM-Powered Agents [Conference presentation]. 2023 Conference on Digital Experimentation @ MIT (CODE@MIT), Cambridge, MA, United States.