Determinants of LLM-assisted Decision-Making

LLMs impact decision-making; study identifies factors and interactions for better informed decisions.

Summary:

• The article provides a comprehensive analysis of the determinants influencing LLM-assisted decision-making, categorizing them into technological, psychological, and decision-specific factors. It aims to develop a dependency framework to systematize the interactions between these determinants and provide a comprehensive understanding of their influence on decision-making processes. The methodology for the literature review, the structure of the identified determinants and their interactions, and illustrative scenarios are presented to demonstrate the practical implications of the findings in various application contexts. Additionally, the article discusses the implications of prompt engineering in shaping the interactions between LLMs and decision-makers, emphasizing its significance for improving the transparency, trustworthiness, and capabilities of LLMs in decision-making contexts. The importance of mental models in LLM-assisted decision-making is highlighted, along with the need for clear guidelines and protocols at the organizational level to optimize decision-making processes. The article also acknowledges criticisms and limitations, such as the focus on theoretical and conceptual aspects and the absence of empirical data, and suggests future research directions to address these limitations and bridge the gap between theoretical frameworks and practical applications.

Major Findings:

1. The article categorizes determinants influencing LLM-assisted decision-making into technological, psychological, and decision-specific factors, aiming to develop a comprehensive understanding of their interactions.
2. Prompt engineering plays a crucial role in shaping the interactions between LLMs and decision-makers, with significant implications for improving transparency, trustworthiness, and capabilities of LLMs in decision-making contexts.
3. Mental models of LLMs influence users’ expectations, trust, and reliance, with implications for decision-making, transparency, and explainability.

Analysis and Critique:

• The article provides a comprehensive overview of the determinants influencing LLM-assisted decision-making, laying the groundwork for a detailed exploration of their implications in subsequent sections.
• The section on prompt engineering and mental models offers valuable insights into the implications of technological determinants and user perceptions on decision-making processes.
• The discussion of organizational guidelines and protocols highlights the importance of addressing over-reliance on LLMs at both individual and organizational levels, with practical strategies for mitigating risks and enhancing decision quality.
• The acknowledgment of criticisms and limitations, along with future research directions, demonstrates a critical reflection on the study’s scope and potential areas for further investigation.

Appendix

Model	gpt-3.5-turbo-1106
Date Generated	2024-02-28
Abstract	https://arxiv.org/abs/2402.17385v1
HTML	https://browse.arxiv.org/html/2402.17385v1
Truncated	True
Word Count	27228