论文标题
使用认知心理学了解GPT-3
Using cognitive psychology to understand GPT-3
论文作者
论文摘要
我们使用认知心理学的工具研究GPT-3，这是一种最近的大型语言模型。更具体地说，我们评估了GPT-3的决策，信息搜索，审议和因果推理能力，这些能力是文献中的一系列规范实验。我们发现，GPT-3的大部分行为令人印象深刻：它与人类受试者相似或更好地解决基于小插图的任务，能够从描述中做出体面的决定，在多臂匪徒任务中超越人类，并显示基于模型的强化学习的签名。然而，我们还发现，对基于小插图的任务的小扰动可能会导致gpt-3极大地误入歧途，它没有显示出定向探索的签名，并且在因果推理任务中会惨败。这些结果丰富了我们对当前大型语言模型的理解，并为未来的研究使用认知心理学的工具铺平了道路，以研究越来越有能力且不透明的人造药物。
We study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3's decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3's behavior is impressive: it solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multi-armed bandit task, and shows signatures of model-based reinforcement learning. Yet we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. These results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.
