论文标题
在现实和语言数据的限制上：将LLM与人类规范保持一致
On Reality and the Limits of Language Data: Aligning LLMs with Human Norms
论文作者
论文摘要
大型语言模型（LLMS）的最新进展正在用于实用应用中广泛的自然语言数据中的语言关联。但是，他们仅使用语言数据理解物理世界的能力仍然是一个问题。在审查了现有协议之后，我们使用新颖且严格控制的推理测试（ART）探讨了这个问题，并将人类规范与GPT-3版本进行比较。我们的发现突出了可以直接从数据和弱点领域学习的常识关系模型的类别。 GPT-3提供了与人类受试者相提并论的多种关系的言语推理的证据，包括同义词，反义词和默认继承，而没有从人类判断中进行强化学习，GPT-3似乎在Hos-part的参考间隔的下端表现出来，并包含。通过必要的质量，大小和强度顺序，在负担性特征中也观察到弱点。将LLM与象征性的世界接地相结合是解决关联学习的有前途的方向。
Recent advancements in Large Language Models (LLMs) harness linguistic associations in vast natural language data for practical applications. However, their ability to understand the physical world using only language data remains a question. After reviewing existing protocols, we explore this question using a novel and tightly controlled reasoning test (ART) and compare human norms against versions of GPT-3. Our findings highlight the categories of common-sense relations models that could learn directly from data and areas of weakness. GPT-3 offers evidence for verbal reasoning on a par with human subjects for several relations including Synonymy, Antonymy, and Default inheritance, Without reinforcement learning from human judgements, it appears GPT-3 performs at the lower end of the reference interval for Has-part and Contained-in. Weaknesses were observed also in affordance characteristics through Necessary-quality, Order-of-size and Order-of-intensity. Combining LLMs with symbolic world grounding is a promising direction to address associative learning.
