论文标题
与Copilot交谈：探索使用自然语言解决CS1问题的及时工程
Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language
论文作者
论文摘要
Github Copilot是一种人工智能模型，用于自然语言问题描述自动生成源代码。自2022年6月以来，Copilot已正式可为所有学生免费提供，以供诸如Visual Studio Code之类的开发环境插入。先前的工作探索openai codex是PACEROT的基本模型，它表明它在典型的CS1问题上表现良好，从而引起了人们对其对介绍性编程课程的影响的担忧。但是，对于副副总裁表现不佳的问题类型，或者学生在解决错误时可能与副驾驶的自然语言相互作用所了解的类型知之甚少。我们通过评估Copilot在166个编程问题的公开数据集中的性能来探讨这些问题。我们发现，它在第一次尝试中成功解决了这些问题的一半，并且仅使用自然语言更改问题描述，可以解决剩余的问题的60％。我们认为，这种类型的及时工程（我们认为它最初失败时它将成为人类和副本人之间的标准互动），是一项潜在有用的学习活动，可促进计算思维技能，并可能改变代码写作技能开发的性质。
GitHub Copilot is an artificial intelligence model for automatically generating source code from natural language problem descriptions. Since June 2022, Copilot has officially been available for free to all students as a plug-in to development environments like Visual Studio Code. Prior work exploring OpenAI Codex, the underlying model that powers Copilot, has shown it performs well on typical CS1 problems thus raising concerns about the impact it will have on how introductory programming courses are taught. However, little is known about the types of problems for which Copilot does not perform well, or about the natural language interactions that a student might have with Copilot when resolving errors. We explore these questions by evaluating the performance of Copilot on a publicly available dataset of 166 programming problems. We find that it successfully solves around half of these problems on its very first attempt, and that it solves 60\% of the remaining problems using only natural language changes to the problem description. We argue that this type of prompt engineering, which we believe will become a standard interaction between human and Copilot when it initially fails, is a potentially useful learning activity that promotes computational thinking skills, and is likely to change the nature of code writing skill development.
