论文标题
剪辑符合游戏机器学：使用零拍传输学习中的游戏视频中的错误标识
CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning
论文作者
论文摘要
游戏视频包含有关玩家如何与游戏互动以及游戏方式响应的丰富信息。在社交媒体平台（例如Reddit）上共享游戏视频已成为许多玩家的普遍做法。通常，玩家会分享展示视频游戏错误的游戏视频。这样的游戏视频是可以用于游戏测试的软件工件，因为它们为错误分析提供了见解。尽管存在大量的游戏视频存储库，但以有效且结构化的方式解析和挖掘它们仍然是一个巨大的挑战。在本文中，我们提出了一种搜索方法，该方法接受任何英文文本查询作为输入，以从游戏视频的大型存储库中检索相关视频。我们的方法不依赖任何外部信息（例如视频元数据）；它仅根据视频的内容来工作。通过利用对比度语言图像预训练（剪辑）模型的零射击传输功能，我们的方法不需要任何数据标记或培训。为了评估我们的方法，我们介绍了$ \ texttt {gamephysics} $数据集，该数据集由1,873场游戏中的26,954个视频组成，这些视频是从Reddit网站上的GamePhysics部分收集的。我们的方法在我们对简单查询，复合查询和错误查询的广泛分析中显示出令人鼓舞的结果，这表明我们的方法在游戏视频中对对象和事件检测很有用。我们方法的一个示例应用是作为游戏玩法搜索引擎，以帮助复制视频游戏错误。请访问以下链接以获取代码和数据：https：//asgaardlab.github.io/clipxgamephysics/
Gameplay videos contain rich information about how players interact with the game and how the game responds. Sharing gameplay videos on social media platforms, such as Reddit, has become a common practice for many players. Often, players will share gameplay videos that showcase video game bugs. Such gameplay videos are software artifacts that can be utilized for game testing, as they provide insight for bug analysis. Although large repositories of gameplay videos exist, parsing and mining them in an effective and structured fashion has still remained a big challenge. In this paper, we propose a search method that accepts any English text query as input to retrieve relevant videos from large repositories of gameplay videos. Our approach does not rely on any external information (such as video metadata); it works solely based on the content of the video. By leveraging the zero-shot transfer capabilities of the Contrastive Language-Image Pre-Training (CLIP) model, our approach does not require any data labeling or training. To evaluate our approach, we present the $\texttt{GamePhysics}$ dataset consisting of 26,954 videos from 1,873 games, that were collected from the GamePhysics section on the Reddit website. Our approach shows promising results in our extensive analysis of simple queries, compound queries, and bug queries, indicating that our approach is useful for object and event detection in gameplay videos. An example application of our approach is as a gameplay video search engine to aid in reproducing video game bugs. Please visit the following link for the code and the data: https://asgaardlab.github.io/CLIPxGamePhysics/
