论文标题
您会问什么机器学习模型？基于人类模型对话的模型解释的用户需求识别
What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversations
论文作者
论文摘要
最近，我们看到可解释的人工智能领域的数量增加。令我们惊讶的是，他们的开发是由模型开发人员驱动的，而不是对人类最终用户需求的研究。对需求的分析（如果完成）进行了A/B测试的形式，而不是对开放问题的研究。要回答一个问题：“人类操作员会想问什么ML模型？”我们提出了一个对话系统，以解释预测模型的决策。在这项实验中，我们开发了一个名为DR_ANT的聊天机器人，讨论了经过训练的机器学习模型，以预测泰坦尼克号的生存赔率。人们可以与Dr_ant谈论模型的不同方面，以了解其预测背后的基本原理。收集了1000多次对话的语料库后，我们分析了用户想提出的最常见类型的问题。据我们所知，这是第一项使用对话系统从预测模型的互动和迭代对话探索中收集人类运营商的需求。
Recently we see a rising number of methods in the field of eXplainable Artificial Intelligence. To our surprise, their development is driven by model developers rather than a study of needs for human end users. The analysis of needs, if done, takes the form of an A/B test rather than a study of open questions. To answer the question "What would a human operator like to ask the ML model?" we propose a conversational system explaining decisions of the predictive model. In this experiment, we developed a chatbot called dr_ant to talk about machine learning model trained to predict survival odds on Titanic. People can talk with dr_ant about different aspects of the model to understand the rationale behind its predictions. Having collected a corpus of 1000+ dialogues, we analyse the most common types of questions that users would like to ask. To our knowledge, it is the first study which uses a conversational system to collect the needs of human operators from the interactive and iterative dialogue explorations of a predictive model.
