论文标题
在多元宇宙分析中了解和支持调试工作流程
Understanding and Supporting Debugging Workflows in Multiverse Analysis
论文作者
论文摘要
多元宇宙分析是一种统计分析的范式，该范式同时考虑了所有合理分析选择的组合，有望提高透明度和可重复性。尽管最近的工具可帮助分析师指定多元宇宙分析，但在实践中仍然难以使用它们。在这项工作中，由于运行分析到检测错误的延迟以及诊断错误所需的元数据处理规模，我们将调试视为关键障碍。为了应对这些挑战，我们原型命令行界面工具Multiverse Debugger，该工具有助于诊断多元宇宙中的错误并传播修复程序。在定性实验室研究（n = 13）中，我们使用Multiverse调试器作为调查，以开发调试工作流程并确定特定挑战的模型，包括难以理解多元宇宙组成的困难。我们以对未来多元宇宙分析创作系统的设计含义结束。
Multiverse analysis, a paradigm for statistical analysis that considers all combinations of reasonable analysis choices in parallel, promises to improve transparency and reproducibility. Although recent tools help analysts specify multiverse analyses, they remain difficult to use in practice. In this work, we identify debugging as a key barrier due to the latency from running analyses to detecting bugs and the scale of metadata processing needed to diagnose a bug. To address these challenges, we prototype a command-line interface tool, Multiverse Debugger, which helps diagnose bugs in the multiverse and propagate fixes. In a qualitative lab study (n=13), we use Multiverse Debugger as a probe to develop a model of debugging workflows and identify specific challenges, including difficulty in understanding the multiverse's composition. We conclude with design implications for future multiverse analysis authoring systems.
