论文标题
OPEVO：张量操作员优化的进化方法
OpEvo: An Evolutionary Method for Tensor Operator Optimization
论文作者
论文摘要
深度神经网络的培训和推理效率高度依赖于张量操作员在硬件平台上的性能。在支持新操作员或硬件平台方面，手动优化张量操作员有局限性。因此，自动优化张量运算符的设备代码配置变得越来越有吸引力。但是，张量操作员优化的当前方法通常由于组合搜索空间而遭受样本效率差。在这项工作中，我们提出了一种新型的进化方法OPEVO，该方法通过基于Q随机步行引入拓扑感知的突变操作来有效地探索张量操作员的搜索空间，以利用搜索空间的拓扑结构来利用拓扑结构。我们的全面实验结果表明，与最先进的方法（SOTA）方法相比，Opevo可以找到最佳的配置，其差异最低，而在试验数量和墙壁通行时间的数量中的差异和最少努力。这项工作的所有代码都可以在线获得。
Training and inference efficiency of deep neural networks highly rely on the performance of tensor operators on hardware platforms. Manually optimizing tensor operators has limitations in terms of supporting new operators or hardware platforms. Therefore, automatically optimizing device code configurations of tensor operators is getting increasingly attractive. However, current methods for tensor operator optimization usually suffer from poor sample-efficiency due to the combinatorial search space. In this work, we propose a novel evolutionary method, OpEvo, which efficiently explores the search spaces of tensor operators by introducing a topology-aware mutation operation based on q-random walk to leverage the topological structures over the search spaces. Our comprehensive experiment results show that compared with state-of-the-art (SOTA) methods OpEvo can find the best configuration with the lowest variance and least efforts in the number of trials and wall-clock time. All code of this work is available online.
