论文标题
神经网络的分布培训和优化
Distributed Training and Optimization Of Neural Networks
论文作者
论文摘要
由于多种因素，深度学习模型的性能越来越好。为了成功，模型可能具有大量参数或复杂的架构，并在大型数据集上进行培训。这导致了对计算资源的巨大要求和时间的转折，甚至在进行高参数优化时（例如，对模型体系结构进行搜索）。尽管这是一个超越粒子物理学的挑战，但我们回顾了并行进行必要计算的各种方法，并将其放在高能量物理学的背景下。
Deep learning models are yielding increasingly better performances thanks to multiple factors. To be successful, model may have large number of parameters or complex architectures and be trained on large dataset. This leads to large requirements on computing resource and turn around time, even more so when hyper-parameter optimization is done (e.g search over model architectures). While this is a challenge that goes beyond particle physics, we review the various ways to do the necessary computations in parallel, and put it in the context of high energy physics.
