论文标题
多个子网络假设：通过隔离馈送神经网络中的任务特定子网络来启用多域学习
The Multiple Subnetwork Hypothesis: Enabling Multidomain Learning by Isolating Task-Specific Subnetworks in Feedforward Neural Networks
论文作者
论文摘要
在过去的十年中，神经网络在使用和研究中发生了爆炸，尤其是在计算机视觉和自然语言处理领域。但是，直到最近，神经网络的进步才能超出狭窄的应用程序的性能改进，并转化为扩展的多任务模型，能够跨多种数据类型和模式概括。同时，已经表明，神经网络被过度参数化，并且修剪技术被证明能够显着减少网络中的主动权重的数量，同时在很大程度上保留性能。在这项工作中，我们确定了一种方法和网络表示结构，该结构允许修剪的网络使用以前未使用的权重来学习后续任务。我们在众所周知的基准数据集上采用这些方法来测试目的，并表明使用我们的方法训练的网络能够学习多个任务，这些任务可能是相关的或无关的，并在不牺牲任何任务或表现出灾难性遗忘的情况下并行或顺序的。
Neural networks have seen an explosion of usage and research in the past decade, particularly within the domains of computer vision and natural language processing. However, only recently have advancements in neural networks yielded performance improvements beyond narrow applications and translated to expanded multitask models capable of generalizing across multiple data types and modalities. Simultaneously, it has been shown that neural networks are overparameterized to a high degree, and pruning techniques have proved capable of significantly reducing the number of active weights within the network while largely preserving performance. In this work, we identify a methodology and network representational structure which allows a pruned network to employ previously unused weights to learn subsequent tasks. We employ these methodologies on well-known benchmarking datasets for testing purposes and show that networks trained using our approaches are able to learn multiple tasks, which may be related or unrelated, in parallel or in sequence without sacrificing performance on any task or exhibiting catastrophic forgetting.
