论文标题
通过神经元对齐的神经网络的安全跨界
Safe Crossover of Neural Networks Through Neuron Alignment
论文作者
论文摘要
使用遗传算法在不断发展神经网络的权重时，主要且在很大程度上未开发的挑战之一是在父网络之间找到明智的交叉操作。确实，天真的跨界导致功能损坏的后代，这些后代不会保留父母的信息。这是因为神经网络对神经元的排列不变，从而导致多种表示相同解决方案的方法。这通常称为竞争惯例问题。在本文中，我们提出了一个两步安全的交叉（SC）操作员。首先，通过计算他们的相关性如何，父母的神经元在功能上是一致的，只有那时，父母才会重新组合。我们比较了两种测量神经元之间关系的方法：成对相关（PWC）和规范相关分析（CCA）。我们通过对馈送前馈神经网络对的重量进行算术交叉来测试MNIST和CIFAR-10上的安全交叉操作员（SC-PWC和SC-CCA）。我们表明，它有效地将信息从父母传输到后代，并在天真的跨界时显着改善。我们的方法在计算上是快速的，可以作为一种更有效地探索健身景观的一种方式，并使安全的跨界在未来的神经进化研究和应用中成为潜在的有前途的操作员。
One of the main and largely unexplored challenges in evolving the weights of neural networks using genetic algorithms is to find a sensible crossover operation between parent networks. Indeed, naive crossover leads to functionally damaged offspring that do not retain information from the parents. This is because neural networks are invariant to permutations of neurons, giving rise to multiple ways of representing the same solution. This is often referred to as the competing conventions problem. In this paper, we propose a two-step safe crossover(SC) operator. First, the neurons of the parents are functionally aligned by computing how well they correlate, and only then are the parents recombined. We compare two ways of measuring relationships between neurons: Pairwise Correlation (PwC) and Canonical Correlation Analysis (CCA). We test our safe crossover operators (SC-PwC and SC-CCA) on MNIST and CIFAR-10 by performing arithmetic crossover on the weights of feed-forward neural network pairs. We show that it effectively transmits information from parents to offspring and significantly improves upon naive crossover. Our method is computationally fast,can serve as a way to explore the fitness landscape more efficiently and makes safe crossover a potentially promising operator in future neuroevolution research and applications.
