Published on Mon Feb 27 2017

Equivariance Through Parameter-Sharing

Siamak Ravanbakhsh, Jeff Schneider, Barnabas Poczos

We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group that acts discretely on the input and output of a standard neural network layer, we show that is equivariant with respect to the action of

0
0
0
Abstract

We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group that acts discretely on the input and output of a standard neural network layer $\phi_{W}: \Re^{M} \to \Re^{N}$, we show that is equivariant with respect to -action iff explains the symmetries of the network parameters . Inspired by this observation, we then propose two parameter-sharing schemes to induce the desirable symmetry on . Our procedures for tying the parameters achieve -equivariance and, under some conditions on the action of , they guarantee sensitivity to all other permutation groups outside .