New method can scale zero-shot direct inversion to deep architectures and complex datasets. Inversion of generators in GANs unveils code of given synthesized face image at 128x128px.
Understanding the behavior and vulnerability of pre-trained deep neural
networks (DNNs) can help to improve them. Analysis can be performed via
reversing the network's flow to generate inputs from internal representations.
Most existing work relies on priors or data-intensive optimization to invert a
model, yet struggles to scale to deep architectures and complex datasets. This
paper presents a zero-shot direct model inversion framework that recovers the
input to the trained model given only the internal representation. The crux of
our method is to inverse the DNN in a divide-and-conquer manner while
re-syncing the inverted layers via cycle-consistency guidance with the help of
synthesized data. As a result, we obtain a single feed-forward model capable of
inversion with a single forward pass without seeing any real data of the
original task. With the proposed approach, we scale zero-shot direct inversion
to deep architectures and complex datasets. We empirically show that modern
classification models on ImageNet can, surprisingly, be inverted, allowing an
approximate recovery of the original 224x224px images from a representation
after more than 20 layers. Moreover, inversion of generators in GANs unveils
latent code of a given synthesized face image at 128x128px, which can even, in
turn, improve defective synthesized images from GANs.