Published on Fri Jun 11 2021

SimSwap: An Efficient Framework For High Fidelity Face Swapping

Renwang Chen, Xuanhong Chen, Bingbing Ni, Yanhao Ge

We propose an efficient framework, called Simple Swap (SimSwap), aiming for generalized and high fidelity face swapping. In contrast to previous approaches that either lack the ability to generalize to arbitrary identity or fail to preserve attributes like facial expression and gaze direction.

3
61
218
Abstract

We propose an efficient framework, called Simple Swap (SimSwap), aiming for generalized and high fidelity face swapping. In contrast to previous approaches that either lack the ability to generalize to arbitrary identity or fail to preserve attributes like facial expression and gaze direction, our framework is capable of transferring the identity of an arbitrary source face into an arbitrary target face while preserving the attributes of the target face. We overcome the above defects in the following two ways. First, we present the ID Injection Module (IIM) which transfers the identity information of the source face into the target face at feature level. By using this module, we extend the architecture of an identity-specific face swapping algorithm to a framework for arbitrary face swapping. Second, we propose the Weak Feature Matching Loss which efficiently helps our framework to preserve the facial attributes in an implicit way. Extensive experiments on wild faces demonstrate that our SimSwap is able to achieve competitive identity performance while preserving attributes better than previous state-of-the-art methods. The code is already available on github: https://github.com/neuralchen/SimSwap.

Tue May 12 2020
Computer Vision
DeepFaceLab: A simple, flexible and extensible face swapping framework
DeepFaceLab is an open-source deepfake system created by \textbf{iperov}. It provides an imperative and easy-to-use pipeline for people to use with no knowledge of deep learning.
3
0
0
Sat Jun 26 2021
Artificial Intelligence
ShapeEditer: a StyleGAN Encoder for Face Swapping
0
0
0
Fri Nov 30 2018
Computer Vision
FSNet: An Identity-Aware Generative Model for Image-based Face Swapping
This paper presents FSNet, a deep generative model for image-based face swapping. Traditionally, face-swapping methods are based on three-dimensional morphable models (3DMMs) The proposed method is not required to fit to the state-of-the-art method.
0
0
0
Fri Jun 18 2021
Computer Vision
HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping
HifiFace can well preserve the face shape of the source face and generate photo-realistic results. Unlike other existing face swapping works that only use face recognition model to keep the identity similarity, we propose 3D shape-aware identity.
2
15
87
Sat Mar 27 2021
Computer Vision
Face Transformer for Recognition
0
0
0
Tue Nov 06 2018
Computer Vision
Super-Identity Convolutional Neural Network for Face Hallucination
Face hallucination is a generative task to super-resolve the facial image with low resolution. Human perception of face heavily relies on identity information. Previous face hallucination approaches largely ignore facial identity recovery. This paper proposes Super-Identity Convolutional Neural Network (SICNN)
0
0
0
Wed Dec 12 2018
Neural Networks
A Style-Based Generator Architecture for Generative Adversarial Networks
We propose an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature. The new architecture leads to an automatically learned, unsupervised separation of high-level attributes and stochastic variation in the generated images.
9
3,500
24,206
Wed Feb 11 2015
Machine Learning
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalized achieves the same
2
342
1,163
Fri Sep 28 2018
Machine Learning
Large Scale GAN Training for High Fidelity Natural Image Synthesis
We train Generative Adversarial Networks at the largest scale yet attempted. We find that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick" Our modifications lead to models which set the new state of the art in class-conditional
2
280
906
Fri Mar 31 2017
Machine Learning
Improved Training of Wasserstein GANs
Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. We find that these problems are often due to the use of weight clipping in WGANs. We propose an alternative to clipping weights: penalize the norm of gradient of the critic.
1
3
14
Mon Nov 21 2016
Computer Vision
Image-to-Image Translation with Conditional Adversarial Networks
conditional adversarial networks are a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping.
5
2
8
Mon Mar 20 2017
Computer Vision
Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization
Gatys et al. recently introduced a neural algorithm that renders a content image in the style of another image, achieving so-called style transfer. Their framework requires a slow iterative optimization process, which limits its practical application.
2
0
1