Published on Sat Jan 19 2019

Synthesizing facial photometries and corresponding geometries using generative adversarial networks

Gil Shamai, Ron Slossberg, Ron Kimmel

Artificial data synthesis is currently a well studied topic with useful applications in data science, computer vision, graphics and many other fields. The study of geometric deep learning is still a debated issue within the academic community as the lack of intrinsic parametrization prohibits the use of convolutional filters.

0
0
0
Abstract

Artificial data synthesis is currently a well studied topic with useful applications in data science, computer vision, graphics and many other fields. Generating realistic data is especially challenging since human perception is highly sensitive to non realistic appearance. In recent times, new levels of realism have been achieved by advances in GAN training procedures and architectures. These successful models, however, are tuned mostly for use with regularly sampled data such as images, audio and video. Despite the successful application of the architecture on these types of media, applying the same tools to geometric data poses a far greater challenge. The study of geometric deep learning is still a debated issue within the academic community as the lack of intrinsic parametrization inherent to geometric objects prohibits the direct use of convolutional filters, a main building block of today's machine learning systems. In this paper we propose a new method for generating realistic human facial geometries coupled with overlayed textures. We circumvent the parametrization issue by imposing a global mapping from our data to the unit rectangle. We further discuss how to design such a mapping to control the mapping distortion and conserve area within the mapped image. By representing geometric textures and geometries as images, we are able to use advanced GAN methodologies to generate new geometries. We address the often neglected topic of relation between texture and geometry and propose to use this correlation to match between generated textures and their corresponding geometries. We offer a new method for training GAN models on partially corrupted data. Finally, we provide empirical evidence demonstrating our generative model's ability to produce examples of new identities independent from the training data while maintaining a high level of realism, two traits that are often at odds.

Sun Dec 03 2017
Computer Vision
GAGAN: Geometry-Aware Generative Adversarial Networks
Geometry-Aware Generative Adversarial Networks (GAGAN) for incorporating geometric information into the image generation process. Experimental results on face generation indicate that the GAGAN can generate realistic images with arbitrary facial attributes.
0
0
0
Fri Feb 15 2019
Computer Vision
GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction
In the past few years, a lot of work has been done towards reconstructing the perceptions3D facial structure from single images. In this paper, we take a radically different approach and harness the power of Generative Adversarial Networks (GANs) and DCNNs.
0
0
0
Mon Mar 25 2019
Computer Vision
MeshGAN: Non-linear 3D Morphable Models of Faces
Generative Adversarial Networks (GANs) are currently the method of choice for generating visual data. We propose the first intrinsic GANs architecture operating directly on 3D meshes. Both quantitative and qualitative results are provided.
0
0
0
Thu Sep 05 2019
Computer Vision
Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networks
Generative Adversarial Networks (GANs) can be used for generating high-quality textures of faces. The process either omits the geometry and normals, or independent processes are used to produce 3D shape information.
0
0
0
Wed Nov 25 2020
Computer Vision
StyleUV: Diverse and High-fidelity UV Map Generative Model
Reconstructing 3D human faces in the wild with the 3D Morphable Model (3DMM) has become popular in recent years. Recent work demonstrates that GANs trained with abundant high-quality UV maps can produce high-fidelity textures superior to those produced by existing methods.
0
0
0
Thu Jun 15 2017
Computer Vision
Recent Progress of Face Image Synthesis
Face synthesis has been a fascinating yet challenging problem in computer vision and machine learning. The main research effort is to design algorithms to generate photo-realistic face images via given semantic domain.
0
0
0