Published on Tue May 18 2021

Content Disentanglement for Semantically Consistent Synthetic-to-RealDomain Adaptation in Urban Traffic Scenes

Mert Keser, Artem Savkin, Federico Tombari

Deep learning techniques trained on synthetic data encounter dramatic performance drops when they are tested on real data. We propose a new, unsupervised, end-to-end domain adaptation network architecture. We evaluate our architecture on the downstream task of semantic segmentation.

16
0
7
Abstract

Synthetic data generation is an appealing approach to generate novel traffic scenarios in autonomous driving. However, deep learning techniques trained solely on synthetic data encounter dramatic performance drops when they are tested on real data. Such performance drop is commonly attributed to the domain gap between real and synthetic data. Domain adaptation methods have been applied to mitigate the aforementioned domain gap. These methods achieve visually appealing results, but the translated samples usually introduce semantic inconsistencies. In this work, we propose a new, unsupervised, end-to-end domain adaptation network architecture that enables semantically consistent domain adaptation between synthetic and real data. We evaluate our architecture on the downstream task of semantic segmentation and show that our method achieves superior performance compared to the state-of-the-art methods.

Fri Apr 17 2020
Computer Vision
IDDA: a large-scale multi-domain dataset for autonomous driving
Semantic segmentation is key in autonomous driving. Using deep visual progressivelylearning architectures is not trivial in this context. This issue has traditionally been circumvented through the use of synthetic datasets.
0
0
0
Wed May 26 2021
Computer Vision
KLIEP-based Density Ratio Estimation for Semantically Consistent Synthetic to Real Images Adaptation in Urban Traffic Scenes
We demonstrate how adversarial training alone can introduce semantic inconsistencies in translated images. To tackle this issue we propose density prematching strategy using KLIEP-based density ratio estimation.
3
0
0
Sun Oct 27 2019
Machine Learning
Multi-source Domain Adaptation for Semantic Segmentation
0
0
0
Mon Jul 16 2018
Computer Vision
Effective Use of Synthetic Data for Urban Scene Semantic Segmentation
Training a deep network to perform semantic segmentation requires large amounts of labeled data. To alleviate the manual effort of annotating real images, researchers have investigated the use of synthetic data.
0
0
0
Thu Nov 30 2017
Computer Vision
ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban Scenes
The intrinsic domain difference between synthetic and real images causes a significant performance drop when applying the learned model to real world scenarios. We propose a target guided distillation approach to learn the real image style, which is achieved by training the model to imitate a pretrained real style model using real images.
0
0
0
Wed Sep 25 2019
Machine Learning
Synthetic Data for Deep Learning
Synthetic data is an increasingly popular tool for training deep learning models. In this work, we attempt to provide a comprehensive survey of the various directions in the development and application of synthetic data. We also survey the work on improving synthetic data development.
2
0
1
Fri Jun 24 2016
Computer Vision
Coupled Generative Adversarial Networks
Coupled generative adversarial network (CoGAN) can learn a joint distribution without any tuples of corresponding images in different domains. This is achieved by enforcing a weight-sharing-constraint that limits the network capacity.
3
1,389
3,767
Thu Jul 30 2020
Computer Vision
Contrastive Learning for Unpaired Image-to-Image Translation
In image-to-image translation, each patch in the output should reflect the corresponding patch in. the input, independent of domain. We propose a straightforward method for doing so, using a framework based on contrastive learning. We demonstrate that our framework enables one-sided translation in the
1
24
106
Mon Nov 21 2016
Computer Vision
Image-to-Image Translation with Conditional Adversarial Networks
conditional adversarial networks are a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping.
5
2
8
Mon Mar 20 2017
Computer Vision
Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization
Gatys et al. recently introduced a neural algorithm that renders a content image in the style of another image, achieving so-called style transfer. Their framework requires a slow iterative optimization process, which limits its practical application.
2
0
1
Mon Dec 22 2014
Machine Learning
Adam: A Method for Stochastic Optimization
Adam is an algorithm for first-order gradient-based optimization of stochastic objective functions. The method is straightforward to implement and has little memory requirements. It is well suited for problems that are large in terms of data and parameters.
2
0
1
Sun May 28 2017
Computer Vision
Dilated Residual Networks
Dilation increases the resolution of output feature maps without reducing the receptive field of individual neurons. We show that dilated residual networks (DRNs) outperform their non-dilated counterparts in image classification without increasing the model's depth or complexity.
0
0
0