Privacy Policy Disclaimer
  Advanced SearchBrowse




Journal Article

Multitask GANs for Semantic Segmentation and Depth Completion With Cycle Consistency


Zhang,  Chongzhen
External Organizations;

Tang,  Yang
External Organizations;

Zhao,  Chaoqiang
External Organizations;

Sun,  Qiyu
External Organizations;

Ye,  Zhencheng
External Organizations;


Kurths,  Jürgen
Potsdam Institute for Climate Impact Research;

External Ressource
No external resources are shared
Fulltext (public)
There are no public fulltexts stored in PIKpublic
Supplementary Material (public)
There is no public supplementary material available

Zhang, C., Tang, Y., Zhao, C., Sun, Q., Ye, Z., Kurths, J. (2021): Multitask GANs for Semantic Segmentation and Depth Completion With Cycle Consistency. - IEEE Transactions on Neural Networks and Learning Systems, 32, 12, 5404-5415.

Cite as: https://publications.pik-potsdam.de/pubman/item/item_26463
Semantic segmentation and depth completion are two challenging tasks in scene understanding, and they are widely used in robotics and autonomous driving. Although several studies have been proposed to jointly train these two tasks using some small modifications, such as changing the last layer, the result of one task is not utilized to improve the performance of the other one despite that there are some similarities between these two tasks. In this article, we propose multitask generative adversarial networks (Multitask GANs), which are not only competent in semantic segmentation and depth completion but also improve the accuracy of depth completion through generated semantic images. In addition, we improve the details of generated semantic images based on CycleGAN by introducing multiscale spatial pooling blocks and the structural similarity reconstruction loss. Furthermore, considering the inner consistency between semantic and geometric structures, we develop a semantic-guided smoothness loss to improve depth completion results. Extensive experiments on the Cityscapes data set and the KITTI depth completion benchmark show that the Multitask GANs are capable of achieving competitive performance for both semantic segmentation and depth completion tasks.