Polarization information cannot be fully utilized in convolutional neural network(CNN) A novel end-to-end CNN framework for polarization vision tasks is proposed. The framework consists of two sub-networks: a polarimetric-parameter-constructing network and a task network.
Pixel-wise operations between polarimetric images are important for
processing polarization information. For the lack of such operations, the
polarization information cannot be fully utilized in convolutional neural
network(CNN). In this paper, a novel end-to-end CNN framework for polarization
vision tasks is proposed, which enables the networks to take full advantage of
polarimetric images. The framework consists of two sub-networks: a
polarization-parameter-constructing network (PPCN) and a task network. PPCN
implements pixel-wise operations between images in the CNN form with 1x1
convolution kernels. It takes raw polarimetric images as input, and outputs
polarization-parametric images to task network so as to complete a vison task.
By training together, the PPCN can learn to provide the most suitable
polarization-parametric images for the task network and the dataset. Taking
faster R-CNN as task network, the experimental results show that compared with
existing methods, the proposed framework achieves much higher
mean-average-precision (mAP) in object detection task