University College London
Convolutional neural networks (CNNs) handle the case where filters extend beyond the image boundary using several heuristics, such as zero, repeat or mean padding. These schemes are applied in an ad-hoc fashion and, being weakly related to the image content and oblivious of the target task, result in low output quality at the boundary. In this paper, we propose a simple and effective improvement that learns the boundary handling itself. At training-time, the network is provided with a separate set of explicit boundary filters. At testing-time, we use these filters which have learned to extrapolate features at the boundary in an optimal way for the specific task. Our extensive evaluation, over a wide range of architectural changes (variations of layers, feature channels, or both), shows how the explicit filters result in improved boundary handling. Consequently, we demonstrate an improvement of 5% to 20% across the board of typical CNN applications (colorization, de-Bayering, optical flow, and disparity estimation).
Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra.
Proc. British Machine Vision Conference (BMVC), 11 pages, Sep 2018.Carlo Innamorati, Tobias Ritschel, Tim Weyrich, and Niloy J. Mitra. Learning on the edge: Explicit boundary handling in CNNs. In Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, September 2018. selected for oral presentation.Innamorati, C., Ritschel, T., Weyrich, T., and Mitra, N. J. 2018. Learning on the edge: Explicit boundary handling in CNNs. In Proceedings of the British Machine Vision Conference (BMVC), BMVA Press. selected for oral presentation.C. Innamorati, T. Ritschel, T. Weyrich, and N. J. Mitra, “Learning on the edge: Explicit boundary handling in CNNs,” in Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, Sep. 2018, selected for oral presentation.
We thank Paul Guerrero, Aron Monszpart and Tuanfeng Yang Wang for their technical help in setting up and fixing the machines used to carry out the experiments in this work. This work was partially funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 642841, by the ERC Starting Grant SmartGeometry (StG-2013-335373), and by the UK Engineering and Physical Sciences Research Council (grant EP/K023578/1).