[Learning on the Edge: Investigating Boundary Filters in CNNs]

Learning on the Edge: Investigating Boundary Filters in CNNs

Carlo Innamorati,  Tobias Ritschel,  Tim Weyrich,  Niloy J. Mitra

University College London

Abstract

Convolutional neural networks (CNNs) handle the case where filters extend beyond the image boundary using several heuristics, such as zero, repeat or mean padding. These schemes are applied in an ad-hoc fashion and, being weakly related to the image content and oblivious of the target task, result in low output quality at the boundary. In this paper, we propose a simple and effective improvement that learns the boundary handling itself. At training time, the network is provided with a separate set of explicit boundary filters. At testing time, we use these filters which have learned to extrapolate features at the boundary in an optimal way for the specific task. Our extensive evaluation, over a wide range of architectural changes (variations of layers, feature channels, or both), shows how the explicit filters result in improved boundary handling. Furthermore, we investigate the efficacy of variations of such boundary filters with respect to convergence speed and accuracy. Finally, we demonstrate an improvement of 5–20% across the board of typical CNN applications (colorization, de-Bayering, optical flow, disparity estimation, and super-resolution). Supplementary material and code can be downloaded from the project page: http://geometry.cs.ucl.ac.uk/projects/2019/investigating-edge/.

Citation Style:    Publication

Learning on the Edge: Investigating Boundary Filters in CNNs.
Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra.
International Journal of Computer Vision (IJCV), 10 pages, Springer, Oct 2019.
Carlo Innamorati, Tobias Ritschel, Tim Weyrich, and Niloy J. Mitra. Learning on the edge: Explicit boundary handling in CNNs. International Journal of Computer Vision (IJCV), October 2019.Innamorati, C., Ritschel, T., Weyrich, T., and Mitra, N. J. 2019. Learning on the edge: Explicit boundary handling in CNNs. International Journal of Computer Vision (IJCV) (Oct.).C. Innamorati, T. Ritschel, T. Weyrich, and N. J. Mitra, “Learning on the edge: Explicit boundary handling in CNNs,” International Journal of Computer Vision (IJCV), Oct. 2019.

Related Publication

[Learning on the Edge: Explicit Boundary Handling in CNNs]
Learning on the Edge: Explicit Boundary Handling in CNNs.
Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra.
Proc. British Machine Vision Conference (BMVC), 11 pages, Sep 2018.
Carlo Innamorati, Tobias Ritschel, Tim Weyrich, and Niloy J. Mitra. Learning on the edge: Explicit boundary handling in CNNs. In Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, September 2018. selected for oral presentation.Innamorati, C., Ritschel, T., Weyrich, T., and Mitra, N. J. 2018. Learning on the edge: Explicit boundary handling in CNNs. In Proceedings of the British Machine Vision Conference (BMVC), BMVA Press. selected for oral presentation.C. Innamorati, T. Ritschel, T. Weyrich, and N. J. Mitra, “Learning on the edge: Explicit boundary handling in CNNs,” in Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, Sep. 2018, selected for oral presentation.
[Web Page] [PDF (8.1 MB)] [Supplemental Material (34 MB)] [External Project Page] [BibTeX]

Acknowledgments

We thank our reviewers for their helpful comments. We also thank Paul Guerrero, Aron Monszpart and Tuanfeng Yang Wang for their technical help in setting up and fixing the machines used to carry out the experiments in this work. This work was partially funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No 642841, by the ERC Starting Grant SmartGeometry (StG-2013-335373), and by the UK Engineering and Physical Sciences Research Council (Grant EP/K023578/1).


Disclaimer: The documents contained in these pages are included to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.