![[Learning on the Edge: Investigating Boundary Filters in CNNs]](innamorati2019learning-400x300.jpg)
Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra
University College London
Convolutional neural networks (CNNs) handle the case where filters extend beyond the image boundary using several heuristics, such as zero, repeat or mean padding. These schemes are applied in an ad-hoc fashion and, being weakly related to the image content and oblivious of the target task, result in low output quality at the boundary. In this paper, we propose a simple and effective improvement that learns the boundary handling itself. At training time, the network is provided with a separate set of explicit boundary filters. At testing time, we use these filters which have learned to extrapolate features at the boundary in an optimal way for the specific task. Our extensive evaluation, over a wide range of architectural changes (variations of layers, feature channels, or both), shows how the explicit filters result in improved boundary handling. Furthermore, we investigate the efficacy of variations of such boundary filters with respect to convergence speed and accuracy. Finally, we demonstrate an improvement of 5–20% across the board of typical CNN applications (colorization, de-Bayering, optical flow, disparity estimation, and super-resolution). Supplementary material and code can be downloaded from the project page: https://geometry.cs.ucl.ac.uk/projects/2019/investigating-edge/.
Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra. International Journal of Computer Vision (IJCV), 10 pages, Springer, Oct 2019.Carlo Innamorati, Tobias Ritschel, Tim Weyrich, and Niloy J. Mitra. Learning on the edge: Explicit boundary handling in CNNs. International Journal of Computer Vision (IJCV), October 2019.Innamorati, C., Ritschel, T., Weyrich, T., and Mitra, N. J. 2019. Learning on the edge: Explicit boundary handling in CNNs. International Journal of Computer Vision (IJCV) (Oct.).C. Innamorati, T. Ritschel, T. Weyrich, and N. J. Mitra, “Learning on the edge: Explicit boundary handling in CNNs,” International Journal of Computer Vision (IJCV), Oct. 2019. |
![[Learning on the Edge: Explicit Boundary Handling in CNNs]](./../../projects/natural-boundaries/innamorati18learning-128x96.jpg) | Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra. Proc. British Machine Vision Conference (BMVC), 11 pages, Sep 2018. Best Student Paper Honourable Mention Award.Carlo Innamorati, Tobias Ritschel, Tim Weyrich, and Niloy J. Mitra. Learning on the edge: Explicit boundary handling in CNNs. In Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, September 2018. selected for oral presentation.Innamorati, C., Ritschel, T., Weyrich, T., and Mitra, N. J. 2018. Learning on the edge: Explicit boundary handling in CNNs. In Proceedings of the British Machine Vision Conference (BMVC), BMVA Press. selected for oral presentation.C. Innamorati, T. Ritschel, T. Weyrich, and N. J. Mitra, “Learning on the edge: Explicit boundary handling in CNNs,” in Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, Sep. 2018, selected for oral presentation. [Web Page][PDF (8.1 MB)][Suppl. Material (34 MB)][External Project Page][BibTeX] |
We thank our reviewers for their helpful comments. We also thank Paul Guerrero, Aron Monszpart and Tuanfeng Yang Wang for their technical help in setting up and fixing the machines used to carry out the experiments in this work. This work was partially funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No 642841 (DISTRO), by the ERC Starting Grant SmartGeometry (StG-2013-335373), and by the UK Engineering and Physical Sciences Research Council (Grant EP/K023578/1).
