APSIPA Transactions on Signal and Information Processing > Vol 10 > Issue 1

Laplacian networks: bounding indicator function smoothness for neural networks robustness

Carlos Lassance, IMT Atlantique, France, cadurosar@gmail.com , Vincent Gripon, IMT Atlantique, France, Antonio Ortega, University of Southern California, USA
 
Suggested Citation
Carlos Lassance, Vincent Gripon and Antonio Ortega (2021), "Laplacian networks: bounding indicator function smoothness for neural networks robustness", APSIPA Transactions on Signal and Information Processing: Vol. 10: No. 1, e2. http://dx.doi.org/10.1017/ATSIP.2021.2

Publication Date: 05 Feb 2021
© 2021 Carlos Lassance, Vincent Gripon and Antonio Ortega
 
Subjects
 
Keywords
Machine learningRobustnessGraph signal processingLaplacianAdversarial attacks
 

Share

Open Access

This is published under the terms of the Creative Commons Attribution licence.

Downloaded: 1588 times

In this article:
I. INTRODUCTION 
II. METHODOLOGY 
III. EXPERIMENTS 
IV. CONCLUSION 

Abstract

For the past few years, deep learning (DL) robustness (i.e. the ability to maintain the same decision when inputs are subject to perturbations) has become a question of paramount importance, in particular in settings where misclassification can have dramatic consequences. To address this question, authors have proposed different approaches, such as adding regularizers or training using noisy examples. In this paper we introduce a regularizer based on the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DL architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness on classical supervised learning vision datasets for various types of perturbations. We also show it can be combined with existing methods to increase overall robustness.

DOI:10.1017/ATSIP.2021.2