Robustness and Upper Bound of Generalization Error in Deep Neural Networks

Abstract

The generalization of Deep Neural Networks(DNNs) is a crucial issue. While DNNs perform well in many domains, it is hard to explain theoretically why DNNs generalize well. The robustness, introduced in Xu, is a good notion to explain the generalization of DNNs. In this work, we branch off the notion of the robustness and generalization error. In addition, we assume that the input space is a bounded d-dimensional subspace of R^n and derive a new upper bound of the generalization error in DNNs. We attempt to demonstrate our work by using the Jacobian regularizer, suggested by Sokolic, based on the Jacobian matrix in DNNs. Also we visualize the result by using t-SNE.