论文标题
香草喂养神经网络作为动态系统的离散化
Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systems
论文作者
论文摘要
深度学习已在数据科学和自然科学领域提出了重要的应用。一些研究将深层神经网络与动态系统联系起来，但网络结构仅限于残留网络。众所周知，残留网络可以被视为动态系统的数值离散化。在本文中，我们回到了经典的网络结构，并证明香草馈电网络也可能是动态系统的数值离散化，其中网络的宽度等于输入和输出的维度。我们的证明基于泄漏 - Relu函数的属性和求解微分方程的分裂方法的数值技术。我们的结果可以为理解前馈神经网络的近似特性提供新的观点。
Deep learning has made significant applications in the field of data science and natural science. Some studies have linked deep neural networks to dynamic systems, but the network structure is restricted to the residual network. It is known that residual networks can be regarded as a numerical discretization of dynamic systems. In this paper, we back to the classical network structure and prove that the vanilla feedforward networks could also be a numerical discretization of dynamic systems, where the width of the network is equal to the dimension of the input and output. Our proof is based on the properties of the leaky-ReLU function and the numerical technique of splitting method to solve differential equations. Our results could provide a new perspective for understanding the approximation properties of feedforward neural networks.
