论文标题
通用近似的最小宽度
Minimum Width for Universal Approximation
论文作者
论文摘要
已经研究了与深度结合的网络上的一系列经典通用近似结果，研究了宽大网络的通用近似属性。但是，启用通用近似的关键宽度尚未通过输入尺寸$ d_x $和输出尺寸$ d_y $进行精确表征。在这项工作中，我们使用Relu激活功能为网络提供了第一个确定的结果：$ l^p $函数的通用近似所需的最小宽度正好是$ \ max \ {d_x+1，d_y \} $。我们还证明，与Relu的均匀近似值相同，但确实具有额外的阈值激活函数。我们的证明技术还可以用来通过具有一般激活函数的网络进行通用近似所需的最小宽度来得出更紧密的上限。
The universal approximation property of width-bounded networks has been studied as a dual of classical universal approximation results on depth-bounded networks. However, the critical width enabling the universal approximation has not been exactly characterized in terms of the input dimension $d_x$ and the output dimension $d_y$. In this work, we provide the first definitive result in this direction for networks using the ReLU activation functions: The minimum width required for the universal approximation of the $L^p$ functions is exactly $\max\{d_x+1,d_y\}$. We also prove that the same conclusion does not hold for the uniform approximation with ReLU, but does hold with an additional threshold activation function. Our proof technique can be also used to derive a tighter upper bound on the minimum width required for the universal approximation using networks with general activation functions.
