论文标题
非convex复合目标的自适应零好的优化
Adaptive Zeroth-Order Optimisation of Nonconvex Composite Objectives
论文作者
论文摘要
在本文中，我们提出和分析非凸复合目标的零阶优化算法，重点是降低复杂性依赖对维度的依赖性。这是通过使用带有熵函数的随机镜下降方法利用决策集的低维结构来实现的，该方法在配备最大范围的空间中执行梯度下降。为了改善梯度估计，我们用基于Rademacher分布的采样方法替换了经典的高斯平滑法，并表明Mini Batch方法与非欧几里得几何形状相抵抗。为了避免调整超参数，我们分析了通用随机镜下降的自适应步骤，并表明所提出算法的自适应版本收敛而无需对问题进行先验知识。
In this paper, we propose and analyze algorithms for zeroth-order optimization of non-convex composite objectives, focusing on reducing the complexity dependence on dimensionality. This is achieved by exploiting the low dimensional structure of the decision set using the stochastic mirror descent method with an entropy alike function, which performs gradient descent in the space equipped with the maximum norm. To improve the gradient estimation, we replace the classic Gaussian smoothing method with a sampling method based on the Rademacher distribution and show that the mini-batch method copes with the non-Euclidean geometry. To avoid tuning hyperparameters, we analyze the adaptive stepsizes for the general stochastic mirror descent and show that the adaptive version of the proposed algorithm converges without requiring prior knowledge about the problem.
