论文标题
重新访问参数有效调整：我们真的在那里吗？
Revisiting Parameter-Efficient Tuning: Are We Really There Yet?
论文作者
论文摘要
许多人认为参数效率调整（petuning）方法是使用验证语言模型（PLM）的新范式。通过与完整的模型芬特相比，只需调整一定数的参数，声称在与Finetuning相当甚至更好的情况下，petuning的方法声称可以实现性能。在这项工作中，我们通过对培训和评估进行首次全面调查来退后一步，重新检查这些脓素方法。我们在当前的研究中发现了有问题的验证和测试实践，伴随着垂体方法的不稳定性，导致了不可靠的结论。在根据真正公平的评估方案进行比较时，pet养不能持续地产生竞争性能，而填充仍然是中等和高资源环境中表现最好的方法。我们更深入地研究了不稳定性的原因，并观察到可训练的参数和训练迭代的数量是两个主要因素：减少可训练的参数和延长的训练迭代可能会导致脓性方法中较高的稳定性。
Parameter-Efficient Tuning (PETuning) methods have been deemed by many as the new paradigm for using pretrained language models (PLMs). By tuning just a fraction amount of parameters comparing to full model finetuning, PETuning methods claim to have achieved performance on par with or even better than finetuning. In this work, we take a step back and re-examine these PETuning methods by conducting the first comprehensive investigation into the training and evaluation of them. We found the problematic validation and testing practice in current studies, when accompanied by the instability nature of PETuning methods, has led to unreliable conclusions. When being compared under a truly fair evaluation protocol, PETuning cannot yield consistently competitive performance while finetuning remains to be the best-performing method in medium- and high-resource settings. We delve deeper into the cause of the instability and observed that the number of trainable parameters and training iterations are two main factors: reducing trainable parameters and prolonging training iterations may lead to higher stability in PETuning methods.
