中文标题#
基于潜在 Kronecker 高斯过程的学习曲线预测的连续分半方法
英文标题#
Successive Halving with Learning Curve Prediction via Latent Kronecker Gaussian Processes
中文摘要#
successive halving 是一种流行的超参数优化算法，它将指数级更多的资源分配给有前景的候选者。 然而，该算法通常依赖于中间性能值来做出资源分配决策，这可能导致过早地剪枝那些起步较慢但最终可能成为最佳候选者的模型。 我们研究了是否可以通过基于潜在 Kronecker 高斯过程的学习曲线预测来引导 successive halving，从而克服这一限制。 在一项涉及不同神经网络架构和点击预测数据集的大规模实证研究中，我们将这种预测方法与基于当前性能值的标准方法进行了比较。 我们的实验表明，尽管预测方法表现具有竞争力，但与将更多资源投入到标准方法相比，它并不是帕累托最优的，因为它需要完全观测到的学习曲线作为训练数据。 然而，通过利用现有的学习曲线数据，可以缓解这一缺点。
英文摘要#
Successive Halving is a popular algorithm for hyperparameter optimization which allocates exponentially more resources to promising candidates. However, the algorithm typically relies on intermediate performance values to make resource allocation decisions, which can cause it to prematurely prune slow starters that would eventually become the best candidate. We investigate whether guiding Successive Halving with learning curve predictions based on Latent Kronecker Gaussian Processes can overcome this limitation. In a large-scale empirical study involving different neural network architectures and a click prediction dataset, we compare this predictive approach to the standard approach based on current performance values. Our experiments show that, although the predictive approach achieves competitive performance, it is not Pareto optimal compared to investing more resources into the standard approach, because it requires fully observed learning curves as training data. However, this downside could be mitigated by leveraging existing learning curve data.
文章页面#
基于潜在 Kronecker 高斯过程的学习曲线预测的连续分半方法
PDF 获取#
抖音扫码查看更多精彩内容