中文标题#
HHNAS-AM：使用自適應突變策略的分層混合神經架構搜索
英文标题#
HHNAS-AM: Hierarchical Hybrid Neural Architecture Search using Adaptive Mutation Policies
中文摘要#
神經架構搜索（NAS）由於其能夠發現優於手動設計的架構而引起了廣泛的研究興趣。 學習文本表示對於文本分類和其他語言相關任務至關重要。 在文本分類中使用的 NAS 模型沒有混合分層結構，並且對架構結構沒有限制，因此搜索空間變得非常大且大部分是冗餘的，所以現有的強化學習模型無法有效地導航搜索空間。 此外，進行扁平架構搜索會導致一個無序的搜索空間，難以遍歷。 為此，我們提出了 HHNAS-AM（具有自適應突變策略的分層混合神經架構搜索），這是一種高效探索多樣化架構配置的新方法。 我們引入了一些架構模板來進行搜索，這些模板組織了搜索空間，其中搜索空間是基於領域特定提示設計的。 我們的方法使用基於 Q 學習的突變策略，根據前幾次迭代的性能反饋動態適應，從而更有效地加速搜索空間的遍歷。 所提出的模型是完全概率的，能夠有效地探索搜索空間。 我們在數據庫 id（db_id）預測任務上評估了我們的方法，在多次實驗中它始終能夠發現高性能的架構。 在 Spider 數據集上，我們的方法在測試準確率上比現有基線提高了 8%。
英文摘要#
Neural Architecture Search (NAS) has garnered significant research interest due to its capability to discover architectures superior to manually designed ones. Learning text representation is crucial for text classification and other language-related tasks. The NAS model used in text classification does not have a Hybrid hierarchical structure, and there is no restriction on the architecture structure, due to which the search space becomes very large and mostly redundant, so the existing RL models are not able to navigate the search space effectively. Also, doing a flat architecture search leads to an unorganised search space, which is difficult to traverse. For this purpose, we propose HHNAS-AM (Hierarchical Hybrid Neural Architecture Search with Adaptive Mutation Policies), a novel approach that efficiently explores diverse architectural configurations. We introduce a few architectural templates to search on which organise the search spaces, where search spaces are designed on the basis of domain-specific cues. Our method employs mutation strategies that dynamically adapt based on performance feedback from previous iterations using Q-learning, enabling a more effective and accelerated traversal of the search space. The proposed model is fully probabilistic, enabling effective exploration of the search space. We evaluate our approach on the database id (db_id) prediction task, where it consistently discovers high-performing architectures across multiple experiments. On the Spider dataset, our method achieves an 8% improvement in test accuracy over existing baselines.
文章页面#
PDF 获取#
抖音掃碼查看更多精彩內容