Efficient Decoupled Neural Architecture Search by Structure and Operation Sampling

Abstract

We propose a novel neural architecture search algorithm via reinforcement learning by decoupling structure and operation search. Our approach samples candidate models from the multinomial distribution over the policy vectors. The proposed technique improves the efficiency of architecture search significantly compared to the existing methods while achieving competitive classification accuracy and model compactness. Our policy vectors are easily interpretable throughout the training procedure, which allows analyzing the search progress and the identified architectures. Note that, on the contrary, the black-box characteristics of the conventional methods based on RNN controllers hamper understanding training progress in terms of policy parameter updates. Our experiments demonstrate the outstanding performance of our approach compared to the state-of-the-art techniques with a fraction of search cost.