MnasNet: Platform-Aware Neural Architecture Search for Mobile
Designing convolutional neural networks (CNN) for mobile devices is challenging because mobile models need to be small and fast, yet still accurate. Although significant efforts have been dedicated to design and improve mobile CNNs on all dimensions, it is very difficult to manually balance these trade-offs when there are so many architectural possibilities to consider.
Also cited · not yet reviewed (7)
- NASNet2017 · cited 13×, 5 in Method“As modern CNN models become increasingly deeper and larger inceptionresnet17; senet18; nas_imagenet18; amoebanets18, they also become slower, and require more computation.”From this paper · §Introduction
- MobileNetV22018 · cited 10×, 3 in Method“Compared to the MobileNetV2 mobilenetv218, our model improves the ImageNet accuracy by 3.0% with similar latency on the Google Pixel phone.”From this paper · §Introduction
- MobileNets2017 · cited 5×, 1 in Method“Given restricted computational resources available on mobile devices, much recent research has focused on designing and improving mobile CNN models by reducing the depth of the network and utilizing less expensive operat…”From this paper · §Introduction
- Goyal large-batch SGD2017 · cited 1×, 1 in Method“Following imagenet1hour17, learning rate is increased from 0 to 0.256 in the first 5 epochs, and then decayed by 0.97 every 2.4 epochs.”From this paper · §Experimental Setup
Show 3 more
- PPO2017 · cited 1×, 1 in Method“At the end of each step, the parameters θ\theta of the controller are updated by maximizing the expected reward defined by equation 5 using Proximal Policy Optimization schulman2017proximal.”From this paper · §Mobile Neural Architecture Search
- MS COCO2014 · cited 2דWe apply our proposed approach to ImageNet classification imagenet15 and COCO object detection coco14.”From this paper · §Introduction
- ResNet2015 · cited 2דCompared to the widely used ResNet-50 resnet16, our MnasNet model achieves slightly higher (76.7%) accuracy with 4.8×4.8\times fewer parameters and 𝟏𝟎×10\times fewer multiply-add operations.”From this paper · §Introduction
Led to
- EfficientNet2019 · cited 10×, 6 in Method“Notably, the effectiveness of model scaling heavily depends on the baseline network; to go even further, we use neural architecture search (Zoph & Le 2017; Tan et al. 2019) to develop a new baseline network, and scale it…”From EfficientNet · §Introduction
Abstract
Designing convolutional neural networks (CNN) for mobile devices is challenging because mobile models need to be small and fast, yet still accurate. Although significant efforts have been dedicated to design and improve mobile CNNs on all dimensions, it is very difficult to manually balance these trade-offs when there are so many architectural possibilities to consider. In this paper, we propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporate model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency. Unlike previous work, where latency is considered via another, often inaccurate proxy (e.g., FLOPS), our approach directly measures real-world inference latency by executing the model on mobile phones. To further strike the right balance between flexibility and search space size, we propose a novel factorized hierarchical search space that encourages layer diversity throughout the network. Experimental results show that our approach consistently outperforms state-of-the-art mobile CNN models across multiple vision tasks. On the ImageNet classification task, our MnasNet achieves 75.2% top-1 accuracy with 78ms latency on a Pixel phone, which is 1.8x faster than MobileNetV2 [29] with 0.5% higher accuracy and 2.3x faster than NASNet [36] with 1.2% higher accuracy. Our MnasNet also achieves better mAP quality than MobileNets for COCO object detection. Code is at https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet