@article{ART003300108},
author={Wangduk Seo},
title={NA-Search: Differentiable Search of Normalization and Activation},
journal={Journal of The Korea Society of Computer and Information},
issn={1598-849X},
year={2026},
volume={31},
number={1},
pages={59-68}
TY - JOUR
AU - Wangduk Seo
TI - NA-Search: Differentiable Search of Normalization and Activation
JO - Journal of The Korea Society of Computer and Information
PY - 2026
VL - 31
IS - 1
PB - The Korean Society Of Computer And Information
SP - 59
EP - 68
SN - 1598-849X
AB - In this paper, we propose NA-Search, a novel differentiable neural architecture search framework that targets normalization-activation operations to achieve both efficiency and lightweight design in on-device environments. Conventional differentiable neural architecture search methods evaluate all candidate operations during search, resulting in computational and memory overhead that limits applicability on resource-constrained devices. To address this limitation, NA-Search reconstructs the search space around normalization-activation combinations and applies a -sampling that selects only candidates, reducing the computational and memory cost of neural network evaluation. Experiments on CIFAR-10 dataset show that the proposed model achieves 89.75% accuracy with 0.124M parameters, outperforming widely used lightweight neural network in terms of accuracy-efficiency trade-off. Additional analyses show that adjusting the value of during search contributes to improved stability and enhanced final performance, highlighting the effectiveness of the proposed sampling-based search design.
KW - Neural Architecture Search;Lightweight Neural Network Design;On-device Deep Learning;;Differentiable Architecture Search;Operation Sampling Strategy
DO -
UR -
ER -
Wangduk Seo. (2026). NA-Search: Differentiable Search of Normalization and Activation. Journal of The Korea Society of Computer and Information, 31(1), 59-68.
Wangduk Seo. 2026, "NA-Search: Differentiable Search of Normalization and Activation", Journal of The Korea Society of Computer and Information, vol.31, no.1 pp.59-68.
Wangduk Seo "NA-Search: Differentiable Search of Normalization and Activation" Journal of The Korea Society of Computer and Information 31.1 pp.59-68 (2026) : 59.
Wangduk Seo. NA-Search: Differentiable Search of Normalization and Activation. 2026; 31(1), 59-68.
Wangduk Seo. "NA-Search: Differentiable Search of Normalization and Activation" Journal of The Korea Society of Computer and Information 31, no.1 (2026) : 59-68.
Wangduk Seo. NA-Search: Differentiable Search of Normalization and Activation. Journal of The Korea Society of Computer and Information, 31(1), 59-68.
Wangduk Seo. NA-Search: Differentiable Search of Normalization and Activation. Journal of The Korea Society of Computer and Information. 2026; 31(1) 59-68.
Wangduk Seo. NA-Search: Differentiable Search of Normalization and Activation. 2026; 31(1), 59-68.
Wangduk Seo. "NA-Search: Differentiable Search of Normalization and Activation" Journal of The Korea Society of Computer and Information 31, no.1 (2026) : 59-68.
