@article{ART003370679},
author={Du-Hwan Hur and Deok-Woong Kim and Seung-Hwan Bae},
title={SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization},
journal={Journal of The Korea Society of Computer and Information},
issn={1598-849X},
year={2026},
volume={31},
number={8},
pages={31-40}
TY - JOUR
AU - Du-Hwan Hur
AU - Deok-Woong Kim
AU - Seung-Hwan Bae
TI - SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization
JO - Journal of The Korea Society of Computer and Information
PY - 2026
VL - 31
IS - 8
PB - The Korean Society Of Computer And Information
SP - 31
EP - 40
SN - 1598-849X
AB - In this paper, we propose SADA-DFQ, a generative data-free quantization (DFQ) framework that restores a low-bit network without any access to the original training data, as required in privacy-sensitive domains such as medical imaging and security. Although generator-based DFQ offers high sample diversity, its accuracy still lags behind. We attribute this gap to two factors: an unreliable intermediate-layer transfer signal at low bit-width, where rounding and clipping errors prevent the quantized model from reproducing the full-precision (FP) model's activation magnitudes, and the limited informativeness of the synthetic dataset used for fine-tuning. To address the first, Self-Attention Correlation Distillation (SACD) reformulates each intermediate feature into multi-head self-attention and transfers the relational structure among channel tokens, which tends to be less sensitive to quantization error and thus provides a more stable signal. To address the second, Difficulty-Aware Sample Generation (DASG) re-weights the synthetic dataset by sample difficulty to emphasize examples near the decision boundary. On CIFAR-100, SADA-DFQ attains the best accuracy among the compared generator-based DFQ methods at both 3-bit and 4-bit, improving Top-1 accuracy by about 2.6 percentage points over AdaDFQ at 3-bit.
KW - Model Quantization;Model Compression;Data-free Quantization;Generative model;Knowledge Distillation;Self-Attention;Image Classification
DO -
UR -
ER -
Du-Hwan Hur, Deok-Woong Kim and Seung-Hwan Bae. (2026). SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization. Journal of The Korea Society of Computer and Information, 31(8), 31-40.
Du-Hwan Hur, Deok-Woong Kim and Seung-Hwan Bae. 2026, "SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization", Journal of The Korea Society of Computer and Information, vol.31, no.8 pp.31-40.
Du-Hwan Hur, Deok-Woong Kim, Seung-Hwan Bae "SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization" Journal of The Korea Society of Computer and Information 31.8 pp.31-40 (2026) : 31.
Du-Hwan Hur, Deok-Woong Kim, Seung-Hwan Bae. SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization. 2026; 31(8), 31-40.
Du-Hwan Hur, Deok-Woong Kim and Seung-Hwan Bae. "SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization" Journal of The Korea Society of Computer and Information 31, no.8 (2026) : 31-40.
Du-Hwan Hur; Deok-Woong Kim; Seung-Hwan Bae. SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization. Journal of The Korea Society of Computer and Information, 31(8), 31-40.
Du-Hwan Hur; Deok-Woong Kim; Seung-Hwan Bae. SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization. Journal of The Korea Society of Computer and Information. 2026; 31(8) 31-40.
Du-Hwan Hur, Deok-Woong Kim, Seung-Hwan Bae. SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization. 2026; 31(8), 31-40.
Du-Hwan Hur, Deok-Woong Kim and Seung-Hwan Bae. "SADA-DFQ: Self-Attention Correlation Distillation and Difficulty-Aware Sample Generation for Generative Data-Free Quantization" Journal of The Korea Society of Computer and Information 31, no.8 (2026) : 31-40.
