@article{ART003063165},
author={Okjoo Choi and Yukyong Kim},
title={Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning},
journal={Journal of Software Assessment and Valuation},
issn={2092-8114},
year={2024},
volume={20},
number={1},
pages={75-85}
TY - JOUR
AU - Okjoo Choi
AU - Yukyong Kim
TI - Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning
JO - Journal of Software Assessment and Valuation
PY - 2024
VL - 20
IS - 1
PB - Korea Software Assessment and Valuation Society
SP - 75
EP - 85
SN - 2092-8114
AB - Data quality of data-based information technologies such as big data analysis and machine learning directly affects the quality of the entire system. In particular, the properties of the data used to train machine learning models change over time, causing the model to become less accurate or behave differently than it was designed to. This phenomenon is called drift. Drift can occur for a variety of reasons, including data collection issues or market volatility. Data drift is difficult to detect immediately and can lead to inaccurate predictions, compromising business decisions based on it. The actions required to manage drift will depend on the type, extent, and nature of the drift. To take appropriate action, it is important to establish repeatable procedures for identifying drift, controlling and assessing data quality, setting thresholds for drift rates, and configuring proactive warnings. In this paper, we propose a two-step data quality assessment framework that can manage drift problems that occur in machine learning projects through data quality assessment indicators. In addition, evaluation indices and evaluation procedures according to drift type for drift detection are also defined.
KW - Data quality assessment;Data quality metric;Data Drift;Concept Drift
DO -
UR -
ER -
Okjoo Choi and Yukyong Kim. (2024). Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning. Journal of Software Assessment and Valuation, 20(1), 75-85.
Okjoo Choi and Yukyong Kim. 2024, "Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning", Journal of Software Assessment and Valuation, vol.20, no.1 pp.75-85.
Okjoo Choi, Yukyong Kim "Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning" Journal of Software Assessment and Valuation 20.1 pp.75-85 (2024) : 75.
Okjoo Choi, Yukyong Kim. Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning. 2024; 20(1), 75-85.
Okjoo Choi and Yukyong Kim. "Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning" Journal of Software Assessment and Valuation 20, no.1 (2024) : 75-85.
Okjoo Choi; Yukyong Kim. Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning. Journal of Software Assessment and Valuation, 20(1), 75-85.
Okjoo Choi; Yukyong Kim. Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning. Journal of Software Assessment and Valuation. 2024; 20(1) 75-85.
Okjoo Choi, Yukyong Kim. Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning. 2024; 20(1), 75-85.
Okjoo Choi and Yukyong Kim. "Two-steps Data Quality Assessment Methodology for Handling Drift of Machine Learning" Journal of Software Assessment and Valuation 20, no.1 (2024) : 75-85.
