@article{ART002640790},
author={WeoniI Jeong},
title={Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks},
journal={Journal of Knowledge Information Technology and Systems},
issn={1975-7700},
year={2020},
volume={15},
number={5},
pages={671-681},
doi={10.34163/jkits.2020.15.5.010}
TY - JOUR
AU - WeoniI Jeong
TI - Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks
JO - Journal of Knowledge Information Technology and Systems
PY - 2020
VL - 15
IS - 5
PB - Korea Knowledge Information Technology Society
SP - 671
EP - 681
SN - 1975-7700
AB - Recently, as the real-time data generated by sensors in connection with location information rapidly increases in the IoT environment, studies on various services using a big data platform have been conducted. In this big data platform, research on continuous query processing including data aggregation operation has been proposed to effectively analyze sensor big data constantly flowing for multi-user needs. However, the existing resource-sharing-based aggregation operation has a problem of concurrent execution of time-based and tuple-based queries and an increase in memory usage and deterioration in query processing performance due to duplicated maintenance of aggregate information. Therefore, in this paper, we propose a resource sharing method to effectively support continuous aggregation query processing based on Spark, an in-memory based distributed processing framework. The proposed method minimizes the increase in the cost of processing aggregate information through linear resource sharing based on summary information for the scope of query processing, and reduces the memory usage required for continuous aggregate query processing. Also, our method improves query processing performance by preventing duplicate generation of aggregate information. The proposed approach is implemented based on Spark framework to ensure real-time performance of continuous aggregate query processing for big data. Finally, through performance evaluation, it is shown that the proposed resource sharing technique can be effectively used for continuous aggregation query processing.
KW - Spark framework;Continuous query;Aggregation;Continuous aggregation query;Resource sharing;Big data
DO - 10.34163/jkits.2020.15.5.010
ER -
WeoniI Jeong. (2020). Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks. Journal of Knowledge Information Technology and Systems, 15(5), 671-681.
WeoniI Jeong. 2020, "Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks", Journal of Knowledge Information Technology and Systems, vol.15, no.5 pp.671-681. Available from: doi:10.34163/jkits.2020.15.5.010
WeoniI Jeong "Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks" Journal of Knowledge Information Technology and Systems 15.5 pp.671-681 (2020) : 671.
WeoniI Jeong. Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks. 2020; 15(5), 671-681. Available from: doi:10.34163/jkits.2020.15.5.010
WeoniI Jeong. "Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks" Journal of Knowledge Information Technology and Systems 15, no.5 (2020) : 671-681.doi: 10.34163/jkits.2020.15.5.010
WeoniI Jeong. Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks. Journal of Knowledge Information Technology and Systems, 15(5), 671-681. doi: 10.34163/jkits.2020.15.5.010
WeoniI Jeong. Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks. Journal of Knowledge Information Technology and Systems. 2020; 15(5) 671-681. doi: 10.34163/jkits.2020.15.5.010
WeoniI Jeong. Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks. 2020; 15(5), 671-681. Available from: doi:10.34163/jkits.2020.15.5.010
WeoniI Jeong. "Resource Sharing Scheme for Continuous Aggregation Query Processing in Spark Frameworks" Journal of Knowledge Information Technology and Systems 15, no.5 (2020) : 671-681.doi: 10.34163/jkits.2020.15.5.010
