A Performance Analysis Based on Hadoop Application's Characteristics in Cloud Computing

금태훈; Lee, Won Joo; Chang Ho Jeon

Journal of The Korea Society of Computer and Information

Abbr : JKSCI
2010, 15(5), pp.49-56
Publisher : The Korean Society Of Computer And Information
Research Area : Computer Science

금태훈 ¹, Lee, Won Joo ², Chang Ho Jeon ³

¹한양대 컴퓨터공학과
²인하공업전문대학
³한양대학교

ABSTRACT

In this paper, we implement a Hadoop based cluster for cloud computing and evaluate the performance of this cluster based on application characteristics by executing RandomTextWriter, WordCount, and PI applications. A RandomTextWriter creates given amount of random words and stores them in the HDFS(Hadoop Distributed File System). A WordCount reads an input file and determines the frequency of a given word per block unit. PI application induces PI value using the Monte Carlo law. During simulation, we investigate the effect of data block size and the number of replications on the execution time of applications. Through simulation, we have confirmed that the execution time of RandomTextWriter was proportional to the number of replications. However, the execution time of WordCount and PI were not affected by the number of replications. Moreover, the execution time of WordCount was optimum when the block size was 64~256MB. Therefore, these results show that the performance of cloud computing system can be enhanced by using a scheduling scheme that considers application's characteristics.
KEYWORDS

Cloud Computing, Hadoop Distributed File System. Cluster, Hadoop
REFERENCES (11)

[web] / Wikipedia / http://en.wikipedia.org/wiki/Christophe_Bisciglia

[web] / Amazon Elastic Compute Cloud / http://aws.amazon.com/ec2

[web] / IBM Blue Cloud project / ttp://www04.ibm.com/jct03001c/press/us/en/pressrelease/22613.wss,2009.

[web] / Google App Engine / http://code.google.com/appengine

[confproc] L.Wang / 2008 / Cloud Computing:A Perspective Study / Proceedings of the Grid Computing Environments(GCE)workshop,

[journal] S.Ghemawat / 2003 / The Google file system / ACM SIGOPS Operating Systems Review 37(5) : 29~43

[journal] J.Dean / 2008 / Mapreduce:Simplified Data Processing on Large Clusters / Communications of the ACM 51(1) : 107~113

[web] / Hadoop / http://hadoop.apache.org

[confproc] J.Boulon / 2008 / Chukwa:A large-scale monitoring system / Proceeding of international conference on Cloud Computing and Its Applications, : 1~5

[confproc] J. Tan / 2009 / Mochi: Visualizing Log-Anlaysis Based Tools for Debugging Hadoop / USENIX Workshop on Hot Topics in Cloud Computing(HotCloud),

[journal] 금태훈 / 2009 / 클라우드 컴퓨팅을 위한 Hadoop 애플리케이션 특성 분석(An Analysis of Hadoop Application's Characteristics for Cloud Computing)," / 한국컴퓨터정보학회 2009 하계학술발표논문집 17(1) : 11~12
KCI Citation (7)

KJC Korea
Journal Central

Statistics

Tools