Big Data Processing and Performance Improvement for Ship Trajectory using MapReduce Technique

Kim Kwang-il; Joo-sung， Kim

doi:https://doi.org/10.9708/jksci.2019.24.10.065

Journal of The Korea Society of Computer and Information

Abbr : JKSCI
2019, 24(10), pp.65-70
DOI :https://doi.org/10.9708/jksci.2019.24.10.065
Publisher : The Korean Society Of Computer And Information
Research Area : Computer Science

Kim Kwang-il ¹ , Joo-sung， Kim ²

¹ 제주대학교
² 목포해양대학교

ABSTRACT

In recently, ship trajectory data consisting of ship position, speed, course, and so on can be obtained from the Automatic Identification System device with which all ships should be equipped. These data are gathered more than 2GB every day at a crowed sea port and used for analysis of ship traffic statistic and patterns. In this study, we propose a method to process ship trajectory data efficiently with distributed computing resources using MapReduce algorithm. In data preprocessing phase, ship dynamic and static data are integrated into target dataset and filtered out ship trajectory that is not of interest. In mapping phase, we convert ship's position to Geohash code, and assign Geohash and ship MMSI to key and value. In reducing phase, key-value pairs are sorted according to the same key value and counted the ship traffic number in a grid cell. To evaluate the proposed method, we implemented it and compared it with IALA waterway risk assessment program(IWRAP) in their performance. The data processing performance improve 1 to 4 times that of the existing ship trajectory analysis program.
KEYWORDS

Ship Trajectory , Automatic Identification System , Geohash , MapReduce , IWRAP
REFERENCES (14)

[journal] K. I. Kim / 2017 / Ship Encounter Risk Evaluation for Coastal Areas with Holistic Maritime Traffic Data Analysis / Advanced Science Letters 23 (10) : 9565~9569

[web] International Maritime Organization (IMO) / IALA Waterway Risk Assessment Tool / http://www.iala-aism.org/wiki/iwrap

[confproc] K. Shvachko / 2010 / The Hadoop Distributed File System. In Mass Storage Systems and Technologies(MSST) / 2010 IEEE 26th symposium on IEEE 1 (1) : 1~10

[web] MarineTraffic website / Live Ship Map - AIS - Vessel Traffic and Positions / https://www.marinetraffic.com

[web] / MyShipTracking website / http://www.myshiptracking.com

[web] / VesselFinder website / http://www.vesselfinder.com

[confproc] H. Yawen / 2010 / Web-based visualization of marine environment data / Geoinformatics, 2010 18th International Conference on / IEEE 1 (1) : 1~6

[confproc] P. R. Lei / 2014 / Discovering maritime traffic route from AIS network / 2016 18th Asia-Pacific Network Operations and Management Symposium (APNOMS) / IEEE 1 (1) : 25~30

[journal] G. Pallotta / 2013 / Vessel pattern knowledge discovery from AIS data : A framework for anomaly detection and route prediction / Entropy 15 (6) : 2218~2245

[confproc] M. Zhou / 2013 / Ais data based identification of systematic collision risk for maritime intelligent transport system / Communications (ICC), 2013IEEE International Conference on 1 (1) : 6158~6162

[confproc] X. Wang / 2014 / Vessel route anomaly detection with Hadoop MapReduce / 2014 IEEE International Conference on Big Data (Big Data) / IEEE 1 (1) : 25~30

[other] International Maritime Organization(IMO) / 2002 / Guidelines for the onboard operational use of shipborne automatic identification systems (AIS), IMO Res A.917

[confproc] A. Fox / 2013 / Spatio-temporal indexing in non-relational distributed databases / 2013 IEEE International Conference on Big Data 1 (1) : 291~299

[journal] J. Dean / 2008 / MapReduce : simplified data processing on large clusters / Communications of the ACM 51 (1) : 107~113
KCI Citation (2)

KJC Korea
Journal Central

Statistics

Tools