Recently, many researches using unmanned aerial vehicles(UAV) has been proposed for applications in coastal areas. However, there is still a need for consecutive researches on various types of UAVs, sensors and regions. Especially, it is necessary to study the practical use of the UAV photogrammetry in marine surveying. In Korea, aquaculture production accounts for 61.8% of total aquatic-product production in 2017 and is increasing year by year. Therefore, there is a growing need to systematically manage, support and monitor aquatic products. In particular, unlicensed and illegal fisheries are increasing in Jeollanam-do. In 2016, the number of unlicensed and illegal fisheries has increased to 180% comparing in 2012. The Jeollanam-do fisheries resources division is implementing special crackdown on illegal aquaculture farmers every year. However, due to the nature of the marine environment, surveillance and enforcement are limited by field surveys alone. In this study, we propose a methodology by using UAV photogrammetry and automatic image recognition technology to increase efficiency of monitoring aquaculture farms. For this purpose, UAV photogrammetry was performed on abalone and seaweed ocean farms in Wando, Jeollanam-do. Then, we developed a methodology for automatically detecting the farm facilities in the marine environment by applying the Faster-RCNN (Regional Convolution Neural Network) to the generated orthophotos. Through the study, it is identified that small UAVs can be effectively used for the surveillance and management of the ocean farms in coastal areas. Also, the automatic method for recognizing aquaculture object using Faster R-CNN technique can be developed.