@article{ART003397400},
author={DongHwi Kim and SunYeong Jeong and JongWook Jang and WooYoung Kim},
title={Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM},
journal={Journal of The Korea Society of Computer and Information},
issn={1598-849X},
year={2026},
volume={31},
number={10},
pages={115-127}
TY - JOUR
AU - DongHwi Kim
AU - SunYeong Jeong
AU - JongWook Jang
AU - WooYoung Kim
TI - Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM
JO - Journal of The Korea Society of Computer and Information
PY - 2026
VL - 31
IS - 10
PB - The Korean Society Of Computer And Information
SP - 115
EP - 127
SN - 1598-849X
AB - Modern naval battlefields are complex, causing heavy workloads for commanders and reducing crew sizes. Therefore, commander decision support systems are essential for ship survival. Thus, this study assumed an environment where limited manuals and finite datasets are constructed due to characteristics of the defense domain. Under these constraints, this study constructed four architectures-Base, Fine-Tuning, Base+RAG, and Fine-Tuning+RAG-using open-source LLMs deployable within a ship’s closed-network environment. And their performance was evaluated using 50 virtual operational combat scenarios. Experimental results showed that the Base+RAG architecture was relatively effective in tactical validity and reliability compared to the QLoRA Fine-Tuning+RAG model, which increased the hallucination rate by approximately 8 percentage points. Although this phenomenon involves factors that require further verification, such as internal knowledge interference or overfitting, it demonstrates that the Base+RAG architecture can serve as a more stable alternative than a Fine-tuning-based architecture under constrained experimental conditions like those in this study. However, the architecture with RAG showed the first response time jumped from an average of 3 seconds to 24 seconds. This delay mainly caused by the load from real-time retrieval calculations and context processing. It indicates that while knowledge retrieval improves accuracy, it simultaneously introduces response latency as an operational trade-off.
KW - Command Decision Support System;Closed Naval Network Environment;Large Language Model (LLM);Retrieval-Augmented Generation (RAG);Fine-Tuning(FT)
DO -
UR -
ER -
DongHwi Kim, SunYeong Jeong, JongWook Jang and WooYoung Kim. (2026). Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM. Journal of The Korea Society of Computer and Information, 31(10), 115-127.
DongHwi Kim, SunYeong Jeong, JongWook Jang and WooYoung Kim. 2026, "Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM", Journal of The Korea Society of Computer and Information, vol.31, no.10 pp.115-127.
DongHwi Kim, SunYeong Jeong, JongWook Jang, WooYoung Kim "Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM" Journal of The Korea Society of Computer and Information 31.10 pp.115-127 (2026) : 115.
DongHwi Kim, SunYeong Jeong, JongWook Jang, WooYoung Kim. Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM. 2026; 31(10), 115-127.
DongHwi Kim, SunYeong Jeong, JongWook Jang and WooYoung Kim. "Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM" Journal of The Korea Society of Computer and Information 31, no.10 (2026) : 115-127.
DongHwi Kim; SunYeong Jeong; JongWook Jang; WooYoung Kim. Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM. Journal of The Korea Society of Computer and Information, 31(10), 115-127.
DongHwi Kim; SunYeong Jeong; JongWook Jang; WooYoung Kim. Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM. Journal of The Korea Society of Computer and Information. 2026; 31(10) 115-127.
DongHwi Kim, SunYeong Jeong, JongWook Jang, WooYoung Kim. Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM. 2026; 31(10), 115-127.
DongHwi Kim, SunYeong Jeong, JongWook Jang and WooYoung Kim. "Comparative and Combined Performance Evaluation of QLoRA Fine-Tuning and RAG for Developing Naval Integrated Weapons System Domain-Specific LLM" Journal of The Korea Society of Computer and Information 31, no.10 (2026) : 115-127.
