@article{ART003349291},
author={Taemin Kim and Junhyeok Lee and Haeun Cho and Hyunju Kim and Sanghyun Kim and Jimin Lee and Won Joo Lee},
title={Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts},
journal={Journal of The Korea Society of Computer and Information},
issn={1598-849X},
year={2026},
volume={31},
number={6},
pages={91-98}
TY - JOUR
AU - Taemin Kim
AU - Junhyeok Lee
AU - Haeun Cho
AU - Hyunju Kim
AU - Sanghyun Kim
AU - Jimin Lee
AU - Won Joo Lee
TI - Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts
JO - Journal of The Korea Society of Computer and Information
PY - 2026
VL - 31
IS - 6
PB - The Korean Society Of Computer And Information
SP - 91
EP - 98
SN - 1598-849X
AB - In this paper, we proposes a Semojum, a multimodal AI-based braille translation support service designed to provide real-time braille books for the visually impaired. Semo-Jeom consists of a DeepSeek-OCR-2 based layout analysis module, a Semantic NMS algorithm, a table layout optimization engine, and a GPT-4o based image captioning module. The layout analysis module scales page images up to 1,536 pixels and outputs element types, OCR text, bounding boxes, and reading orders in a single pass using specialized markup tokens. To refine the output, the Semantic NMS algorithm determines redundancy based on the semantic inclusion relationships between extracted text contents.
The table layout optimization engine extracts structured data using the JSON Schema strict output of GPT-4o-mini Vision. For visual accessibility, the image captioning module utilizes GPT-4o Vision, injecting adjacent text from the preceding and following pages as context; it then classifies images into seven distinct patterns and applies specific descriptive strategies for each. Furthermore, Semo-Jeom implements a hybrid braille translation engine that combines the Braillify library with a custom LaTeX tokenizer to support the 2024 Revised Korean Mathematics Braille Regulations. By integrating a Human-in-the-loop structure that presents AI outputs as verifiable drafts and a tab-based three-mode interface, the system significantly enhances the operational efficiency of professional braille translators.
KW - Braille Translation Automation;Multimodal AI;Layout Analysis;Korean Mathematical Braille;Human-in-the-loop;BRFI
DO -
UR -
ER -
Taemin Kim, Junhyeok Lee, Haeun Cho, Hyunju Kim, Sanghyun Kim, Jimin Lee and Won Joo Lee. (2026). Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts. Journal of The Korea Society of Computer and Information, 31(6), 91-98.
Taemin Kim, Junhyeok Lee, Haeun Cho, Hyunju Kim, Sanghyun Kim, Jimin Lee and Won Joo Lee. 2026, "Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts", Journal of The Korea Society of Computer and Information, vol.31, no.6 pp.91-98.
Taemin Kim, Junhyeok Lee, Haeun Cho, Hyunju Kim, Sanghyun Kim, Jimin Lee, Won Joo Lee "Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts" Journal of The Korea Society of Computer and Information 31.6 pp.91-98 (2026) : 91.
Taemin Kim, Junhyeok Lee, Haeun Cho, Hyunju Kim, Sanghyun Kim, Jimin Lee, Won Joo Lee. Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts. 2026; 31(6), 91-98.
Taemin Kim, Junhyeok Lee, Haeun Cho, Hyunju Kim, Sanghyun Kim, Jimin Lee and Won Joo Lee. "Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts" Journal of The Korea Society of Computer and Information 31, no.6 (2026) : 91-98.
Taemin Kim; Junhyeok Lee; Haeun Cho; Hyunju Kim; Sanghyun Kim; Jimin Lee; Won Joo Lee. Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts. Journal of The Korea Society of Computer and Information, 31(6), 91-98.
Taemin Kim; Junhyeok Lee; Haeun Cho; Hyunju Kim; Sanghyun Kim; Jimin Lee; Won Joo Lee. Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts. Journal of The Korea Society of Computer and Information. 2026; 31(6) 91-98.
Taemin Kim, Junhyeok Lee, Haeun Cho, Hyunju Kim, Sanghyun Kim, Jimin Lee, Won Joo Lee. Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts. 2026; 31(6), 91-98.
Taemin Kim, Junhyeok Lee, Haeun Cho, Hyunju Kim, Sanghyun Kim, Jimin Lee and Won Joo Lee. "Semojum: A Multimodal AI-Based Braille Transcription Support Service for Visual Materials and Layouts" Journal of The Korea Society of Computer and Information 31, no.6 (2026) : 91-98.
