PENERAPAN ALGORITMA MOBILENETV2 UNTUK KLASIFIKASI HURUF HIJAIYAH BERBASIS GESTUR TANGAN
##plugins.themes.academic_pro.article.main##
Abstract
The digitalization of religious education offers significant opportunities to enhance Hijaiyah letter learning, particularly for the hearing-impaired community through visual gesture recognition. This study aims to develop and evaluate a real-time web-based classification system for 28 Hijaiyah hand gestures using the MobileNetV2 architecture. The research methodology involves a quantitative approach utilizing transfer learning with a balanced dataset of augmented images. The model was trained using fine-tuning techniques and deployed on a web platform using TensorFlow.js and MediaPipe for efficient on-device inference. Experimental results demonstrate that the model achieved an overall accuracy of 84% on the independent test set, with specific classes reaching near-perfect detection in real-time scenarios, although misclassification persisted among visually similar gestures. The system effectively balances computational efficiency with classification performance, minimizing latency during user interaction. In conclusion, the implementation of MobileNetV2 facilitates a responsive and accessible educational tool, proving the viability of computer vision in creating inclusive religious learning environments without requiring complex server-side infrastructure.
##plugins.themes.academic_pro.article.details##
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
Riswan, M., Wahyuni, T., Danuputri, C., Habi Talib, E. A., Faisal, M., Anas, L., & Agung, A. (2026). PENERAPAN ALGORITMA MOBILENETV2 UNTUK KLASIFIKASI HURUF HIJAIYAH BERBASIS GESTUR TANGAN. Jurnal Informatika Progres, 18(1), 102-110. https://doi.org/10.56708/progres.v18i1.535
References
[1] C. del R. Navas-Bonilla, J. A. Guerra-Arango, D. A. Oviedo-Guado, and D. E. Murillo-Noriega, “Inclusive education through technology: a systematic review of types, tools and characteristics,” 2025, Frontiers Media SA. doi: 10.3389/feduc.2025.1527851.
[2] D. Priyanto and Hairani, “A Hijaiyah Letters Sign Language Recognition Approach utilizing Deep Learning,” Engineering, Technology and Applied Science Research, vol. 15, no. 3, pp. 22452–22458, Jun. 2025, doi: 10.48084/etasr.10285.
[3] R. Saputra et al., “Sistem Klasifikasi Alfabet Bahasa Isyarat Indonesia … 237.” [Online]. Available: https://data.mendeley.com/datasets/ywnjpbcz8m/1.
[4] Y. Brianorman and R. Munir, “Perbandingan Pre-Trained CNN: Klasifikasi Pengenalan Bahasa Isyarat Huruf Hijaiyah,” J. Sistem Info. Bisnis, vol. 13, no. 1, pp. 52–59, Jul. 2023, doi: 10.21456/vol13iss1pp52-59.
[5] R. Haris Alfikri et al., “PEMBANGUNAN APLIKASI PENERJEMAH BAHASA ISYARAT DENGAN METODE CNN BERBASIS ANDROID,” 2022. [Online]. Available: https://ejurnal.teknokrat.ac.id/index.php/teknoinfo/index
[6] N. T. Adam, Z. A. Tyas, and T. Hardiani, “Deteksi Gestur Sistem Isyarat Bahasa Indonesia Menggunakan Metode Deep learning SSD MobileNet V2 FPNLite,” Sainteks, vol. 21, no. 2, p. 129, Oct. 2024, doi: 10.30595/sainteks.v21i2.24006.
[7] G. Latif, N. Mohammad, J. Alghazo, R. AlKhalaf, and R. AlKhalaf, “ArASL: Arabic Alphabets Sign Language Dataset,” Data Brief, vol. 23, Apr. 2019, doi: 10.1016/j.dib.2019.103777.
[8] S. Suki, C. Lubis, and J. Pragantha, “IMPLEMENTASI VIRTUAL MOUSE BERBASIS HAND GESTURE RECOGNITION DENGAN MEDIAPIPE DAN CONVOLUTIONAL NEURAL NETWORK,” Jurnal INSTEK (Informatika Sains dan Teknologi), vol. 10, no. 1, pp. 40–51, May 2025, doi: 10.24252/instek.v10i1.53138.
[9] M. Faisal, Rahman, F. Shabir, and Ida, “Design and Implementation of Plantation Commodities Price Information Broadcaster via Autoreply Short Message Service on Smartphone,” in Proceedings - 2nd East Indonesia Conference on Computer and Information Technology: Internet of Things for Industry, EIConCIT 2018, IEEE, Nov. 2018, pp. 212–217. doi: 10.1109/EIConCIT.2018.8878575.
[10] Ricky Putra Sardika and W. Widhiarso, “Klasifikasi Otomatis Tingkat Kerusakan Retak Bangunan pada Citra Digital Menggunakan MobileNetV2 dan Augmentasi Data,” Arcitech: Journal of Computer Science and Artificial Intelligence, vol. 5, no. 1, pp. 108–124, Jun. 2025, doi: 10.29240/arcitech.v5i1.13938.
[11] N. M. K. K. Handayani, E. Y. Hidayat, M. Naufal, and P. L. W. E. Putra, “Pengenalan Ekspresi Wajah Menggunakan Transfer Learning MobileNetV2 dan EfficientNet-B0 dalam Memprediksi Perkelahian,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 8, no. 1, p. 106, Jan. 2024, doi: 10.30865/mib.v8i1.7048.
[12] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” Mar. 2019, [Online]. Available: http://arxiv.org/abs/1801.04381
[13] L. N. Hayati, A. N. Handayani, W. S. G. Irianto, R. A. Asmara, D. Indra, and N. S. Damanhuri, “Improving Indonesian Sign Alphabet Recognition for Assistive Learning Robots Using Gamma-Corrected MobileNetV2,” Buletin Ilmiah Sarjana Teknik Elektro, vol. 7, no. 3, pp. 350–361, Sep. 2025, doi: 10.12928/biste.v7i3.13300.
[14] M. Faisal et al., “A Hybrid MOO, MCGDM, and Sentiment Analysis Methodologies for Enhancing Regional Expansion Planning: A Case Study Luwu - Indonesia,” Int. J. Math. Eng. Manag. Sci., vol. 10, no. 1, pp. 163–188, Feb. 2025, doi: 10.33889/IJMEMS.2025.10.1.010.
[15] I. U. W. Mulyono, E. H. Rachmawanto, C. A. Sari, and M. K. Sarker, “A high accuracy of deep learning based CNN architecture: classic, VGGNet, and RestNet50 for Covid-19 image classification,” Telkomnika (Telecommunication Computing Electronics and Control), vol. 22, no. 5, pp. 1187–1195, Oct. 2024, doi: 10.12928/TELKOMNIKA.v22i5.26017.
[16] A. Alnuaim, M. Zakariah, W. A. Hatamleh, H. Tarazi, V. Tripathi, and E. T. Amoatey, “Human-Computer Interaction with Hand Gesture Recognition Using ResNet and MobileNet,” Comput. Intell. Neurosci., vol. 2022, 2022, doi: 10.1155/2022/8777355.
[17] M. B. Subkhi, M. Yuda Trinurais, R. Kuncoro, A. Wibowo, and B. R. Prakosa, “Program Studi Teknik Informatika,” 2024.
[18] M. Faisal, T. K. A. Rahman, I. Mulyadi, K. Aryasa, Irmawati, and M. Thamrin, “A Novelty Decision-Making Based on Hybrid Indexing, Clustering, and Classification Methodologies: An Application to Map the Relevant Experts Against the Rural Problem,” Decis. Mak. Appl. Manag. Eng., vol. 7, no. 2, pp. 132–171, Feb. 2024, doi: 10.31181/dmame7220241023
[19] D. E. Putri, J. Jumadi, and E. Nurlatifah, “Klasifikasi Pengenalan Huruf Hijaiyah Pada Bahasa Isyarat Arab Menggunakan Transfer Learning EfficientNetB1,” JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika), vol. 10, no. 2, pp. 1371–1384, Mar. 2025, doi: 10.29100/jipi.v10i2.6174.
[20] D. Mulya, “Classification of Hijaiyah Letters Using Hybrid CNN-CatBoost,” Intelmatics, vol. 3, no. 2, pp. 39–44, Aug. 2023, doi: 10.25105/itm.v3i2.17521.
[21] N. S. W. Nugroho and M. P. K. Putra, “LEVERAGING DEEP LEARNING APPROACH FOR ACCURATE ALPHABET RECOGNITION THROUGH HAND GESTURES IN SIGN LANGUAGE,” Jurnal Teknik Informatika (Jutif), vol. 6, no. 1, pp. 259–268, Feb. 2025, doi: 10.52436/1.jutif.2025.6.1.3912.
[22] B. Kurniawan Pratama, S. Lestanti, and Y. Primasari, “Implementasi Algoritma You Only Look Once (YOLO) untuk Mendeteksi Bahasa Isyarat SIBI,” Jurnal ProTekInfo |, vol. 11, no. 2, 2024.
[23] R. Kuncoro, A. Wibowo, A. Sanjaya, U. Mahdiyah, U. Nusantara, and P. Kediri, “Prosiding SEMNAS INOTEK (Seminar Nasional Inovasi Teknologi) 139 Implementasi YOLOv8 Pada Pengenalan Sistem Isyarat Bahasa Indonesia,” Online, 2024.
[2] D. Priyanto and Hairani, “A Hijaiyah Letters Sign Language Recognition Approach utilizing Deep Learning,” Engineering, Technology and Applied Science Research, vol. 15, no. 3, pp. 22452–22458, Jun. 2025, doi: 10.48084/etasr.10285.
[3] R. Saputra et al., “Sistem Klasifikasi Alfabet Bahasa Isyarat Indonesia … 237.” [Online]. Available: https://data.mendeley.com/datasets/ywnjpbcz8m/1.
[4] Y. Brianorman and R. Munir, “Perbandingan Pre-Trained CNN: Klasifikasi Pengenalan Bahasa Isyarat Huruf Hijaiyah,” J. Sistem Info. Bisnis, vol. 13, no. 1, pp. 52–59, Jul. 2023, doi: 10.21456/vol13iss1pp52-59.
[5] R. Haris Alfikri et al., “PEMBANGUNAN APLIKASI PENERJEMAH BAHASA ISYARAT DENGAN METODE CNN BERBASIS ANDROID,” 2022. [Online]. Available: https://ejurnal.teknokrat.ac.id/index.php/teknoinfo/index
[6] N. T. Adam, Z. A. Tyas, and T. Hardiani, “Deteksi Gestur Sistem Isyarat Bahasa Indonesia Menggunakan Metode Deep learning SSD MobileNet V2 FPNLite,” Sainteks, vol. 21, no. 2, p. 129, Oct. 2024, doi: 10.30595/sainteks.v21i2.24006.
[7] G. Latif, N. Mohammad, J. Alghazo, R. AlKhalaf, and R. AlKhalaf, “ArASL: Arabic Alphabets Sign Language Dataset,” Data Brief, vol. 23, Apr. 2019, doi: 10.1016/j.dib.2019.103777.
[8] S. Suki, C. Lubis, and J. Pragantha, “IMPLEMENTASI VIRTUAL MOUSE BERBASIS HAND GESTURE RECOGNITION DENGAN MEDIAPIPE DAN CONVOLUTIONAL NEURAL NETWORK,” Jurnal INSTEK (Informatika Sains dan Teknologi), vol. 10, no. 1, pp. 40–51, May 2025, doi: 10.24252/instek.v10i1.53138.
[9] M. Faisal, Rahman, F. Shabir, and Ida, “Design and Implementation of Plantation Commodities Price Information Broadcaster via Autoreply Short Message Service on Smartphone,” in Proceedings - 2nd East Indonesia Conference on Computer and Information Technology: Internet of Things for Industry, EIConCIT 2018, IEEE, Nov. 2018, pp. 212–217. doi: 10.1109/EIConCIT.2018.8878575.
[10] Ricky Putra Sardika and W. Widhiarso, “Klasifikasi Otomatis Tingkat Kerusakan Retak Bangunan pada Citra Digital Menggunakan MobileNetV2 dan Augmentasi Data,” Arcitech: Journal of Computer Science and Artificial Intelligence, vol. 5, no. 1, pp. 108–124, Jun. 2025, doi: 10.29240/arcitech.v5i1.13938.
[11] N. M. K. K. Handayani, E. Y. Hidayat, M. Naufal, and P. L. W. E. Putra, “Pengenalan Ekspresi Wajah Menggunakan Transfer Learning MobileNetV2 dan EfficientNet-B0 dalam Memprediksi Perkelahian,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 8, no. 1, p. 106, Jan. 2024, doi: 10.30865/mib.v8i1.7048.
[12] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” Mar. 2019, [Online]. Available: http://arxiv.org/abs/1801.04381
[13] L. N. Hayati, A. N. Handayani, W. S. G. Irianto, R. A. Asmara, D. Indra, and N. S. Damanhuri, “Improving Indonesian Sign Alphabet Recognition for Assistive Learning Robots Using Gamma-Corrected MobileNetV2,” Buletin Ilmiah Sarjana Teknik Elektro, vol. 7, no. 3, pp. 350–361, Sep. 2025, doi: 10.12928/biste.v7i3.13300.
[14] M. Faisal et al., “A Hybrid MOO, MCGDM, and Sentiment Analysis Methodologies for Enhancing Regional Expansion Planning: A Case Study Luwu - Indonesia,” Int. J. Math. Eng. Manag. Sci., vol. 10, no. 1, pp. 163–188, Feb. 2025, doi: 10.33889/IJMEMS.2025.10.1.010.
[15] I. U. W. Mulyono, E. H. Rachmawanto, C. A. Sari, and M. K. Sarker, “A high accuracy of deep learning based CNN architecture: classic, VGGNet, and RestNet50 for Covid-19 image classification,” Telkomnika (Telecommunication Computing Electronics and Control), vol. 22, no. 5, pp. 1187–1195, Oct. 2024, doi: 10.12928/TELKOMNIKA.v22i5.26017.
[16] A. Alnuaim, M. Zakariah, W. A. Hatamleh, H. Tarazi, V. Tripathi, and E. T. Amoatey, “Human-Computer Interaction with Hand Gesture Recognition Using ResNet and MobileNet,” Comput. Intell. Neurosci., vol. 2022, 2022, doi: 10.1155/2022/8777355.
[17] M. B. Subkhi, M. Yuda Trinurais, R. Kuncoro, A. Wibowo, and B. R. Prakosa, “Program Studi Teknik Informatika,” 2024.
[18] M. Faisal, T. K. A. Rahman, I. Mulyadi, K. Aryasa, Irmawati, and M. Thamrin, “A Novelty Decision-Making Based on Hybrid Indexing, Clustering, and Classification Methodologies: An Application to Map the Relevant Experts Against the Rural Problem,” Decis. Mak. Appl. Manag. Eng., vol. 7, no. 2, pp. 132–171, Feb. 2024, doi: 10.31181/dmame7220241023
[19] D. E. Putri, J. Jumadi, and E. Nurlatifah, “Klasifikasi Pengenalan Huruf Hijaiyah Pada Bahasa Isyarat Arab Menggunakan Transfer Learning EfficientNetB1,” JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika), vol. 10, no. 2, pp. 1371–1384, Mar. 2025, doi: 10.29100/jipi.v10i2.6174.
[20] D. Mulya, “Classification of Hijaiyah Letters Using Hybrid CNN-CatBoost,” Intelmatics, vol. 3, no. 2, pp. 39–44, Aug. 2023, doi: 10.25105/itm.v3i2.17521.
[21] N. S. W. Nugroho and M. P. K. Putra, “LEVERAGING DEEP LEARNING APPROACH FOR ACCURATE ALPHABET RECOGNITION THROUGH HAND GESTURES IN SIGN LANGUAGE,” Jurnal Teknik Informatika (Jutif), vol. 6, no. 1, pp. 259–268, Feb. 2025, doi: 10.52436/1.jutif.2025.6.1.3912.
[22] B. Kurniawan Pratama, S. Lestanti, and Y. Primasari, “Implementasi Algoritma You Only Look Once (YOLO) untuk Mendeteksi Bahasa Isyarat SIBI,” Jurnal ProTekInfo |, vol. 11, no. 2, 2024.
[23] R. Kuncoro, A. Wibowo, A. Sanjaya, U. Mahdiyah, U. Nusantara, and P. Kediri, “Prosiding SEMNAS INOTEK (Seminar Nasional Inovasi Teknologi) 139 Implementasi YOLOv8 Pada Pengenalan Sistem Isyarat Bahasa Indonesia,” Online, 2024.