Adaptive hybrid optimization for backpropagation neural networks in image classification

Authors

  • Samuel O. Essang Department of Mathematics and Computer Science, Arthur Jarvis University, Akpabuyo, Nigeria
  • Stephen I. Okeke Department of Industrial Mathematics and Health Statistics, David Umahi Federal University of Health Sciences Uburu, Ebonyi State, Nigeria
  • Jackson E. Ante Department of Mathematics, TopFaith University, Mkpatak, Nigeria
  • Runyi E. Francis Federal Polytechnic Ugep, Nigeria
  • Sunday E. Fadugba Department of Mathematics, Ekiti State University, Ado Ekiti, Nigeria
  • Augustine O. Ogbaji Department of Computer Science, University of Calabar
  • Jonathan T. Auta Department of Mathematics, African University of Science and Technology, Abuja
  • Chikwe F. Chukwuka Department of Mathematics, University of Calabar
  • Michael O. Ogar-Abang Department of Physics, Arthur Jarvis University, Akpabuyo
  • Ede M. Aigberemhon Department of Electrical and Electronic, Cross River State University, Calabar, Nigeria

Keywords:

Hybrid optimization, Backpropagation neural networks, Particle swarm optimization, AdaGrad optimization

Abstract

Image classification is essential in artificial intelligence, with applications in medical diagnostics, autonomous navigation, and industrial automation. Traditional training methods like stochastic gradient descent (SGD) often suffer from slow convergence and local minima. This research presents a hybrid Particle Swarm Optimization (PSO)-Genetic Algorithm (GA)-Backpropagation framework to enhance neural network training. By integrating AdaGrad and PSO for weight optimization, GA for refinement, and backpropagation for fine-tuning, the model improves performance. Results show a 97.5% accuracy on MNIST, a 5% improvement over Adam, and 40% faster convergence than SGD. This approach enhances efficiency, accuracy, and generalization, making it valuable for high-dimensional AI tasks.

Dimensions

[1] S. Paheding & A. A. Reyes-Angulo, “Forward-forward algorithm for hyperspectral image classification: A preliminary study”, arXiv preprint arXiv:2307.00231, (2023). [Online]. https://doi.org/10.48550/arXiv.2307.00231.

[2] C. Alessandro & T. G. de Jong, “How neural networks learn to classify chaotic time series”, Chaos: An Interdisciplinary Journal of Nonlinear Science 33 (2023) 123101. https://doi.org/10.1063/5.0160813.

[3] S. O. Essang, J. E. Ante, J. Oboyi, J. N. Ezeorah, F. E. Runyi & C. F. Chikwe, “Mathematical modeling of marital success: A quantitative analysis of communication, conflict resolution, and financial synergy”, Scholars Journal of Physics, Mathematics and Statistics 11 (2024) 192. https://doi.org/10.36347/sjpms.2024.v11i12.002.

[4] S. O. Essang, K. O. Michael, R. E. Francis, J. E. Ante, O. A. M. Obi, A. J. Timothy, O. P. Edet, E. R. Dominic & U. A. Jimmy, “Application of AI algorithms for the prediction of the likelihood of sickle cell crises”, Scholars Journal of Engineering and Technology 12 (2024) 394. https://doi.org/10.36347/sjet.2024.v12i12.005.

[5] S. Ding, B. Qu & Y. Xiang, “Training artificial neural networks using self-organizing migrating algorithm for skin segmentation tasks”, Scientific Reports 14 (2024) 72884. https://doi.org/10.1038/s41598-024-72884-0.

[6] R. Kumar & V. Singh, “Optimization of convolutional neural network architecture by PSO algorithm for MRI brain tumor image classification”, International Journal of Advanced Computer Science and Applications 15 (2024) 1. https://doi.org/10.14569/IJACSA.2024.0150101.

[7] J. Li & H. Zhang, “Enhanced long short-term memory architectures for chaotic systems modeling”, Chaos: An Interdisciplinary Journal of Nonlinear Science 34 (2024) 123152. https://doi.org/10.1063/5.0134567.

[8] M. T. El-Saadony, A. M. Saad, D. M. Mohammed, M. A. Fahmy, I. E. Elesawi, A. E. Ahmed & K. A. El-Tarabily, “Drought-tolerant plant growth-promoting rhizobacteria alleviate drought stress and enhance soil health for sustainable agriculture: A comprehensive review”, Plant Stress 14 (2024) 100632. https://doi.org/10.1016/j.stress.2024.100632.

[9] D. Singh & G. J. Sreejith, “Initializing ReLU networks in an expressive subspace of weights”, arXiv preprint arXiv:2103.12499, (2021). [Online]. https://doi.org/10.48550/arXiv.2103.12499.

[10] Y. Li & Y. Yuan, “Convergence analysis of two-layer neural networks with ReLU activation”, IEEE Transactions on Neural Networks and Learning Systems 32 (2021) 3064. https://doi.org/10.1109/TNNLS.2020.2993587.

[11] X. Liu, L. Song, S. Liu & G. Wang, “A review of deep-learning-based medical image segmentation methods”, Sustainability 11 (2019) 1897. https://doi.org/10.3390/su11071897.

[12] C. Cardona, “Scaling and resizing symmetry in feedforward networks”, (2023). arXiv preprint arXiv:2306.15015. [Online]. https://doi.org/10.48550/arXiv.2306.15015.

[13] T. Lamjiak, B. Sirinaovakul, S. Kornthongnimit, J. Polvichai, A. Sohail, “Optimizing artificial neural network learning using improved reinforcement learning in artificial bee colony algorithm”, Applied Computational Intelligence and Soft Computing 2024 (2024) 6357270. https://doi.org/10.1155/2024/6357270.

[14] S. O. Essang, J. E. Ante, A. O. Otobi, S. I. Okeke, U. D. Akpan, R. E. Francis, J. T. Auta, D. E. Essien, S. E. Fadugba, O. M. Kolawole, E. E. Asanga & B. I. Ita, “Optimizing neural networks with convex hybrid activations for improved gradient flow”, Advanced Journal of Science, Technology and Engineering 5 (2025) 10. https://doi.org/10.52589/AJSTE-UOBYFV1B.

[15] D. P. Kingma & J. Ba, “Adam: a method for stochastic optimization”, (2015), arXiv preprint arXiv:1412.6980. [Online]. https://doi.org/10.48550/arXiv.1412.6980.

[16] T. Akiba, S. Katayama & Y. Iwamura, “Optuna: a next-generation hyperparameter optimization framework”, (2019). [Online]. https://doi.org/10.48550/arXiv.1907.10902.

[17] S. Gupta & A. Sharma, “Comparison of optimization algorithms based on swarm intelligence for designing convolutional neural networks”, Human-centric Computing and Information Sciences 12 (2024) 10. https://doi.org/10.3233/HIS-220010.

[18] M. Munsarif & M. Sam?an, “Convolution neural network hyperparameter optimization using modified particle swarm optimization”, Bulletin of Electrical Engineering and Informatics 13 (2024) 1268. https://doi.org/10.11591/eei.v13i2.6112.

[19] Q. Q. He, C. Wu & Y. W. Si, “LSTM with particle Swam optimization for sales forecasting”, Electronic Commerce Research and Applications 51 (2022) 101118. https://doi.org/10.1016/j.elerap.2022.101118.

[20] Y. Qin & H. Wang, “A hybrid LSTM-PSO model for improved forecasting accuracy”, PLoS ONE 19 (2024) e0273171. https://doi.org/10.1371/journal.pone.0273171.

[21] S. Huang & Y. Wang, “Defense against adversarial attacks: Robust and efficient optimization strategies for neural networks”, Journal of Neural Engineering 21 (2024) 036017. https://doi.org/10.1088/1741-2552/abf7c9.

[22] A. O. Otobi, J. O. Esin, I. E. Eteng, B. I. Ele, S. I. Ele, D. U. Ashishie & C. A. Okpan, “The computational effect and hyperparameters tuning of deep convolutional layer depth of high-ranking tuberculosis detection models”, Global Journal of Pure and Applied Sciences 30 (2024) 577. https://www.ajol.info/index.php/gjpas/article/view/282583.

[23] S. I. Okeke, “Modelling transportation problem using harmonic mean”, International Journal of Transformation in Applied Mathematics & Statistics 3 (2020) 7. https://www.researchgate.net/publication/341941026_Modelling_Transportation_Problem_Using_Harmonic_Mean.

[24] T.T.K. Lau, H. Liu & M. Kolar, “AdAdaGrad: adaptive batch size schemes for adaptive gradient methods”, (2024). arXiv preprint arXiv:2402.11215. [Online]. https://arxiv.org/pdf/2402.11215.

[25] Z. Zhang, H. Zhang & D. Wang, “Multi-modal learning for automated sickle cell disease detection”, Journal of Medical Systems 44 (2020) 7. https://doi.org/10.1007/s10916-019-1487-5.

[26] B. Zhang, D. Zou & D. Tao, “Improved initialization for training deep neural networks”, (2022). arXiv preprint arXiv:2202.04055. https://doi.org/10.48550/arXiv.2202.04055.

[27] A. Suggala, V. Kazemi & D. Ramanan, “On the interplay between adaptive optimization and regularization in deep learning”, (2023) arXiv preprint arXiv:2305.11691. https://doi.org/10.48550/arXiv.2305.11691.

[28] D. Wang, D. Tan & L. Liu, “Particle swarm optimization algorithm: an overview”, Soft computing 22 (2018) 387. https://doi.org/10.1007/s00500-016-2474-6.

[29] A. S. Berahas, M. Jahani, P. Richtárik & M. Taká?, “Quasi-Newton methods for machine learning: forget the past, just sample”, Optimization Methods and Software 37 (2022) 1668. https://doi.org/10.1080/10556788.2021.1977806.

Published

2025-03-11

How to Cite

Adaptive hybrid optimization for backpropagation neural networks in image classification. (2025). Proceedings of the Nigerian Society of Physical Sciences, 2(1), 150. https://doi.org/10.61298/pnspsc.2025.2.150

Issue

Volume 2, NSPS-FUOYE Conference, Feb. 3 - 6, 2025

Section

Articles
Copyright & Licensing

Copyright (c) 2025 Samuel O. Essang, Stephen I. Okeke, Jackson E. Ante, Runyi E. Francis, Sunday E. Fadugba, Augustine O. Ogbaji, Jonathan T. Auta, Chikwe F. Chukwuka, Michael O. Ogar-Abang, Ede M. Aigberemhon (Author)

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Adaptive hybrid optimization for backpropagation neural networks in image classification. (2025). Proceedings of the Nigerian Society of Physical Sciences, 2(1), 150. https://doi.org/10.61298/pnspsc.2025.2.150