Development of Artificial Intelligence (AI)-Driven Aid to Enhance Visual and Hearing- Impaired Students

  • Joseph Sospeter Salawa Department of Electronics and Telecommunication Engineering, School of Engineering, Mbeya University of Science and Technology(MUST), P.O. BOX 131 Mbeya
  • Phocas Sebastian Department of Electronics and Telecommunication Engineering, School of Engineering, Mbeya University of Science and Technology(MUST), P.O. BOX 131 Mbeya
  • Charles Okanda Nyatega Department of Electronics and Telecommunication Engineering, School of Engineering, Mbeya University of Science and Technology(MUST), P.O. BOX 131 Mbeya
  • Juma Said Ally Department of Electronics and Telecommunication Engineering, School of Engineering, Mbeya University of Science and Technology(MUST), P.O. BOX 131 Mbeya
  • Cuthbert John Karawa Department of Electronics and Telecommunication Engineering, School of Engineering, Mbeya University of Science and Technology(MUST), P.O. BOX 131 Mbeya
  • Elizabeth Odrick Koola Department of Electronics and Telecommunication Engineering, School of Engineering, Mbeya University of Science and Technology(MUST), P.O. BOX 131 Mbeya
  • Richard Mwanjalila Department of Electronics and Telecommunication Engineering, School of Engineering, Mbeya University of Science and Technology(MUST), P.O. BOX 131 Mbeya
##article.subject##: Artificial Intelligence, Optical Character Recognition, Speech-To-Text, Text-To- Speech

##article.abstract##

This paper introduces a series of works on Artificial Intelligence (AI)-based assistive devices that improve students’ learning with visual and hearing impairments.  Artificial intelligence technology provides personal help and support for various learning tasks and activities. The system uses computer vision techniques to read visual information such as text and images, which can be made available in usable formats such as audio descriptions. In addition, the system can recognize and respond to the user’s voice. commands and requests using speech recognition technologies. Stu- dents can view learning materials, get instant help with classroom activities, and participate in engaging learning exercises designed for their specific needs through an intuitive user interface. Ensuring equal access to educational resources. The project focuses on the effective and efficient teaching and mentoring of students with visual and hearing impairments. To understand the impact and applicability of the proposed AI-based tool in enhancing students with vision or hearing impairments and overall educational engagement, user surveys, and feedback are taken, and it is clear that, to a greater extent, it shows the potential and utility of the system to be included in real-world classroom settings. The importance of this paper is particularly based on the contention that it will bring about a major change as far as education is concerned in a bid to make these noble provisions available for students with disabilities. It speeds up access to the required information, fosters differentiation in delivering the instructions, offers quick help, helps improve academic achievement significantly, and helps learners develop confidence in themselves. Further refinements and extensive user evaluations are ongoing to ensure the system meets the diverse needs of students with visual and hearing impairments.

References

Brownlee, J. (2019, June). A Gentle Introduction to Generative Adversarial Networks (GANs). Retrieved 2024-02-29, from https://machinelearningmastery.com/what-are-generative-adversarial-networks-gans/

Feuerriegel, S., Shrestha, Y. R., Von Krogh, G., & Zhang, C. (2022, July). Bringing artificial intelligence to business management. Nature Machine Intelligence, 4(7), 611–613. Retrieved 2024-02-29, from https://www.nature.com/articles/s42256-022-00512-5 doi: 10.1038/s42256- 022- 00512-5

Iqbal, A., Sharif, M., Yasmin, M., Raza, M., & Aftab, S. (2022, September). Generative adversarial networks and its applications in the biomedical image segmentation: a comprehensive survey. International Journal of Multimedia Information Retrieval, 11(3), 333–368. Retrieved 2024-02-29, from https://link.springer.com/10.1007/s13735-022-00240-x doi: 10.1007/s13735-022-00240-x for Blind. International Journal of Innovative and Emerging Research in Engineering, 5(1). Retrieved 2024-02-26, from http://ijiere.com/FinalPaper/finalPaperRaspberry%20Pi%20based%20reader%20for%20Blind191540.pdfdoi:10.26769/IJIERE.2018.5.1.191540

Jadhav, M. S., Koli, S. M., & Kulkarni, S. S. (2018, March). Raspberry Pi Based Reader

Mukhiddinov, M., & Cho, J. (2021, November). Smart Glass System Using Deep Learing-
& Wyner, A. (2022, December). Thirty years of artificial intelligence and law: the third decade. Artificial Intelligence and Law, 30(4), 561–591. Retrieved 2024-02-29, from https://link.springer.com/10.1007/s10506-022-09327-6 doi:10.1007/s10506-022-09327-6

Nemade, S., Patil, R., Bijwe, I., Bhangale, K., & Mapari, R. (2022, May). Artificial Vision- Raspberry Pi Based Reader for Visually
Impaired. Artificial Vision-Raspberry Pi Based Reader for Visually Impaired, 1(1), 1–3.

Pandya, K., & Goradiya, D. B. C. (2021). Audio Assisted Electronic Glasses For Blind & Visually Impaired People Using Deep Learning. ,
16.

Punith, A., Manish, G., Sumanth, M. S., Vinay, A., Karthik, R., & Jyothi, K. (2021). Design and implementation of a smart reader for blind
and visually impaired people. In (p. 060002). Kuching, Malaysia. Retrieved 2024-02-25, from https://pubs.aip.org/aip/ acp/article/1002159 doi: 10.1063/5.0036140

Ramyasri, M., Sridevi, G., Lavanya, K., & Sindhuja, S. (2023, June). Raspberry PiBased Reader for Blind.
Journal on Electronic and Automation Engineering, 2(2), 08–WIBORD as computer assisted learning facilities for
children with visual impair- ment. Journal of Physics: Conference Series, 1835(1), 012080. Retrieved
2024-02-29, from https://iopscience.iop.org/article/10.1088/1742-6596/1835/1/012080 doi: 10.1088/1742
6596/1835/1/012080

Rattanaphinyowanich, T., & Nunta, S. (2021, March). Development of DAISY-

Sarkar, S., Pansare, G., Patel, B., Gupta, A., Chauhan, A., Yadav, R., & Attula, N. B(2021, April). Smart Reader
for Visually Impaired Using Raspberry Pi. IOP Conference Series: Materials Science and Engineering, 1132(1), 012032.
Re- trieved 2024-02-29, from https://iopscience.iop.org/article/10.1088/

The Generator | Machine Learning | Google for Developers. (n.d.). Retrieved 2024-03-24,
fromhttps://developers.google.com/machine-learning/gan/generator

Venkatesan, R., & Li, B. (2017). Convolutional Neural Networks in Visual Computing: A Con-
cise Guide (1st ed.). Boca Raton ; London : Taylor & Francis, CRC Press, 2017.: CRC Press. Retrieved 2024-02-29, from https://www.taylorfrancis.com/books/9781498770408 doi: 10.4324/9781315154282

Villata, S., Araszkiewicz, M., Ashley, K., Bench-Capon, T., Branting, L. K., Conrad, J. G.Wang, X., Wang, C., Liu, B., Zhou, X.,
Zhang, L., Zheng, J., & Bai, X. (2021, December). Multi-view stereo in the Deep Learning Era: A comprehensive review. Displays, 70,
102102. Retrieved 2024-02-29, from https://linkinghub.elsevier.com/retrieve/ pii/S0141938221001062 doi:
10.1016/j.displa.2021.102102

Yeo, J. H., Bae, S. H., Lee, S. H., Kim, K. W., & Moon, N. J. (2022, June). Clinical performance1757-899X/1132/1/012032 doi:
10.1088/1757-899X/1132/1/012032uploads/2023/05/10.46632-jeae-2-2-3.pdf doi: 10.46632/jeae/2/2/3

Wang, J., Wang, S., & Zhang, Y. (2023, April). Artificial intelligence for visually impaired.Displays, 77, 102391. Retrieved 2024-02
28, from https://linkinghub.elsevier.com/retrieve/pii/S0141938223000240 doi: 10.1016/j.displa.2023.102391

World Report on Disability. (n.d.). Retrieved 2024-03-22, from
https://www.who.int/teams/noncommunicablediseases/sensoryfunctionsdisability-and- rehabilitation/world-report-on-disabilityof a
smartphone-based low vision aid. Scientific Reports, 12(1), 10752. Retrieved 2024-02-29, from
https://www.nature.com/articles/s41598-022-14489-z doi:10.1038/s41598-022-14489-z
##submissions.published##
2024-10-25
##issue.issue##
##issue.vol## 8 ##issue.no## 1 (2024)
##section.section##
Articles