|
Document Type |
: |
Thesis |
|
Document Title |
: |
IMPROVING ARABIC SIGN LANGUAGE TO SUPPORT COMMUNICATION BETWEEN VEHICLE DRIVERS AND PASSENGERS FROM DEAF PEOPLE. تحسين لغة الإشارة العربية لدعم التواصل ما بين سائقي المركبات والراكبين من فئة الصم. |
|
Subject |
: |
Faculty of Computing and Information Technology |
|
Document Language |
: |
Arabic |
|
Abstract |
: |
According to Saudi Arabia's Vision 2030, our country seeks to improve the citizen's lifestyle and integrate the special needs category with ordinary citizens in the workplace and life in general. One of the barriers faced by people with special needs, especially people who are deaf in their lives and work environment, is working as drivers of vehicles.
In this scientific research, the researcher seeks to find a proposed system that contributes to solving the problem of communication between the deaf driver and non-deaf passenger or vice versa by using artificial intelligence techniques instead of using the personal translator.
The researcher collects and creates an adaptable dictionary (text dataset) for the deaf driver system. Then, the researcher creates and annotates the ArSL video corpus based on the sign words that create in cooperation with experts and volunteers in the ArSL. Where the total signers are four people, including three from the deaf people, and one is an expert in ArSL but not deaf. These videos consider a corpus dataset for the proposed system; after creating an ArSL video corpus, the researcher testes and evaluates using two methods. The first method is a quantitative as a traditional method (a questionnaire to verify the validity of ArSL videos with four people, two of them are deaf and two not deaf but experts in the ArSL). The second method is an advanced statistical method (Kappa factor to measure the rater reliability between ArSL videos made by three deaf people). The researcher does the evaluation for deaf driver corpus in order to adopt the correct data in terms of deaf utilizing and understanding. The researcher designs the architecture of the proposed system based two-phase. The first phase is from voice into ArSL in addition to a written text that can be read. The second phase is recognizing the ArSL and convert it into a spoken voice that the non-deaf passenger or driver understands, in addition to written text appear. The designing system takes into account the principles and rules of the Arabic and ArSL language.
This proposed system for the first phase implements using Python that supports voice and image recognition and many libraries specialized in the field of artificial intelligence and natural language processing. The second phase of the proposed system will be in future work.
The research result for evaluation of 215 ArSL video corpus dataset that created and annotated is 10% of WER with using traditional method while using the advance method (Cohens Kappa with measuring agreement criteria) is 61% between signer 2 and signer 3, which means good agreement and it is useful for using their ArSL video corpus in implementation. Also, in order to enhance and facilitate the communication between deaf drivers and passengers, the researcher uses the Knowledge-Based (KBMT) approach for translating the Arabic text into the ArSL sign. This approach used a matching algorithm for converting the source “speech” into the target “sign”, which the accuracy is 99%. Moreover, the researcher uses this matching algorithm because the ArSL video corpus is small. |
|
Supervisor |
: |
Dr. Fahad M. Al-Otaibi |
|
Thesis Type |
: |
Master Thesis |
|
Publishing Year |
: |
1442 AH
2020 AD |
|
Co-Supervisor |
: |
Prof. Hassanin M. Al-Barhamtoshy |
|
Added Date |
: |
Sunday, August 23, 2020 |
|
Researchers
| سماح أنور عباس | Abbas, Samah Anwar | Researcher | Master | |
|