Unheard
Voices

I selected to represent the ISL community because out of 63 million(2023) people suffering from
Significant Auditory Impairment in India, there are less than 300 ISL certified interpreters.

So I decided to make a two communication; live recognizing program and pamphlet to bridge gap.

The initial ideation and selection of topic along with the plan overview
I did a survey and interview on few people from
different age groups to ask about their experiences in 3
languages- English (formal language), Hindi ( the most spoken
language) and Bengali(regional language).
Almost all agreed, they would like a new way of communication rather than
being pitied or looked upon.
Here, I tried to use Google’s Teachable Machine (CNN model) but the accuracy
was horribly low as the hand colors or background itself changed.

I needed to use and attach a pre-scanning and background extraction software to
source code if I was able to obtain it. I wasn’t able to thus used print out of
manually done hand signs but it defeated the purpose for me

SO RESEARCHING...

Indian Sign Language Generation System- Sugandhi, Parteek Kumar, and Sanmeet Kaur
Sign Language Translation Systems for Hearing/Speech Impaired People: A Review- Yuvraj Grover, Prasant K. Gupta, Riya Aggarwal, Deepak Sharma
Indian Sign Language Recognition using Convolutional Neural Network -Rachana Patil, Vivek Patil, Abhishek Bahuguna, Gaurav Datkhile
Review on Text and Speech Conversion Techniques based on Hand Gesture- Gunasagari G. S., Achuth M., Chetana S. R.,...
An Efficient Approach for Interpretation of Indian Sign Language using Machine Learning- Dhivyasri S, Sona M. Akash M.,...
TouchDesigner Kinect CHOP Tutorial Stream (uncut recording)
Interactive Art Demo + New Particle Based Work from Scratch :: Touchdesigner & Kinect
ASL Tutor -- Leap Motion + machine learning to recognize sign language -- TAMUHack 2015
Custom Hand Gesture Recognition with Hand Landmarks Using Google’s Mediapipe + OpenCV in Python
Touchdesigner- American Sign Language Reader
Real Time Sign Language Detection with Tensorflow Object Detection and Python | Deep Learning SSD
American Sign Language (ASL) Alphabet Recognition Using Leap Motion Controller
Face, Hand, Pose Tracking & More in TouchDesigner with @MediaPipe GPU Plugin
Solution
Actually Google’s open source software but I found a plugin
for the Touch Designer a live time programming software I use
After Data Gathering I set some constraints and here’s
the end network for A,B,C,D hand signs for time being
Refer to ‘Pamphlet’ to see the development of graphics
prototype

Final

According to interview, the younger generation was more proactive in finding features like live transcribe, read aloud while older
were not so tech savvy and often heard it from children or relatives. So I set primary audience of this as the younger generation.

For art style I wanted to imitate the NCERT books, the school books which are made on cheap paper with gaudy colors.
Some of my inspiration, I wanted the color and illustration to pop like pop are

Some pictures which initially I was planning to use for training AI model

I tried to combine gallery print and the pictures

Because it was a double sided pamphlet, I decided to make some abstract backgrounds
Just Reused some of the galley prints and hand cut outs to make the background of program

Final