Resources - hpache/SignLanguageWebsite GitHub Wiki
Project Shuwa
Project Shuwa is a pretty neat project dedicated to teaching people JSL and HKSL through a web interface. They used TensorFlow.js along with like 4 other models which recognize hand gestures, facial expressions, and arm movements all of which are needed for different sign language gestures. The Project Shuwa GitHub repo has a lot of different resources, and it feels like it's all over the place, so I think it would be a good idea to have all those resources in one page.
Documentation
Documentation for Project Shuwa can be found on their GitHub page. Since we will be using this for the gestures, I suggest cloning the GitHub repo immediately and playing around with some of its methods.
Web Demo
Project Shuwa has a web demo in its repo but it can only be run locally. This means that you need to clone the repo to your computer! Once you clone the repo, cd to the web_demo directory and run the following commands
yarn update(looks into the yarn.lock file and downloads dependencies)yarn start(Starts the website locally, should be on localhost:8000)
The demo will ask you for permission for your webcam and then have you pick a sign language. Then you will be prompted to pick a gesture and repeat that gesture, you will then be shown the predictions that the ML model makes, it is stupidly accurate.
More information can be found in this link
Project Shuwa Web Demo Instructions
Training Classifier
Project Shuwa only supports JSL and HKSL, since we want to do ASL, we will need to train the classifier with our own data. Before we start anything, we will need to gather our own ASL datasets and train the model to recognize gestures. Since we only have a month to do this, I want to start with the ASL alphabet since it is the most simple thing to gather data for and also train (only 26 different gestures). I think we should also have the ability to let a user upload a gesture to gather more data, that way we get to use a database, and our model will get increased accuracy when more people use it.