Project One

I was moved into the Assistive Technology group halfway through my project one semester. I was to work with the Oculus Rift and Pupil Labs headset to track the user’s eye movements to see where they were looking inside virtual reality, as Pupil Labs had hardware designed to fit inside the Oculus Rift. My initial tasks where to familiarise myself with the hardware I had received, read documentation on the devices and build a computer that could run the Oculus Rift without issues, while my overall objective was to extract the location on screen the user was focussed on from the Pupil Labs hardware.

I quickly found a computer that would be suitable for my project but was delayed until the purchase had been finalised. In the meantime I visited the Oculus Rift developer documentation and Pupil Labs GitHub pages daily learning everything I could about the hardware. Once the parts to the computer arrived, I assembled them computer and got the Oculus Rift up and running. Before the end of semester, I had managed to build an application that gave the X and Y coordinates the user wearing the Pupil Labs headset was looking at on the screen. A GitHub repository I made containing this code can be found here.

I found this semester challenging as I was not only working with hardware that many people have not had experience working with, but also I was working alone. Although it was challenging, I enjoyed getting the chance to work with brand new hardware and building a useful application. It was also challenging reading a lot of documentation, but found out first-hand how important reading the documentation can be.

Project Two

The start of my project two semester began with finding out Oculus were working themselves on eye tracking for the Oculus Rift now, making my work redundant. This meant I was now tasked with creating an application that would allow users who had lost the ability to use their arms, the chance to control the use of the Oculus Rift with just their eyes. I also had a project one student join my team.

We quickly went back to research and reading documentation on ways we could implement this and decided to build an application using Unity. After sometime attempting this, we ran into issues of running multiple applications at one time inside the Oculus Rift. We had only found one example of this being done before, by a company named V who unfortunately it closed source. This issue was beginning to cause us stress, so the decision was made to ask some external people for help. Emails were sent, and forums were posted on but still we had no luck. In the end the decision to abandon the project was made, and I was moved to another project.

I moved to the Iconic Keyboard group, where they were tasked with creating an on screen keyboard that was controlled by the user’s gaze. The keyboard however would not contain letters but icons instead for user’s that had physical and/or cognitive impairment. The group had two extra members making four of us in total and already had decent progress made.

My first tasks in the group was to familiarise myself with the code base and documentation, and also the OptiKey and OptiKey Lite repositories that Icon Keyboard is based on. I then created keyboards for people, Sport, Emotions, Gestures, Clothes, Household, Entertainment and Time. Issues occurred with some of the images used for icons being different sizes, resulting in some images being tiny on screen. To fix this, I made a Photoshop action that all images would be run through before adding them to the project, making them all the same size and resolution.

With the project ready for deployment, I was tasked with creating an installer to allow users around the world to download our application. I did this using InstallShield which allows you to build an installer from inside Visual Studio, taking the required project files and building a set up wizard for users to install Iconic Keyboard.

Throughout this semester I encountered many issues. Working with a team had both its pros and cons. It was great to work with other people on a large scale project for the first time and viewing ideas from another angle that I had not thought about. However, we had several merge conflicts using GitHub with multiple members editing the same file at once. It was also daunting diving into the middle of a large code base but gave me a good idea of what to expect when leaving Polytech. Switching between projects and groups was also frustrating at times as it felt like I was constantly having to start again.