Gesture recognition technology is software that allows computers to analyze human movements to provide interactive features. For this project, we selected a popular online application and remapped the controls for hand gesture interactions. To complete this task, we took various factors into consideration for selecting an application, including ease of user movement and ease of interaction. Once decided, we tested initial gesture ideations for performance and made the needed modifications to enhance the user experience. Within this design document, we will discuss the process of choosing an application, ideating which gestures to implement for common controls, interactive testing, and the implementation process.
The process began through the research of compatible applications. In this stage, each group member researched various applications to understand what interactions were supported and tested the difficulties of these interactions.
Examples of some applications we looked into included:
During the research process, various factors were considered in correlation to usability. First, we considered Fitts’s Law to determine if any controls would be difficult for the user to click/drag through gestures alone. This narrowed down our decision pool to mostly applications that are able to be controlled through keyboard commands. Additionally, speed and direct/indirect manipulation were taken into account. Speed became a factor in the decision-making process because we realized that, between the combined reaction time of the user and gesture recognition software, delayed movement times could become frustrating to the user. Direct manipulation was an important element because we wanted the user to immediately see their actions and be able to undo movements if the incorrect hand gesture was mistakenly thrown or there was misinterpretation from the software.
At the conclusion of this process, our group narrowed down our candidates to:
2048 Gameplay
Tank Wars Gameplay
2048 was one candidate because it does not require a fast reaction time, the user is immediately able to see the results of their gestures, and the game is able to be fully controlled through keyboard input.
Tank Wars was an additional candidate because it is able to be fully controlled through keyboard input once the game has begun, shows results to the input commands, and most movements are able to be undone apart from the “fire” command. However, this game does rely on input reaction time to fire from the tank with the desired power.
In the gesture ideation stage, we brainstormed potential gestures to correspond to the needed input actions for each game. In this process, we narrowed down our chosen game to 2048. This game was chosen because of the lack of reaction time needed to perform essential features of the game and its ability to be fully controlled through keyboard input. After researching common hand gestures used for the recognition software, we generated a list of hand gestures and their corresponding commands: