| dc.description.abstract | In digital games, sound and music play an important role in players game experience, as they are usually used to convey important information and also to contribute to game atmosphere. Losing the reception of that stimulus may impose barriers to game play, rendering players who cannot hear or that have some degree of hearing loss, at least, disadvantaged towards hearing players. This accessibility issue is mostly tackled by providing subtitles and textual representations of sounds called closed captions, as can be seen on general accessibility guidelines and in a few mainstream games. However, not everyone may fully benefit from the use of textual alternatives for sounds. In fact, under certain conditions, people that become deaf before having enough experience with the sound world may present difficulties in reading, as the oral language acquisition (e.g. English or Portuguese) must occur as a second language. In this work, we investigate the impact of not receiving audio stimulus on game experience. As this issue can be depicted as a communication problem, we resorted to Semiotic Engineering and the Semiotic Inspection Method to identify systematically how sound and music are used in game design to convey information. We took as case study a first-person shooter game called Half-Life 2, which has been well appraised by deaf and hard of hearing player for offering the closed captions alternative. As a result of the inspection, we identified the use of eight distinct features of the sound effects to convey information (e.g. perceived volume to communicate distance from the sound source), which were used in seven different communication strategies (e.g. feedback). Those features and strategies were then evaluated according to their relevance to game play. We also found that the closed caption system used in the game does not communicate part of the information conveyed through audio. With the identified sound effect features and the results from an exploratory questionnaire we had applied with deaf and hard of hearing people, we propose an alternative for the closed captions that presents sounds using visual metaphors. It is based on the concept of Synaesthesia, a neurological condition in which sensations from one stimulus is perceived through different senses. As an experience, we created a modification of the game Half-Life 2 to evaluate whether the Synaesthesia approach could be used to enhance game experience of deaf and hard of hearing players. The modification used a simple implementation of the Synaesthesia model, which was tested with deaf and hearing players to evaluate the quality of the transmission of the information from sounds. The results showed that the implementation was able to communicate visually sound content that normally would not be conveyed to non-hearing players, thus complementing the other assistive resources of the game. | |
