Title: Emotional control and visual representation using advanced audiovisual interaction

Authors: Vasillis Psarras; Andreas Floros; Konstantinos Drosos; Marianne Strapatsakis

Addresses: Camberwell College of Arts, University of the Arts London, Peckham Road, London SE5 8UF, UK ' Department of Audiovisual Arts, Ionian University, Plateia Tsirigoti 7, Corfu 49100, Greece ' Department of Audiovisual Arts, Ionian University, Plateia Tsirigoti 7, Corfu 49100, Greece ' Department of Audiovisual Arts, Ionian University, Plateia Tsirigoti 7, Corfu 49100, Greece

Abstract: Modern interactive means combined with new digital media processing and representation technologies can provide a robust framework for enhancing user experience in multimedia entertainment systems and audiovisual artistic installations with non-traditional interaction/feedback paths based on user affective state. In this work, the ‘Elevator’ interactive audiovisual platform prototype is presented, which aims to provide a framework for signalling and expressing human behaviour related to emotions (such as anger) and finally produce a visual outcome of this behaviour, defined here as the emotional ‘thumbnail’ of the user. Optimised, real-time audio signal processing techniques are employed for monitoring the achieved anger-like behaviour, while the emotional elevation is attempted using appropriately selected combined audio/visual content reproduced using state-of-the-art audiovisual playback technologies that allow the creation of a realistic immersive audiovisual environment. The demonstration of the proposed prototype has shown that affective interaction is possible, allowing the further development of relative artistic and technological applications.

Keywords: affective interaction; voice anger detection; audio; emotions; emotional control; audiovisual arts; audiovisual interaction; AV interaction; user experience; multimedia entertainment.

DOI: 10.1504/IJART.2011.043446

International Journal of Arts and Technology, 2011 Vol.4 No.4, pp.480 - 498

Received: 25 Nov 2009
Accepted: 29 Oct 2010
Published online: 11 Jan 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article