Physiology-based control of affective audio-visual database

Study of affective modulation through a rated audio-visual database

Could we create narratives that adapt to the viewer and vice-versa?

Doing so, we could create media content that understands emotions in real-time and therefore change itself to modulate our watching and listening experiences.

Adaptive Narrative

Foreseeable uses cases of this type of media could be:

  • Treatment for disorders such as chronic pain, depression, etc
  • Interactive installations
  • Plot modulation in films
  • Adaptive music
  • Relaxation devices

During my Master Thesis, I studied human emotions and how audiovisual content could influence the human affective state. I worked on a biofeedback system that uses emotion-rated media clips (IAPS database) to create "pseudo-narratives" that influence the affective state of the user which at the same time influences the narrative.

System Diagram 1

The system was developed using g-tec hardware, custom-made software using Matlab for the signal processing and java for the adaptative media player.

System Diagram 2

The visual stimuli used during the experiments was taken from the International Affective Picture System IAPS automatically with the software using the ratings. To better understand the database, I created a custom visualization tool to display the how the media clips are rated in the database and therefore know which clips are chosen during the "pseudonarrative" creation.

IAPS visualizarion
  • 12/2009 - 09/2010
  • universitythesisJavaMatlabGSR Sensor
  • Context

    MSc Thesis. CSIM Master UPF