10:00   Oral Session 4-GZ – Affect-Based Tagging of Multimedia
Chair: Thierry Pun
10:00
25 mins 		A Bayesian Framework for Video Affective Representation
Mohammad Soleymani, Joep J.M. Kierkels, Guillaume Chanel, Thierry Pun
Abstract: Emotions that are elicited in response to a video scene contain valuable information for multimedia tagging and indexing. The novelty of this paper is to introduce a Bayesian classification framework for affective video tagging that allows taking contextual information into account. A set of 21 full length movies was first segmented and informative content-based features were extracted from each shot and scene. Shots were then emotionally annotated, providing ground truth affect. The arousal of shots was computed using a linear regression on the content-based features. Bayesian classification based on the shots arousal and content-based features allowed tagging these scenes into three affective classes, namely calm, positive excited and negative excited. To improve classification accuracy, two contextual priors have been proposed: the movie genre prior, and the temporal dimension prior consisting of the probability of transition between emotions in consecutive scenes. The f1 classification measure of 54.9% that was obtained on three emotional classes with a naïve Bayes classifier was improved to 63.4% after utilizing all the priors
10:25
25 mins 		Simultaneous exploitation of explicit and implicit tags in affect-based multimedia retrieval
Joep J.M. Kierkels, Thierry Pun
Abstract: Affect-based retrieval of multimedia items requires tags that describe the content of these items. These tags are added by users that interact with the items. In this paper, it is shown to what extent different ways of creating the tags result in similar or non-similar information about the item. Three types of affective tags are being considered here: explicit self-assessed tags, implicit multimedia-based tags, and implicit physiology-based tags. The novelty of this paper is to show that affect-based retrieval accuracy (as measured by precision and recall) benefits from having a database that contains both explicit and implicit tags. A database that contains a mixture of explicit and implicit tags has higher retrieval accuracy when compared to a database that contains uniquely either explicit or implicit tags. This shows that information in explicit and implicit tags is complementary rather than redundant. The improvement in retrieval accuracy is immediately evident when explicit tags are added. Results for low recall rates are of particular importance because these focus on the most relevant items in a database. When over 60% of the items in the database with implicit tags are also tagged by explicit tags, retrieval accuracy at a low recall rate (0.1) is higher than accuracy based uniquely on explicit tags.
10:50
25 mins 		Understanding Affective Interaction: Emotion, Engagement, and Internet Videos
Shaowen L Bardzell, Jeffrey S Bardzell, Tyler M Pace
Abstract: As interest in experience and affect in HCI continues to grow, particularly with regard to social media and Web 2.0 technologies, research on techniques for evaluating user engagement is needed. This paper presents a study of popular Internet videos involving a mixed method approach to user engagement. Instruments included physiological measures, emotional self-report measures, and personally expressive techniques, such as open-ended prose reviews. Using triangulation to interpret the results, we describe relationships among perceived emotion, experienced emotion, video preference, and contextual factors.