Abstract
The design of intelligent personalized interactive systems, having knowledge about the user’s state, his desires, needs and wishes, currently poses a great challenge to computer scientists. In this study we propose an information fusion approach combining acoustic, and bio-physiological data, comprising multiple sensors, to classify emotional states. For this purpose a multimodal corpus has been created, where subjects undergo a controlled emotion eliciting experiment, passing several octants of the valence arousal dominance space. The temporal and decision level fusion of the multiple modalities outperforms the single modality classifiers and shows promising results.
Chapter PDF
Similar content being viewed by others
References
Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics), 1st edn. Springer, Heidelberg (2006)
Brown, G., Kuncheva, L.I.: “Good” and “Bad” diversity in majority vote ensembles. In: El Gayar, N., Kittler, J., Roli, F. (eds.) MCS 2010. LNCS, vol. 5997, pp. 124–133. Springer, Heidelberg (2010)
Davis, S., Mermelstein, P.: Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustics, Speech and Signal Processing 28(4), 357–366 (1980)
Dietrich, C., Schwenker, F., Palm, G.: Classification of time series utilizing temporal and decision fusion. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 378–387. Springer, Heidelberg (2001)
Ekman, P., Friesen, W.V.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)
Hermansky, H.: The modulation spectrum in automatic recognition of speech. In: Proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 140–147. IEEE, Los Alamitos (1997)
Hermansky, H., Hanson, B., Wakita, H.: Perceptually based linear predictive analysis of speech. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 1985), vol. 10, pp. 509–512 (1985)
Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2067–2083 (2008), http://portal.acm.org/citation.cfm?id=1477073.1477535
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley Interscience, Hoboken (2004)
Maganti, H.K., Scherer, S., Palm, G.: A novel feature for emotion recognition in voice based applications. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 710–711. Springer, Heidelberg (2007)
Peter, J., Lang, M.M.B., Cuthbert, B.N.: International affective picture system (iaps): Affective ratings of pictures and instruction manual. Tech. rep., NIMH Center for the Study of Emotion & Attention, University of Florida (2008)
Picard, R.W.: Affective Computing. MIT Press, Cambridge (2000)
Platt, J.: Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. Advances in Large Margin Classifiers, 61–74 (1999)
Rabiner, L.R., Schafer, R.W.: Digital processing of speech signals. Prentice-Hall Signal Processing Series. Prentice-Hall, Englewood Cliffs (1978)
Russell, J.: A circumplex model of affect. Journal of Personality and Social Psychology 39, 1161–1178 (1980)
Schwenker, F., Dietrich, C., Thiel, C., Palm, G.: Learning of decision fusion mappings for pattern recognition. International Journal on Artificial Intelligence and Machine Learning (AIML) 6, 17–21 (2006)
Smyth, P.: Clustering sequences with hidden markov models. Advances in Neural Information Processing Systems 9, 648–654 (1997)
Tax, D.M.J., van Breukelen, M., Duin, R.P.W., Kittler, J.: Combining multiple classifiers by averaging or by multiplying. Pattern Recognition 33(9), 1475–1485 (2000)
Vapnik, V.N.: The nature of statistical learning theory. Springer-Verlag New York, Inc., New York (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Walter, S. et al. (2011). Multimodal Emotion Classification in Naturalistic User Behavior. In: Jacko, J.A. (eds) Human-Computer Interaction. Towards Mobile and Intelligent Interaction Environments. HCI 2011. Lecture Notes in Computer Science, vol 6763. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21616-9_68
Download citation
DOI: https://doi.org/10.1007/978-3-642-21616-9_68
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21615-2
Online ISBN: 978-3-642-21616-9
eBook Packages: Computer ScienceComputer Science (R0)Springer Nature Proceedings Computer Science
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.