Synopses & Reviews
With contributions by leading scientists in the field, this book gives the first comprehensive overview of the results of the seminal SmartKom project - one of the most advanced multimodal dialogue systems worldwide.
Review
From the reviews: "The book is organized into six parts, each of which contains several chapters. ... This book should be useful for many types of readers. Those who want to get a sense of the state of the art of multimodal communication ... will find it invaluable. Students who have had a good grounding in artificial intelligence or natural language processing should read the book to get an overview of how the techniques from these fields can be integrated into a comprehensive system." (J. P. E. Hodgson, ACM Computing Reviews, Vol. 49 (4), April, 2008)
Review
From the reviews:
"The book is organized into six parts, each of which contains several chapters. ... This book should be useful for many types of readers. Those who want to get a sense of the state of the art of multimodal communication ... will find it invaluable. Students who have had a good grounding in artificial intelligence or natural language processing should read the book to get an overview of how the techniques from these fields can be integrated into a comprehensive system." (J. P. E. Hodgson, ACM Computing Reviews, Vol. 49 (4), April, 2008)
Synopsis
Dialogue Systems Go Multimodal: The SmartKom Experience.- Facts and Figures About the SmartKom Project.- An Exemplary Interaction with SmartKom.- Multimodal Input Analysis.- The SmartKom Architecture: A Framework for Multimodal Dialogue Systems.- Modeling Domain Knowledge: Know-How and Know-What.- Speech Recognition.- Class-Based Language Model Adaptation.- The Dynamic Lexicon.- The Prosody Module.- The Sense of Vision: Gestures and Real Objects.- The Facial Expression Module.- Multiple Biometrics.- Natural Language Understanding.- The Gesture Interpretation Module.- Multimodal Dialogue Processing.- Modality Fusion.- Discourse Modeling.- Overlay: The Basic Operation for Discourse Processing.- In Context: Integrating Domain- and Situation-Specific Knowledge.- Intention Recognition.- Plan-Based Dialogue Management for Multiple Cooperating Applications.- Emotion Analysis and Emotion-Handling Subdialogues.- Problematic, Indirect, Affective, and Other Nonstandard Input Processing.- Multimodal Output Generation.- Realizing Complex User Wishes with a Function Planning Module.- Intelligent Integration of External Data and Services into SmartKom.- Multimodal Fission and Media Design.- Natural Language Generation with Fully Specified Templates.- Multimodal Speech Synthesis.- Scenarios and Applications.- Building Multimodal Dialogue Applications: System Integration in SmartKom.- SmartKom-English: From Robust Recognition to Felicitous Interaction.- SmartKom-Public.- SmartKom-Home: The Interface to Home Entertainment.- SmartKom-Mobile: Intelligent Interaction with a Mobile System.- SmartKom-Mobile Car: User Interaction with Mobile Services in a Car Environment.- Data Collection and Evaluation.- Wizard-of-Oz Recordings.- Annotation of Multimodal Data.- Multimodal Emogram, Data Collection and Presentation.- Empirical Studies for Intuitive Interaction.- Evaluation of Multimodal Dialogue Systems.
Synopsis
The result of four years of intensive research in a large multimodal dialogue project involving 12 partners from academia and industry, SmartKom is one of the most advanced multimodal dialogue systems worldwide and is a landmark project in the history of intelligent user interfaces. The system provides symmetric multimodality in a mixed-initiative dialogue system with an embodied conversational agent. The same software architecture and components are used in three fully operational application scenarios. The theoretical and practical foundations of SmartKom represent a new generation of multimodal dialogue systems that deal not only with simple modality integration and synchronization, but cover the full spectrum of multimodal dialogue. With contributions by leading scientists in the field, this book gives the first comprehensive overview of the results of this seminal project.
About the Author
Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster is the Director and CEO of the German Research Center for Artificial Intelligence (DFKI GmbH) and a Professor of Computer Science at the Universität des Saarlandes, Saarbrücken. In 2000, he was coopted as a Professor of Computational Linguistics at the same university. In addition, he is the Head of the Intelligent User Interfaces Lab at DFKI. He was the Scientific Director of the VERBMOBIL consortium on spontaneous speech translation (1993-2000) as well as the SmartKom consortium on multimodal dialog systems (1999-2003) and currently serves as the Scientific Director of the SmartWeb consortium on mobile multimodal access to semantic web services (2004-2008). He has published more than 150 technical papers and 6 books on language technology and intelligent user interfaces. His current research includes multimodal and perceptive user interfaces, user modeling, embodied conversational agents, smart navigation systems, semantic web services, and resource-adaptive cognitive technologies. He is the editor of the book "Verbmobil: Foundations of Speech-to-Speech Translation" (Springer) and the Co-Editor of the Readings in Intelligent User Interfaces.