Rich Representation Language
Encyclopedia
The Rich Representation Language, often abbreviated as RRL, is a computer animation
language specifically designed to facilitate the interaction of two or more animated characters. The research effort was funded by the European Commission
as part of the NECA Project
. The NECA (Net Environment for Embodied Emotional Conversational Agents) framework within which RRL was developed was not oriented towards the animation of movies, but the creation of intelligent "virtual characters" that interact within a virtual world
and hold conversations with emotional content, coupled with suitable facial expressions.
RRL was a pioneering research effort which influenced the design of other languages such as the Player Markup Language which extended parts of the design of RRL. The language design specifically intended to lessen the training needed for modeling the interaction of multiple characters within a virtual world and to automatically generate much of the facial animation
as well as the skeletal animation
based on the content of the conversations. Due to the interdependence of nonverbal communication
components such as facial features on the spoken words, no animation is possible in the language without considering the context of the scene
in which the animation takes place - e.g. anger versus joy.
, facial expressions, semantic content
and meaning of conversations, etc. The design challenge is that often information of one type is dependent on another type of information, e.g. the body posture, the facial expression and the semantic content of the conversation need to coordinate. An example is that in an angry conversation, the semantics of the conversation dictate the body posture and facial expressions in a distinct from which is quite different from a a joyful conversation. Hence any commands within the language to control facial expressions must inherently depend on the context of the conversation.
The different types of information used in RRL require different forms of expression within the language, e.g. while semantic information is be represented by grammar
s, the facial expression component requires graphic manipulation primitives.
A key goal in the design of RRL was the ease of development, to make scenes and interaction construction available to users without advanced knowledge of programming. Moreover, the design aimed to allow for incremental development in a natural form, so that scenes could be partially prototyped, then refined to more natural looking renderings, e.g. via the later addition of blinking or breathing.
involved in the scene, including the type and the intensity of the emotions, as well as their causes. The affective reasoner uses emotion dimensions such as intensity and assertiveness.
Although XML is used as the base representation format, the scenes are described at a higher level within an object oriented framework. In this framework nodes (i.e. objects) are connected via arrows or links. For instance, a scene is the top level node which is linked to others. The scene may have three specific attributes: the agents/people who participate in the scene, the discourse representation which provides the basis for conversations and a history which records the temporal relationships between various actions.
The scene descriptions are fed to the natural language generation module which produces suitable sentences. The generation of natural flow in a conversation requires a high degree of representational power for the emotional elements. RRL uses a discourse representation system based the standard method of referents and conditions. The affective reasoner supplies the suitable information to select the words and structures that correspond to specific sentences.
behavior of the gesture assignment module. The speech synthesis component must operate before the gesture assignment system because it includes the timing information for the spoken words and emotional interjection
s. After interpreting the natural language text to be spoken, this component adds prosodic
structure such as rhythm, stress and intonations.
The speech elements, once enriched with stress, intonation and emotional markers are passed to the gesture assignment system. RRL supports three separate aspects of emotion management. First, specific emotion tags may be provided for scenes and specific sentences. A number of specific commands support the display a wide range of emotions in the faces of animated characters.
Secondly, there are built in mechanisms for aligning specific facial features to emotive body postures. Third, specific emotive interjections such as sighs, yawns, chuckles, etc. may be interleaved within actions to enhance the believability of the character's utterances.
The gesture assignment system has specific gesture types such as body movements (e.g. shrug of shoulders as indifference vs hanging shoulders of sadness), emblematic movements (gestures that by convention signal yes/no), iconic (e.g. imitating a telephone via fingers), deictic (pointing gestures), contrast (e.g. on one hand, but on the other hand), facial features (e.g. raised eyebrows, frowning, surprise or a gaze).
Computer animation
Computer animation is the process used for generating animated images by using computer graphics. The more general term computer generated imagery encompasses both static scenes and dynamic images, while computer animation only refers to moving images....
language specifically designed to facilitate the interaction of two or more animated characters. The research effort was funded by the European Commission
European Commission
The European Commission is the executive body of the European Union. The body is responsible for proposing legislation, implementing decisions, upholding the Union's treaties and the general day-to-day running of the Union....
as part of the NECA Project
NECA Project
The NECA Project was a research project that focused on multimodal communication with animated agents in a virtual world...
. The NECA (Net Environment for Embodied Emotional Conversational Agents) framework within which RRL was developed was not oriented towards the animation of movies, but the creation of intelligent "virtual characters" that interact within a virtual world
Virtual world
A virtual world is an online community that takes the form of a computer-based simulated environment through which users can interact with one another and use and create objects. The term has become largely synonymous with interactive 3D virtual environments, where the users take the form of...
and hold conversations with emotional content, coupled with suitable facial expressions.
RRL was a pioneering research effort which influenced the design of other languages such as the Player Markup Language which extended parts of the design of RRL. The language design specifically intended to lessen the training needed for modeling the interaction of multiple characters within a virtual world and to automatically generate much of the facial animation
Computer facial animation
Computer facial animation is primarily an area of computer graphics that encapsulates models and techniques for generating and animating images of the human head and face. Due to its subject and output type, it is also related to many other scientific and artistic fields from psychology to...
as well as the skeletal animation
Skeletal animation
Skeletal animation is a technique in computer animation in which a character is represented in two parts: a surface representation used to draw the character and a hierarchical set of interconnected bones used to animate the mesh...
based on the content of the conversations. Due to the interdependence of nonverbal communication
Nonverbal communication
Nonverbal communication is usually understood as the process of communication through sending and receiving wordless messages. Messages can be communicated through gestures and touch , by body language or posture, by facial expression and eye contact...
components such as facial features on the spoken words, no animation is possible in the language without considering the context of the scene
Context (language use)
Context is a notion used in the language sciences in two different ways, namely as* verbal context* social context- Verbal context :...
in which the animation takes place - e.g. anger versus joy.
Language design issues
The application domain for RRL consists of scenes with two or more virtual characters. The representation of these scenes requires multiple information types such as body posturesBody language
Body language is a form of non-verbal communication, which consists of body posture, gestures, facial expressions, and eye movements. Humans send and interpret such signals almost entirely subconsciously....
, facial expressions, semantic content
Semantics
Semantics is the study of meaning. It focuses on the relation between signifiers, such as words, phrases, signs and symbols, and what they stand for, their denotata....
and meaning of conversations, etc. The design challenge is that often information of one type is dependent on another type of information, e.g. the body posture, the facial expression and the semantic content of the conversation need to coordinate. An example is that in an angry conversation, the semantics of the conversation dictate the body posture and facial expressions in a distinct from which is quite different from a a joyful conversation. Hence any commands within the language to control facial expressions must inherently depend on the context of the conversation.
The different types of information used in RRL require different forms of expression within the language, e.g. while semantic information is be represented by grammar
Grammar
In linguistics, grammar is the set of structural rules that govern the composition of clauses, phrases, and words in any given natural language. The term refers also to the study of such rules, and this field includes morphology, syntax, and phonology, often complemented by phonetics, semantics,...
s, the facial expression component requires graphic manipulation primitives.
A key goal in the design of RRL was the ease of development, to make scenes and interaction construction available to users without advanced knowledge of programming. Moreover, the design aimed to allow for incremental development in a natural form, so that scenes could be partially prototyped, then refined to more natural looking renderings, e.g. via the later addition of blinking or breathing.
Scene description
Borrowing theatrical terminology, each interaction session between the synthetic characters in RRL is called a scene. A scene description specifies the content, timing, and emotional features employed within a scene. A specific module called the affective reasoner computes the emotional primitivesEmotion classification
The means by which we distinguish one emotion from another is a hotly contested issue in emotion research and affective science. This page summarises some of the major theories.-Basic and Complex Emotions:...
involved in the scene, including the type and the intensity of the emotions, as well as their causes. The affective reasoner uses emotion dimensions such as intensity and assertiveness.
Although XML is used as the base representation format, the scenes are described at a higher level within an object oriented framework. In this framework nodes (i.e. objects) are connected via arrows or links. For instance, a scene is the top level node which is linked to others. The scene may have three specific attributes: the agents/people who participate in the scene, the discourse representation which provides the basis for conversations and a history which records the temporal relationships between various actions.
The scene descriptions are fed to the natural language generation module which produces suitable sentences. The generation of natural flow in a conversation requires a high degree of representational power for the emotional elements. RRL uses a discourse representation system based the standard method of referents and conditions. The affective reasoner supplies the suitable information to select the words and structures that correspond to specific sentences.
Speech synthesis and emotive markers
The speech synthesis component is highly dependent on the semantic information and thebehavior of the gesture assignment module. The speech synthesis component must operate before the gesture assignment system because it includes the timing information for the spoken words and emotional interjection
Interjection
In grammar, an interjection or exclamation is a word used to express an emotion or sentiment on the part of the speaker . Filled pauses such as uh, er, um are also considered interjections...
s. After interpreting the natural language text to be spoken, this component adds prosodic
Prosody (linguistics)
In linguistics, prosody is the rhythm, stress, and intonation of speech. Prosody may reflect various features of the speaker or the utterance: the emotional state of the speaker; the form of the utterance ; the presence of irony or sarcasm; emphasis, contrast, and focus; or other elements of...
structure such as rhythm, stress and intonations.
The speech elements, once enriched with stress, intonation and emotional markers are passed to the gesture assignment system. RRL supports three separate aspects of emotion management. First, specific emotion tags may be provided for scenes and specific sentences. A number of specific commands support the display a wide range of emotions in the faces of animated characters.
Secondly, there are built in mechanisms for aligning specific facial features to emotive body postures. Third, specific emotive interjections such as sighs, yawns, chuckles, etc. may be interleaved within actions to enhance the believability of the character's utterances.
Gesture assignment and body movements
In RRL the term gesture is used in a general sense and applies to facial expressions, body posture and proper gestures. Three levels of information are processed within gesture assignment:- Assignment of specific gestures within a scene to specific modules, e.g. "turn taking" being handled in the natural language generation module.
- Refinement and elaboration of gesture assignment following a first level synthesis of speech, e.g. the addition of blinking and breathing to a conversation.
- Interface to external modules that handle player-specific renderings such as MPEG-4MPEG-4MPEG-4 is a method of defining compression of audio and visual digital data. It was introduced in late 1998 and designated a standard for a group of audio and video coding formats and related technology agreed upon by the ISO/IEC Moving Picture Experts Group under the formal standard ISO/IEC...
facial animation parameters (FAPs).
The gesture assignment system has specific gesture types such as body movements (e.g. shrug of shoulders as indifference vs hanging shoulders of sadness), emblematic movements (gestures that by convention signal yes/no), iconic (e.g. imitating a telephone via fingers), deictic (pointing gestures), contrast (e.g. on one hand, but on the other hand), facial features (e.g. raised eyebrows, frowning, surprise or a gaze).