NECA Project
Encyclopedia
The NECA Project was a research project that focused on multimodal
communication with animated agents
in a virtual world
. NECA was funded by the European Commission
from 1998-2002 and the research results were published up to 2005.
The project focused on communication between animated agents in a virtual world
, using characters that exhibit realistic personality traits and natural looking behavior that reflects the emotional features
of conversations. The project goal was to combine different research efforts such as situation-based natural language
and speech generation
, representation of non-verbal expression
, and the modeling of emotion
s and personality
.
of the distance between their bodies as they interact, to the kinesics
of body language
at the individual level, and the level of eye contact
between individuals, as well as the paralinguistics
of tone and intonation
of sentences had to be considered.
Based on that the there were three main goals for NECA. The first goal was the general development of a platform that allowed the simulation and interaction of conversational characters.
The second goal was the design of a multi-user web-application called the Socialite, that allowed social "face to face" emotion-based
interactions between animated agents
on the internet. The Socialite user could select a set of avatars
to interact with and after learning about the user's personal preferences, the avatars helped the user navigate a virtual world and get in touch with other agents and users.
The third component was eShowRoom as an e-commerce platform demonstration that allowed for the display of products in the commercial domain. In the eShowRoom application, two or three virtual agents could be seen discussing various features of a product among themselves in a natural setting.
, specifically designed to facilitate the interaction of two or more animated agents. RRL influenced the design of other languages such as the Player Markup Language which extended parts of the design of RRL.
The design of RRL aimed to automatically generate much of the facial animation
as well as the skeletal animation
based on the content of the conversations. Due to the interdependence of nonverbal communication
components such as facial features on the spoken words, no animation is possible in the language without considering the context of the scene
in which the animation takes place - e.g. anger versus joy.
Multimodal interaction
Multimodal interaction provides the user with multiple modes of interfacing with a system. A multimodal interface provides several distinct tools for input and output of data.- Multimodal input :...
communication with animated agents
Computer animation
Computer animation is the process used for generating animated images by using computer graphics. The more general term computer generated imagery encompasses both static scenes and dynamic images, while computer animation only refers to moving images....
in a virtual world
Virtual world
A virtual world is an online community that takes the form of a computer-based simulated environment through which users can interact with one another and use and create objects. The term has become largely synonymous with interactive 3D virtual environments, where the users take the form of...
. NECA was funded by the European Commission
European Commission
The European Commission is the executive body of the European Union. The body is responsible for proposing legislation, implementing decisions, upholding the Union's treaties and the general day-to-day running of the Union....
from 1998-2002 and the research results were published up to 2005.
The project focused on communication between animated agents in a virtual world
Virtual world
A virtual world is an online community that takes the form of a computer-based simulated environment through which users can interact with one another and use and create objects. The term has become largely synonymous with interactive 3D virtual environments, where the users take the form of...
, using characters that exhibit realistic personality traits and natural looking behavior that reflects the emotional features
Affect display
In psychology, affect display or affective display is a subject's externally displayed affect. The display can be by facial, vocal, or gestural means . When displayed affect is different from the subjective affect, it is incongruent affect...
of conversations. The project goal was to combine different research efforts such as situation-based natural language
Natural language generation
Natural Language Generation is the natural language processing task of generating natural language from a machine representation system such as a knowledge base or a logical form...
and speech generation
Speech synthesis
Speech synthesis is the artificial production of human speech. A computer system used for this purpose is called a speech synthesizer, and can be implemented in software or hardware...
, representation of non-verbal expression
Nonverbal communication
Nonverbal communication is usually understood as the process of communication through sending and receiving wordless messages. Messages can be communicated through gestures and touch , by body language or posture, by facial expression and eye contact...
, and the modeling of emotion
Emotion
Emotion is a complex psychophysiological experience of an individual's state of mind as interacting with biochemical and environmental influences. In humans, emotion fundamentally involves "physiological arousal, expressive behaviors, and conscious experience." Emotion is associated with mood,...
s and personality
Personality psychology
Personality psychology is a branch of psychology that studies personality and individual differences. Its areas of focus include:* Constructing a coherent picture of the individual and his or her major psychological processes...
.
Goals and milestones
The underlying research direction of the NECA Project was the development of a computing platform in which animated characters within a virtual world could be capable of realistic behavior. For character interactions to look natural, various factors such the proxemicsProxemics
Proxemics is the study of measurable distances between people as they interact. The term was introduced by anthropologist Edward T. Hall in 1966...
of the distance between their bodies as they interact, to the kinesics
Kinesics
Kinesics is the interpretation of body language such as facial expressions and gestures — or, more formally, non-verbal behavior related to movement, either of any part of the body or the body as a whole.-Birdwhistell's work:...
of body language
Body language
Body language is a form of non-verbal communication, which consists of body posture, gestures, facial expressions, and eye movements. Humans send and interpret such signals almost entirely subconsciously....
at the individual level, and the level of eye contact
Eye contact
Eye contact is a meeting of the eyes between two individuals.In human beings, eye contact is a form of nonverbal communication and is thought to have a large influence on social behavior. Coined in the early to mid-1960s, the term has come in the West to often define the act as a meaningful and...
between individuals, as well as the paralinguistics
Paralanguage
Paralanguage refers to the non-verbal elements of communication used to modify meaning and convey emotion. Paralanguage may be expressed consciously or unconsciously, and it includes the pitch, volume, and, in some cases, intonation of speech. Sometimes the definition is restricted to...
of tone and intonation
Intonation (linguistics)
In linguistics, intonation is variation of pitch while speaking which is not used to distinguish words. It contrasts with tone, in which pitch variation does distinguish words. Intonation, rhythm, and stress are the three main elements of linguistic prosody...
of sentences had to be considered.
Based on that the there were three main goals for NECA. The first goal was the general development of a platform that allowed the simulation and interaction of conversational characters.
The second goal was the design of a multi-user web-application called the Socialite, that allowed social "face to face" emotion-based
Affect display
In psychology, affect display or affective display is a subject's externally displayed affect. The display can be by facial, vocal, or gestural means . When displayed affect is different from the subjective affect, it is incongruent affect...
interactions between animated agents
Avatar (computing)
In computing, an avatar is the graphical representation of the user or the user's alter ego or character. It may take either a three-dimensional form, as in games or virtual worlds, or a two-dimensional form as an icon in Internet forums and other online communities. It can also refer to a text...
on the internet. The Socialite user could select a set of avatars
Avatar (computing)
In computing, an avatar is the graphical representation of the user or the user's alter ego or character. It may take either a three-dimensional form, as in games or virtual worlds, or a two-dimensional form as an icon in Internet forums and other online communities. It can also refer to a text...
to interact with and after learning about the user's personal preferences, the avatars helped the user navigate a virtual world and get in touch with other agents and users.
The third component was eShowRoom as an e-commerce platform demonstration that allowed for the display of products in the commercial domain. In the eShowRoom application, two or three virtual agents could be seen discussing various features of a product among themselves in a natural setting.
Examples of NECA research
One of NECA's designs was the Rich Representation LanguageRich Representation Language
The Rich Representation Language, often abbreviated as RRL, is a computer animation language specifically designed to facilitate the interaction of two or more animated characters. The research effort was funded by the European Commission as part of the NECA Project...
, specifically designed to facilitate the interaction of two or more animated agents. RRL influenced the design of other languages such as the Player Markup Language which extended parts of the design of RRL.
The design of RRL aimed to automatically generate much of the facial animation
Computer facial animation
Computer facial animation is primarily an area of computer graphics that encapsulates models and techniques for generating and animating images of the human head and face. Due to its subject and output type, it is also related to many other scientific and artistic fields from psychology to...
as well as the skeletal animation
Skeletal animation
Skeletal animation is a technique in computer animation in which a character is represented in two parts: a surface representation used to draw the character and a hierarchical set of interconnected bones used to animate the mesh...
based on the content of the conversations. Due to the interdependence of nonverbal communication
Nonverbal communication
Nonverbal communication is usually understood as the process of communication through sending and receiving wordless messages. Messages can be communicated through gestures and touch , by body language or posture, by facial expression and eye contact...
components such as facial features on the spoken words, no animation is possible in the language without considering the context of the scene
Context (language use)
Context is a notion used in the language sciences in two different ways, namely as* verbal context* social context- Verbal context :...
in which the animation takes place - e.g. anger versus joy.
See also
- Facial Action Coding SystemFacial Action Coding SystemFacial Action Coding System is a system to taxonomize human facial expressions, originally developed by Paul Ekman and Wallace V. Friesen in 1978...
- Humanoid animationHumanoid AnimationHumanoid Animation is an approved ISO standard for humanoid modeling and animation. H-Anim defines a specification for defining interchangeable human figures so that those characters can be used across a variety of 3D games and simulation environments.The H-Anim Standard was developed in the late...
- International Computer Science InstituteInternational Computer Science InstituteThe International Computer Science Institute is an independent, non-profit research organization located in Berkeley, California, USA. Since its founding in 1988, ICSI has maintained an affiliation with the University of California, Berkeley, where several of its members hold faculty appointments...
- Interactive Multimodal Information Management (IM2)
- Virtual human