Immersive virtual musical instrument
Encyclopedia
An immersive
virtual musical instrument, or immersive virtual environment for music and sound, represents sound processes and their parameters as 3D entities of a virtual reality
so that they can be perceived not only through auditory feedback but also visually in 3D and possibly through tactile as well as haptic feedback, using 3D interface metaphors
consisting of interaction techniques
such as navigation, selection and manipulation (NSM). It builds on the trend in electronic musical instruments to develop new ways to control sound and perform music such as explored in conferences like NIME.
Jared Bott created an environment that used 3D spatial control techniques as used in known musical instruments, with symbolic 2D visual representation of musical events.
Richard Polfreman made a 3D virtual environment for musical composition with visual representations of musical and sound data similar to 2D composition environments but placed in a 3D space.
Leonel Valbom created a 3D immersive virtual environment with visual 3D representations of musical events and audio spatialization with which could be interacted using NSM interaction techniques.
Teemu Mäki-Patola explored interaction metaphors based on existing musical instruments as seen in his Virtual Xylophone, Virtual Membrane, and Virtual Air Guitar implementations.
Axel Mulder explored the sculpting interaction metaphor by creating a 3D virtual environment that allowed interaction with abstract deformable shapes, such as a sheet and a sphere, which parameters were mapped to sound effects in innovative ways. The work focused on proving the technical feasibility of 3D virtual musical instruments. Gestural control was based on 3D object manipulation such as a subset of prehension.
Early work was done by Jaron Lanier
with his Chromatophoria band and separately by Niko Bolas
who developed the Soundsculpt Toolkit, a software interface that allows the world of music to communicate with the graphical elements of virtual reality.
Immersion (virtual reality)
Immersion is the state of consciousness where an immersant's awareness of physical self is diminished or lost by being surrounded in an engrossing total environment; often artificial. This mental state is frequently accompanied with spatial excess, intense focus, a distorted sense of time, and...
virtual musical instrument, or immersive virtual environment for music and sound, represents sound processes and their parameters as 3D entities of a virtual reality
Virtual reality
Virtual reality , also known as virtuality, is a term that applies to computer-simulated environments that can simulate physical presence in places in the real world, as well as in imaginary worlds...
so that they can be perceived not only through auditory feedback but also visually in 3D and possibly through tactile as well as haptic feedback, using 3D interface metaphors
Interface metaphor
An Interface metaphor is a set of user interface visuals, actions and procedures that exploit specific knowledge that users already have of other domains. The purpose of the interface metaphor is to give the user instantaneous knowledge about how to interact with the user interface...
consisting of interaction techniques
Interaction technique
An interaction technique, user interface technique or input technique is a combination of hardware and software elements that provides a way for computer users to accomplish a single task. For example, one can go back to the previously visited page on a Web browser by either clicking a button,...
such as navigation, selection and manipulation (NSM). It builds on the trend in electronic musical instruments to develop new ways to control sound and perform music such as explored in conferences like NIME.
State of the art
Florent Berthaut created a variety of 3D reactive widgets involving novel representations of musical events and sound, that required a special 3D input device to interact with them using adapted 3D interaction techniques.Jared Bott created an environment that used 3D spatial control techniques as used in known musical instruments, with symbolic 2D visual representation of musical events.
Richard Polfreman made a 3D virtual environment for musical composition with visual representations of musical and sound data similar to 2D composition environments but placed in a 3D space.
Leonel Valbom created a 3D immersive virtual environment with visual 3D representations of musical events and audio spatialization with which could be interacted using NSM interaction techniques.
Teemu Mäki-Patola explored interaction metaphors based on existing musical instruments as seen in his Virtual Xylophone, Virtual Membrane, and Virtual Air Guitar implementations.
Axel Mulder explored the sculpting interaction metaphor by creating a 3D virtual environment that allowed interaction with abstract deformable shapes, such as a sheet and a sphere, which parameters were mapped to sound effects in innovative ways. The work focused on proving the technical feasibility of 3D virtual musical instruments. Gestural control was based on 3D object manipulation such as a subset of prehension.
Early work was done by Jaron Lanier
Jaron Lanier
Jaron Zepel Lanier is an American computer scientist, best known for popularizing the term virtual reality .A pioneer in the field of VR, Lanier and Thomas G. Zimmerman left Atari in 1985 to found VPL Research, Inc., the first company to sell VR goggles and gloves...
with his Chromatophoria band and separately by Niko Bolas
Niko Bolas
Niko Bolas is an American music producer, sound engineer, and consultant and business developer in the fields of virtual reality and Internet radio...
who developed the Soundsculpt Toolkit, a software interface that allows the world of music to communicate with the graphical elements of virtual reality.