MPEG-7
Encyclopedia
MPEG-7 is a multimedia
content
description standard
. It was standardized in ISO/IEC
15938 (Multimedia content description interface). This description will be associated with the content itself, to allow fast and efficient searching for material that is of interest to the user. MPEG-7 is formally called Multimedia Content Description Interface. Thus, it is not a standard which deals with the actual encoding of moving pictures and audio, like MPEG-1
, MPEG-2
and MPEG-4
. It uses XML
to store metadata, and can be attached to timecode in order to tag particular events, or synchronise lyrics
to a song
, for example.
It was designed to standardize:
The combination of MPEG-4
and MPEG-7 has been sometimes referred to as MPEG-47.
Description and deploy it. According to the
requirements document,1 “a Description consists
of a Description Scheme (structure) and the
set of Descriptor Values (instantiations) that
describe the Data.” A Descriptor Value is “an
instantiation of a Descriptor for a given data set
(or subset thereof).”
Neither automatic nor
semiautomatic feature
extraction algorithms are
inside the scope of the
standard because their
standardization isn’t required
to allow interoperability.
An MPEG-7 architecture requirement is that description must be separate from the audiovisual content.
On the other hand there must be a relation between the content and description. Thus the description is multiplexed with the content itself.
On the right side you can see this relation between description and content.
MPEG-7 uses the following tools:
On the right side you can see the relation between MPEG-7 tools.
Multimedia
Multimedia is media and content that uses a combination of different content forms. The term can be used as a noun or as an adjective describing a medium as having multiple content forms. The term is used in contrast to media which use only rudimentary computer display such as text-only, or...
content
Content (media and publishing)
In media production and publishing, content is information and experiences that may provide value for an end-user/audience in specific contexts. Content may be delivered via any medium such as the internet, television, and audio CDs, as well as live events such as conferences and stage performances...
description standard
Standardization
Standardization is the process of developing and implementing technical standards.The goals of standardization can be to help with independence of single suppliers , compatibility, interoperability, safety, repeatability, or quality....
. It was standardized in ISO/IEC
International Electrotechnical Commission
The International Electrotechnical Commission is a non-profit, non-governmental international standards organization that prepares and publishes International Standards for all electrical, electronic and related technologies – collectively known as "electrotechnology"...
15938 (Multimedia content description interface). This description will be associated with the content itself, to allow fast and efficient searching for material that is of interest to the user. MPEG-7 is formally called Multimedia Content Description Interface. Thus, it is not a standard which deals with the actual encoding of moving pictures and audio, like MPEG-1
MPEG-1
MPEG-1 is a standard for lossy compression of video and audio. It is designed to compress VHS-quality raw digital video and CD audio down to 1.5 Mbit/s without excessive quality loss, making video CDs, digital cable/satellite TV and digital audio broadcasting possible.Today, MPEG-1 has become...
, MPEG-2
MPEG-2
MPEG-2 is a standard for "the generic coding of moving pictures and associated audio information". It describes a combination of lossy video compression and lossy audio data compression methods which permit storage and transmission of movies using currently available storage media and transmission...
and MPEG-4
MPEG-4
MPEG-4 is a method of defining compression of audio and visual digital data. It was introduced in late 1998 and designated a standard for a group of audio and video coding formats and related technology agreed upon by the ISO/IEC Moving Picture Experts Group under the formal standard ISO/IEC...
. It uses XML
XML
Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....
to store metadata, and can be attached to timecode in order to tag particular events, or synchronise lyrics
Lyrics
Lyrics are a set of words that make up a song. The writer of lyrics is a lyricist or lyrist. The meaning of lyrics can either be explicit or implicit. Some lyrics are abstract, almost unintelligible, and, in such cases, their explication emphasizes form, articulation, meter, and symmetry of...
to a song
Song
In music, a song is a composition for voice or voices, performed by singing.A song may be accompanied by musical instruments, or it may be unaccompanied, as in the case of a cappella songs...
, for example.
It was designed to standardize:
- a set of Description Schemes ("DS") and Descriptors ("D")
- a language to specify these schemes, called the Description Definition LanguageDescription Definition LanguageDDL is part of the MPEG-7 standard. It gives an important set of tools for the users to create their own Description Schemes and Descriptors...
("DDL") - a scheme for coding the description
The combination of MPEG-4
MPEG-4
MPEG-4 is a method of defining compression of audio and visual digital data. It was introduced in late 1998 and designated a standard for a group of audio and video coding formats and related technology agreed upon by the ISO/IEC Moving Picture Experts Group under the formal standard ISO/IEC...
and MPEG-7 has been sometimes referred to as MPEG-47.
Introduction
MPEG-7 is intended to provide complementary functionality to the previous MPEG standards, representing information about the content, not the content itself ("the bits about the bits"). This functionality is the standardization of multimedia content descriptions. MPEG-7 can be used independently of the other MPEG standards - the description might even be attached to an analog movie. The representation that is defined within MPEG-4, i.e. the representation of audio-visual data in terms of objects, is however very well suited to what will be built on the MPEG-7 standard. This representation is basic to the process of categorization. In addition, MPEG-7 descriptions could be used to improve the functionality of previous MPEG standards.With these tools, we can build an MPEG-7Description and deploy it. According to the
requirements document,1 “a Description consists
of a Description Scheme (structure) and the
set of Descriptor Values (instantiations) that
describe the Data.” A Descriptor Value is “an
instantiation of a Descriptor for a given data set
(or subset thereof).”
Neither automatic nor
semiautomatic feature
extraction algorithms are
inside the scope of the
standard because their
standardization isn’t required
to allow interoperability.
Parts
The MPEG-7 (ISO/IEC 15938) consists of different Parts. Each part covers a certain aspect of the whole specification.| Part | Number | First public release date (First edition) | Latest public release date (edition) | Latest amendment | Title | Description |
|---|---|---|---|---|---|---|
| Part 1 | ISO/IEC 15938-1 | 2002 | 2002 | 2006 | Systems | the architectural framework of MPEG-7, the carriage of MPEG-7 content - TeM (Textual format for MPEG-7) and the binary format for MPEG-7 descriptions (BiM BiM BiM is an international standard defining a generic binary format for encoding XML documents.The technical specifications for BiM are found in: MPEG systems technologies - Part 1: Binary MPEG format for XML It is also known as MPEG-B Part 1.- Overview :BiM addresses a broad spectrum of... ) |
| Part 2 | ISO/IEC 15938-2 | 2002 | 2002 | Description definition language | ||
| Part 3 | ISO/IEC 15938-3 | 2002 | 2002 | 2010 | Visual | |
| Part 4 | ISO/IEC 15938-4 | 2002 | 2002 | 2006 | Audio | |
| Part 5 | ISO/IEC 15938-5 | 2003 | 2003 | 2008 | Multimedia description schemes | |
| Part 6 | ISO/IEC 15938-6 | 2003 | 2003 | 2010 (2011) | Reference software | |
| Part 7 | ISO/IEC 15938-7 | 2003 | 2003 | 2010 (2011) | Conformance testing | |
| Part 8 | ISO/IEC TR 15938-8 | 2002 | 2002 | 2010 | Extraction and use of MPEG-7 descriptions | |
| Part 9 | ISO/IEC 15938-9 | 2005 | 2005 | (2011) | Profiles and levels | |
| Part 10 | ISO/IEC 15938-10 | 2005 | 2005 | Schema definition | ||
| Part 11 | ISO/IEC TR 15938-11 | 2005 | 2005 | MPEG-7 profile schemas | ||
| Part 12 | ISO/IEC 15938-12 | 2008 | 2008 | (2011) | Query format |
Relation between description and content
On the other hand there must be a relation between the content and description. Thus the description is multiplexed with the content itself.
On the right side you can see this relation between description and content.
MPEG-7 tools
- Descriptor (D): It is a representation of a feature defined syntactically and semantically. It could be that a unique object was described by several descriptors.
- Description Schemes (DS): Specify the structure and semantics of the relations between its components, these components can be descriptors (D) or description schemes (DS).
- Description Definition LanguageDescription Definition LanguageDDL is part of the MPEG-7 standard. It gives an important set of tools for the users to create their own Description Schemes and Descriptors...
(DDL): It is based on XML language used to define the structural relations between descriptors. It allows the creation and modification of description schemes and also the creation of new descriptors (D).
- System tools: These tools deal with binarization, synchronization, transport and storage of descriptors. It also deals with Intellectual PropertyIntellectual propertyIntellectual property is a term referring to a number of distinct types of creations of the mind for which a set of exclusive rights are recognized—and the corresponding fields of law...
protection.
On the right side you can see the relation between MPEG-7 tools.
MPEG-7 applications
There are many applications and application domains which will benefit from the MPEG-7 standard. A few application examples are:- Digital libraryDigital libraryA digital library is a library in which collections are stored in digital formats and accessible by computers. The digital content may be stored locally, or accessed remotely via computer networks...
: Image/video catalogue, musical dictionary. - Multimedia directory services: e.g. yellow pages.
- Broadcast media selection: Radio channel, TV channel.
- Multimedia editing: Personalized electronic news service, media authoring.
- Security services: Traffic control, production chains...
- E-business: Searching process of products.
- Cultural services: Art-galleries, museums...
- Educational applications.
- Biomedical applications.
Software and demonstrators for MPEG-7
- Caliph & Emir: Creation and Retrieval of images based on MPEG-7 (GPL).
- img(Rummager): Open Source implementation of the MPEG-7 descriptors in C#.
- Frameline 47 Video Notation: Frameline 47 from Versatile Delivery Systems. The first commercial MPEG-7 application, Frameline 47 uses an advanced content schema based on MPEG-7 so as to be able to notate entire video files, or segments and groups of segments from within that video file according to the MPEG-7 convention (commercial tool)
- Eptascape ADS200 uses a real-time MPEG 7 encoder on an analog camera video signal to identify interesting events, especially in surveillance applications, check the demos to see MPEG-7 in action (commercial tool)
- IBM VideoAnnEx Annotation Tool: Creating MPEG-7 documents for video streams describing structure and giving keywords from a controlled vocabulary (binary release, restrictive license)
- iFinder Medienanalyse- und Retrievalsystem: Metadata extraction and search engine based on MPEG-7 (commercial tool)
- MPEG-7 Audio Encoder: Creating MPEG-7 documents for audio documents describing low level audio characteristics (binary & source release, Java, GPL)
- XM Feature Extraction Web Service: The functionalities of the eXperimentation Model(XM) are made available via web service interface to enable automatic MPEG-7 low-level visual description characterization of images.
- TU Berlin MPEG-7 Audio Analyzer (Web-Demo): Creating MPEG-7 documents (XML) for audio documents (WAV, MP3). All 17 MPEG-7 low level audio descriptors are implemented (commercial)
- TU Berlin MPEG-7 Spoken Content Demonstrator (Web-Demo): Creating MPEG-7 documents (XML) with SpokenContent description from an input speech signal (WAV, MP3) (commercial)
- MP7JRS C++ Library Complete MPEG-7 implementation of part 3, 4 and 5 (visual, audio and MDS) by Joanneum ResearchJoanneum ResearchJOANNEUM RESEARCH Forschungsgesellschaft mbH is one of the largest non-academic research institutes in Austria. Besides its headquarters in Graz it is also based in Weiz, Hartberg, Leoben, Niklasdorf and Vienna...
Institute for Information and Communication Technologies - Audiovisual Media Group. - BilVideo-7: MPEG-7 compatible, distributed video indexing and retrieval system, supporting complex, multimodal, composite queries; developed by Bilkent University Multimedia Database Group (BILMDG).
- UniSay: Sophisticated Post-production file analysis and audio processing based on MPEG-7.
See also
- ID3ID3ID3 is a metadata container most often used in conjunction with the MP3 audio file format. It allows information such as the title, artist, album, track number, and other information about the file to be stored in the file itself....
- Exif
- Metadata standardsMetadata standardsMetadata standards are requirements which are intended to establish a common understanding of the meaning or semantics of the data, to ensure correct and proper use and interpretation of the data by its owners and users...
- Query by hummingQuery by hummingQuery by humming is a music retrieval system that branches off the original classification systems of title, artist, composer, and genre. It normally applies to songs or other music with a distinct single theme or melody. The system involves taking a user-hummed melody and comparing it to an...
- MPEG-4 Part 11MPEG-4 Part 11MPEG-4 Part 11 Scene description and application engine was published as ISO/IEC 14496-11 in 2005. MPEG-4 Part 11 is also known as BIFS, XMT, MPEG-J...
- Scene description and application engine - Multimedia Information RetrievalMultimedia Information RetrievalMultimedia Information Retrieval is a research discipline of computer science that aims at extracting semantic information from multimedia data sources. Data sources include directly perceivable media such as audio, image and video, indirectly perceivable sources such as text, biosignals as well...
Compare
- Material Exchange Format (MXF), a container format for professional digital video and audio media defined by SMPTE.