|MPEG Forum report
November 24th, 1998
This first edition of the ACTS/ESPRIT MPEG 7 Forum, was organised at the request of MM Badique and Aigrain (CEU-DG XIII). The two key objectives of this session were firstly to inform everybody of the status of the different standardisation initiatives in the field of metadata and hypermedia content structuring and representation, and secondly to get an overview of different European projects which contribute to research and development activities in this field, and which will submit contributions to the current MPEG 7 Call for Proposals.
MPEG7 aims at proposing a standard to describe in a normalized form content description of multimedia documents. It will standardise descriptors, Description Schemes or structures which shall be associated with the content itself, and a Description Definition Language (DDL). Multimedia material may include: still pictures, graphics, 3D models, audio, speech, video, and information about how these elements are combined in a multimedia presentation.
MPEG 7 will provide the building blocks for multimedia authoring, enable exchange of document metadata and facilitate information retrieval based on the syntax and descriptors provided. It is both important for streamed material (e.g. : broadcast applications) and stored material (e.g. : multimedia technical documentation, audiovisual programmes archives, etc.).
The MPEG 7 Forum gathered about 40 representatives from many organisations involved in R&D activities in this field. This strong participation indicates the growing importance of multimedia content description in the hypermedia networked environment we foresee.
Questions and Issues raised
Due to the very dense agenda, little time was available for discussion. The main issues raised were :
Eric Badique raised two important and related questions : there is a convergence of interests between the different initiatives for standardisation which were presented (i.e : mainly DAVIC, MPEG 7 and EBU/SMPTE, but other organisations like CEN are working on Metadata). It is not clear yet firstly how these initiatives relate to each other, and secondly how the results or standards they produce will diverge or converge, encapsulate, etc.
Metadata and content representation should probably become one of the recurrent themes for concertation activities in the next Framework Programme. Note that exchanges of information can also be done using the ACTS-MPEG reflector.
Eric Badique also proposed that one of the next issues of the ACTS Newsletter could focus on an overview of MPEG 7 related activities and projects.
Philippe Aigrain raised an issue on the relationship between generic standards and media-specific standards. When do we need to separately deal with still images, sound, audio, etc. Should MPEG7 focus mainly on video ?
Philippe Aigrain also mentioned that HYPERMEDIA should consult with related projects DIVAN and VICAR.
FernandoPereira considers that MPEG 7 should not exclude anything to focus on video. He also noted that the road is long to go from a good list of descriptors (as will be submitted in the MPEG 7 CFP) to a completely specified set of encoded descriptor bitstreams.
Fernando Pereira also suggested that ACTS possibly together with ESPRIT could build a concerted European proposal for the MPEG 7 eXperimentation Model.
Guillermo Cisneros proposed issues to be discussed in forthcoming MPEG 7 Forum meetings : how to use the MPEG7 test set provided by HYPERMEDIA, how to contribute to the MPEG7 eXperimentation Model, etc.
Christoph Dosch : Motion and detailed scene analysis is a requirement for broadcast applications.
Ebrahimi Touradj : MPEG 7 work is largely focused on television applications. How should MPEG7 deal with non-television applications ? (e.g. : Universal mailbox : media conversion between text, speech and images). Image analysis and pattern analysis would benefit a lot from the availability of high-level labels.
ANNEX I Notes related to the presentations
Please note that the following notes are highlights of the presentations. The slides enclosed after give more detail of each of the presentations.
1 Standardisation efforts in hypermedia structuring and metadata
Rémi Ronfard (Institut National de l’Audiovisuel) presented the key issues for search and navigation in audiovisual documents.
The key objectives of MPEG 7 are to standardise :
A set of « description schemes» including
pixel-based video features & descriptors
low level audio features & descriptors
domain-specific features & descriptors
A syntax for defining description schemes
Coding tools for efficient storage and transmission of descriptions
From the point of view of television archivers, there is a need for standards in :
feature extraction and storage
repurposing : stock-shot libraries
repurposing : video sequence libraries
Mr Ronfard evoked the MPEG 7 calendar, particularly the call for proposals to close on February 1st, 1999, the lancaster evaluation week later in the month which should produce a set of recommendations, and the Seoul Meeting in March 1999 which should give orientations based on the Lancaster results.
0.2EBU-SMPTE Task Force for the Harmonization of Standards for the Exchange of Program Material as Bit Streams
Christoph Dosch (IRT) presented the results of the EBU/SMPTE task-force, compiled in a Final Report. American companies and European broadcasters met for two years.
During the first phase, the TF produced a report on user requirements. During the second phase, the TF produced the Final Report, included in the EBU Technical Review. It contains a set of recommendations, areas for standardisation.
ITU standards can be issued from those recommendations. ITU draft for SDTI was adopted. Mapping of metadata into SDTIwas proposed. New requirements emerged like « faster than real time » transfers of video.
Among responses to the TF CFPs on File Wrappers were Apple’s QuickTime and Avid’s OMFI. None of the responses satisfied completely the requirements, but the sum of the proposals covered all the requirements.
Christoph Dosch proposed to define Metadata as everything that is not audio or video – teletext, subtitles, program descriptions, etc. SMPTE has created the " SMPTE Engineering Committee on Wrappers and Metadata ".
The Metadata dictionary of the SMPTE will be a dynamical document, and has been submitted in MPEG7 as a Description Scheme. SMPTE has created a Registration Authority for managing the evolution of the Dictionary, and is defining two procedures for upgrades : light modifications backward compatible, and deep changes requiring to change the decoders.
KLV (Key, Length, Value) protocol is submitted in MPEG7 as Coding Scheme, and can be used to encode both essence and metadata He introduced the nine metadata classes, according to SMPTE.
0.3DAVIC Metadata group
Presented by Christian Bertin (CCETT). The DAVIC reference architecture and specification documents were an important source of inspiration for the EBU/SMPTE Task Force. More recently, a Metadata Ad Hoc Group has been set up in DAVIC. The approach of this AHG starts from two application concepts defined as: TV Anytime, TV Anywhere.
They specify descriptions that could be broadcasted :
Inheritance tree and composition tree
Attractors (why get the content)
Locators (how to get the content)
Access and protection descriptors
One example : Content Item (CI) is one episode of a Content Package (the series)
Difficult requirements were identified : large vocabularies, compatibility between different metadata systems.
DAVIC is open to non-members for one meeting.
1Presentations of MPEG 7 related work by ACTS and ESPRIT Projects
Prime : IRT. Partners : Bosch, HHI, KPN, NRK, QMW,Telenor
The initial objective is to insert MPEG4 services in MPEG2/DVB transport streams and the ultimate Goal to contribute to MPEG and DAVIC for inclusion of new MPEG4 and MPEG7 indexes into MPEG2/DVB streams.
Reference architecture : Custom TV has an extended server, containing an MPEG4 object database, and MPEG7 index database and a traditional MPEG2 archive. They multiplex everything into a composite, enriched multi-service broadcast.
Planned MPEG7 contributions :
MPEG7 requirements on broadcast applications
Description schemes for EPGs and broadcast information (based on DVB-SI)
Streaming of MPEG7 data and cyclic repetition in the MPEG7 XM
A debate on the deficiencies of DVB-SI was opened. In response, it was said that DVB has a java API for defining all kinds of services around DVB-SI (approved by DAVIC).
1.2STORIT (Edwin Montie, Philips)
Prime : Philips. Partners : Olivetti, BBC,Delft, Tandberg Data, Univ. Of Ljubljana, Space Application Services (continuation of the SMASH Project)
Their objective is to bring internet, broadcasting and publications together, to provide more efficient services. Examples – remote learning, and education. STORIT takes part in the DAVIC TV ANYTIME specifications.
The STORIT box is a small home-archive on a Set-Top Box. The hard disk is the « freezer » where media food can be stored. The project studies how to help in the selection of material to be stored, based on user interests, programs, etc. It should become possible for users to issue requests to broadcasters on their web site.
STORIT needs a universal program identifier and an XML-like language for TV programs, with an identification for channel-day-time information.
BBC intends to make an MPEG7 submission. Philips will submit through the AVIR project. It is not decided on the opportunity to make a joint STORIT contribution.
1.3Diceman (Fernando Pereira)
Prime : Teltec. Partners : INA, IBM, KPN, CSELT, UPC, Starlab, IST
Very strong participation in MPEG. Goals : Build a database and some agents that can handle user queries with the database over the internet. DICEMAN proposals to MPEG 7 :
A DDL based on XML with additional « strong data types », and a « standard DS ».
Visual DS - Provide capabilities for queries based on object properties.
Audio DS for Script and scene analysis, representing regions, events, etc.
The DICEMAN demonstrator will attach detailed theatre scripts to many different theatre films.
1.4Modest (Patrick Piscaglia, UCL)
Prime : UCL. Small project with four partners.
Video surveillance – example of traffic analysis, with exchange between observation agents about surveillance scenes. They will submit to MPEG 7 :
Description schemes for surveillance, timestamps, camera identification, etc
Serialization of complex data types
Video Surveillance Object : color and 3D position, speed, size, orientation, trajectory, contour.
Question : how can the project participate to the MPEG7 XM ?
1.5Hypermedia (Guillermo Cisneros, University of Madrid)
Contributions to MPEG 7 :Video sequences, accompanied by metadata obtained from spanish television
They have an « pyramidal » model :
Basic tools at the lowest level
Next level : services
Next level : more elaborated services
Highest levels : business cases and applications
For each application , they set up a glossary of metadata types/categories.
1.6AVIR (Marco Ciccarreli, Philips)
Prime : Philips. partners : TECMATH, RAI, TV spielfilm, LIP6, Univ Brescia.
Objective : Indexing and retrieval for non-expert users.
On the production side : assisted indexing, including speech recognition. Multiplexing of video and metadata.
On the reception side – implementation of an EPG based on this data (Tvspielfilm, RAI). Filtering based on user profiles.
MPEG7 Application : personalized TV services. They have numereous requirements on MPEG7, including compatibility with other standards such as DVB-SI or PSIP.
AVIR will propose to MPEG7 :
DS for visual representation by Brescia
DS for TV programmes
DDL by LIP6
AVIR will not implement an XM, but are willing to provide elements.
1.7Cuidad (Vincent Puig, IRCAM)
CIUDAD is a working group, not a research project, working inthe area of content processing of music.
Fields of interest :
The parameters of synthesis models, FM Models, etc . are good descriptors of musical sound.
Content-based retrieval of melodies, musical sounds, etc .
Spatialization, 3D audio
CIUDAD separate perceptual (semantic) descriptors, musical (symbolic) descriptors and signal descriptors.
They will contribute SDIF as a DS for sound description (simple, open description scheme). It is a streamable interchange format. Specs are available (www.ircam.fr/sdif). A simple sound description in SDIF is a matrix of elements and dimensions.
They will submit audio descriptors.
Olivier Lescurieux & Rémi Ronfard
Institut National de l’Audiovisuel 25/06/16