Mpeg forum report acm meeting

November 24^th, 1998

This first edition of the ACTS/ESPRIT MPEG 7 Forum, was organised at the request of MM Badique and Aigrain (CEU-DG XIII). The two key objectives of this session were firstly to inform everybody of the status of the different standardisation initiatives in the field of metadata and hypermedia content structuring and representation, and secondly to get an overview of different European projects which contribute to research and development activities in this field, and which will submit contributions to the current MPEG 7 Call for Proposals.

MPEG7 aims at proposing a standard to describe in a normalized form content description of multimedia documents. It will standardise descriptors, Description Schemes or structures which shall be associated with the content itself, and a Description Definition Language (DDL). Multimedia material may include: still pictures, graphics, 3D models, audio, speech, video, and information about how these elements are combined in a multimedia presentation.

MPEG 7 will provide the building blocks for multimedia authoring, enable exchange of document metadata and facilitate information retrieval based on the syntax and descriptors provided. It is both important for streamed material (e.g. : broadcast applications) and stored material (e.g. : multimedia technical documentation, audiovisual programmes archives, etc.).

The MPEG 7 Forum gathered about 40 representatives from many organisations involved in R&D activities in this field. This strong participation indicates the growing importance of multimedia content description in the hypermedia networked environment we foresee.

Questions and Issues raised

Eric Badique raised two important and related questions : there is a convergence of interests between the different initiatives for standardisation which were presented (i.e : mainly DAVIC, MPEG 7 and EBU/SMPTE, but other organisations like CEN are working on Metadata). It is not clear yet firstly how these initiatives relate to each other, and secondly how the results or standards they produce will diverge or converge, encapsulate, etc.

Metadata and content representation should probably become one of the recurrent themes for concertation activities in the next Framework Programme. Note that exchanges of information can also be done using the ACTS-MPEG reflector.

Eric Badique also proposed that one of the next issues of the ACTS Newsletter could focus on an overview of MPEG 7 related activities and projects.

Philippe Aigrain raised an issue on the relationship between generic standards and media-specific standards. When do we need to separately deal with still images, sound, audio, etc. Should MPEG7 focus mainly on video ?

Philippe Aigrain also mentioned that HYPERMEDIA should consult with related projects DIVAN and VICAR.

FernandoPereira considers that MPEG 7 should not exclude anything to focus on video. He also noted that the road is long to go from a good list of descriptors (as will be submitted in the MPEG 7 CFP) to a completely specified set of encoded descriptor bitstreams.

Fernando Pereira also suggested that ACTS possibly together with ESPRIT could build a concerted European proposal for the MPEG 7 eXperimentation Model.

Guillermo Cisneros proposed issues to be discussed in forthcoming MPEG 7 Forum meetings : how to use the MPEG7 test set provided by HYPERMEDIA, how to contribute to the MPEG7 eXperimentation Model, etc.

Christoph Dosch : Motion and detailed scene analysis is a requirement for broadcast applications.

Ebrahimi Touradj : MPEG 7 work is largely focused on television applications. How should MPEG7 deal with non-television applications ? (e.g. : Universal mailbox : media conversion between text, speech and images). Image analysis and pattern analysis would benefit a lot from the availability of high-level labels.

ANNEX I Notes related to the presentations

1 Standardisation efforts in hypermedia structuring and metadata

0.1MPEG 7

The key objectives of MPEG 7 are to standardise :

• 
  A set of « description schemes» including
  
• 
  pixel-based video features & descriptors
  
• 
  low level audio features & descriptors
  
• 
  domain-specific features & descriptors
  
• 
  A syntax for defining description schemes
  
• 
  Coding tools for efficient storage and transmission of descriptions
  

• 
  interoperability
  
• 
  feature extraction and storage
  
• 
  production information
  
• 
  repurposing : stock-shot libraries
  
• 
  repurposing : video sequence libraries
  

0.2EBU-SMPTE Task Force for the Harmonization of Standards for the Exchange of Program Material as Bit Streams

During the first phase, the TF produced a report on user requirements. During the second phase, the TF produced the Final Report, included in the EBU Technical Review. It contains a set of recommendations, areas for standardisation.

ITU standards can be issued from those recommendations. ITU draft for SDTI was adopted. Mapping of metadata into SDTIwas proposed. New requirements emerged like « faster than real time » transfers of video.

Among responses to the TF CFPs on File Wrappers were Apple’s QuickTime and Avid’s OMFI. None of the responses satisfied completely the requirements, but the sum of the proposals covered all the requirements.

Christoph Dosch proposed to define Metadata as everything that is not audio or video – teletext, subtitles, program descriptions, etc. SMPTE has created the " SMPTE Engineering Committee on Wrappers and Metadata ".

The Metadata dictionary of the SMPTE will be a dynamical document, and has been submitted in MPEG7 as a Description Scheme. SMPTE has created a Registration Authority for managing the evolution of the Dictionary, and is defining two procedures for upgrades : light modifications backward compatible, and deep changes requiring to change the decoders.

KLV (Key, Length, Value) protocol is submitted in MPEG7 as Coding Scheme, and can be used to encode both essence and metadata He introduced the nine metadata classes, according to SMPTE.

0.3DAVIC Metadata group

They specify descriptions that could be broadcasted :

• 
  Unique identifiers
  
• 
  Inheritance tree and composition tree
  
• 
  Attractors (why get the content)
  
• 
  Locators (how to get the content)
  
• 
  Access and protection descriptors
  
• 
  Segmentation descriptors
  
• 
  Objet descriptions
  
• 
  Locally-generarated descriptions
  

Difficult requirements were identified : large vocabularies, compatibility between different metadata systems.

DAVIC is open to non-members for one meeting.

1Presentations of MPEG 7 related work by ACTS and ESPRIT Projects

1.1CustomTV

The initial objective is to insert MPEG4 services in MPEG2/DVB transport streams and the ultimate Goal to contribute to MPEG and DAVIC for inclusion of new MPEG4 and MPEG7 indexes into MPEG2/DVB streams.

Reference architecture : Custom TV has an extended server, containing an MPEG4 object database, and MPEG7 index database and a traditional MPEG2 archive. They multiplex everything into a composite, enriched multi-service broadcast.

Planned MPEG7 contributions :

• 
  MPEG7 requirements on broadcast applications
  
• 
  Description schemes for EPGs and broadcast information (based on DVB-SI)
  
• 
  Streaming of MPEG7 data and cyclic repetition in the MPEG7 XM
  

1.2STORIT (Edwin Montie, Philips)

Their objective is to bring internet, broadcasting and publications together, to provide more efficient services. Examples – remote learning, and education. STORIT takes part in the DAVIC TV ANYTIME specifications.

The STORIT box is a small home-archive on a Set-Top Box. The hard disk is the « freezer » where media food can be stored. The project studies how to help in the selection of material to be stored, based on user interests, programs, etc. It should become possible for users to issue requests to broadcasters on their web site.

STORIT needs a universal program identifier and an XML-like language for TV programs, with an identification for channel-day-time information.

BBC intends to make an MPEG7 submission. Philips will submit through the AVIR project. It is not decided on the opportunity to make a joint STORIT contribution.

1.3Diceman (Fernando Pereira)

Very strong participation in MPEG. Goals : Build a database and some agents that can handle user queries with the database over the internet. DICEMAN proposals to MPEG 7 :

• 
  A DDL based on XML with additional « strong data types », and a « standard DS ».
  
• 
  Visual DS - Provide capabilities for queries based on object properties.
  
• 
  Audio DS for Script and scene analysis, representing regions, events, etc.
  
• 
  Isolated descriptors
  

1.4Modest (Patrick Piscaglia, UCL)

Video surveillance – example of traffic analysis, with exchange between observation agents about surveillance scenes. They will submit to MPEG 7 :

1. 
   Description schemes for surveillance, timestamps, camera identification, etc
   
2. 
   Serialization of complex data types
   

Video Surveillance Object : color and 3D position, speed, size, orientation, trajectory, contour.

1.5Hypermedia (Guillermo Cisneros, University of Madrid)

They have an « pyramidal » model :

• 
  Basic tools at the lowest level
  
• 
  Next level : services
  
• 
  Next level : more elaborated services
  
• 
  Highest levels : business cases and applications
  

1.6AVIR (Marco Ciccarreli, Philips)

Objective : Indexing and retrieval for non-expert users.

• 
  On the production side : assisted indexing, including speech recognition. Multiplexing of video and metadata.
  
• 
  On the reception side – implementation of an EPG based on this data (Tvspielfilm, RAI). Filtering based on user profiles.
  

AVIR will propose to MPEG7 :

1. 
   DS for visual representation by Brescia
   
2. 
   DS for TV programmes
   
3. 
   DDL by LIP6
   

1.7Cuidad (Vincent Puig, IRCAM)

Fields of interest :

1. 
   The parameters of synthesis models, FM Models, etc . are good descriptors of musical sound.
   
2. 
   Content-based retrieval of melodies, musical sounds, etc .
   
3. 
   Spatialization, 3D audio
   

They will contribute SDIF as a DS for sound description (simple, open description scheme). It is a streamable interchange format. Specs are available (www.ircam.fr/sdif). A simple sound description in SDIF is a matrix of elements and dimensions.

They will submit audio descriptors.

Olivier Lescurieux & Rémi Ronfard