Abstract on MULTIMEDIA DATABASES

MULTIMEDIA DATABASES

ABSTRACT:

With the increasing popularity of the WWW, the main challenge in computer science has become content-based retrieval of multimedia objects. Access to multimedia objects in databases has long been limited to the information provided in manually assigned keywords. Now, with the integration of feature-detection algorithms in database systems software, content-based retrieval can be fully integrated with query pro-cessing. This paper describes experimentation platform under development, making database technology available to multimedia.  The approach is based on the new notion of feature databases.  Its architecture fully integrates traditional query processing and content-based retrieval techniques.

 INTRODUCTION:

       

      Large-scale multimedia information retrieval is one of the major scientific challenges of this decade.  This focus of attention seems natural given the significant advances in technology to capture and store raw material in databases and in files on the world-wide web.

        

       First generation multimedia database systems focused on kernel support for blobs (binary large objects); to efficiently store the sizeable objects. Database vendors have provided support for these non-interpreted byte streams in their core products leaving timing, synchronization, and quality-of-service to specialized co-processors.  These developments paved the way for video-on-demand applications, which are slowly making their way into the homes.

       

      Second generation is concerned with techniques for annotation and linking media objects. Most of this activity found itself a breeding ground in user interface re-search and multimedia authoring systems. The database merely contains textual annotations, made accessible efficiently using conventional information retrieval (IR) techniques. Still, multimedia objects remain non-interpreted with respect to retrieval.

       

     Third generation of multimedia database retrieval research focuses
on effective techniques for indexing and retrieval by content .Progress in this area has already demonstrated the feasibility of this approach for retrieving still images, using features based on low-level perceptual image properties such as color distribution and texture ,speech recognition ,audio

 as well as video archives .
 

CHALLENGES:

1) Information access:

    How to effectively satisfy a user's information need using the multimedia objects stored in a database, bridging the gap between the low-level internal representations of multimedia content and the high-level cognitive processes of the user?

2) Data management:

  How to efficiently derive simple and complex multimedia features for widely distributed sources of raw material and making this available as an index for query resolution?

DRIVING FORCE:

The following informal example illustrates the scope of problems that we address in our approach. Imagine a journalist in the near future, working on a TV news item on most peculiar bowling actions in cricket ; assume he is looking for some video fragments and background data to support his story.

                      

It is reasonable to assume that she will have access to distributed, multimedia database at her work, containing news items collected over the years.

A NOVEL DATABASE ARCHITECTURE:

            This Section argues that extending database systems to support multimedia search requires more than just the abstraction from data structures, for, the problem underlying multimedia search is that the DBMS has to reason about the content of multimedia objects as well. We therefore propose that a multimedia DBMS should explicitly support the modeling of content, a process we identify as “Content Abstraction”.

           “Content Abstraction” is the process of describing the content of multimedia objects through metadata, either assigned manually, or extracted automatically. It requires the integration of database query processing with information retrieval. Both the query and the contents of the database (the subtitles) undergo some processing, such as replacing query words by their stems. The extra functionality required can be supported easily in modern extensible database management systems.

           The gap between the query formulated by the user and the content abstractions that should be used to satisfy the information need is simply too large. The various content abstractions available provide different views on the data, but it is not clear a priori how to combine these views to find relevant objects. In theory, the best candidate to specify this combination is indeed the end user.

            A blueprint of the architecture pursued is illustrated in the following Figure. The architecture separates the design of a multimedia database into three components: a data abstraction component, a content abstraction component, and a retrieval engine. Next to this blueprint, we outline its current implementation in the Acoi platform (The Acoi database is an experimentation platform for multimedia storage, indexing and retrieval.)

Content abstraction and data abstraction are managed by the notion of a feature database. The interaction with the user is managed by the query processor. 

        

Multimedia DBMS                 Content abstraction component Retrieval Engine       Data  Abstraction                 Component                                                         Query interface            Query processor           Feature database   Extensible database system

          

LAYERS OF ACOI PLATFORM:

Query interface:

The query interface for multimedia databases differs considerably from the traditional approach of OQL and SQL. Query formulation involves a mix of textual descriptions, component clipping, and expression of temporal, spatial, as well as topological relationships between objects. An example for the mechanism employed in query interface is spot-based retrieval. 

                           

A spot is a distinctive small region of a sample image used to steer the search process. The spot is used to localize similar images in the collection, using a few controllable parameters, such as their position in the target image, their scale, and their possible color distortions.  For example, searching tigers in a database is translated into identifying a spot on the skin of a sample tiger image. This defines a small color set and their spatial relations. The user may relax the spot features by controlling the color range, brightness, textures, and their spatial locality in a target image, e.g. tigers do not  fly therefore a spot is not likely of interest if it appears at the top. Subsequently, we inspect all color-space indices for similar spots. A subset of qualifying index clips is returned as an approximate answer. The user can then use this information to refine his query and obtain the desired result.

The retrieval engine:

The retrieval engine controls the dialogue with the user. Its implemen-
tation is strongly rooted in the theory and techniques developed in information retrieval. Although IR theory is usually only applied to text documents, it can be adapted for the case of multimedia.

     Users first express their information needs in the form of an initial query, that consists of a combination of an explicit statement of properties of the objects desired, like keywords that should occur, with an implicit statement of such properties, using a (possibly empty) set of example objects with relevance judgments. The retrieval engine then determines which objects are likely to be relevant for the user's information need. Since the query cannot possibly capture all aspects of the information need, this process will be iterated. This layer of the architecture supports the primitives to specify example objects, and relevance judgments about previously retrieved objects.  Also, it supports directives to declare the start and end of a query session related to some particular information need. Using the information elicited from the interaction with the user, the retrieval engine estimates which content abstractions are most likely to explain the relevance judgements, and formulates a query accordingly.

Feature databases:

In principle, the retrieval engine may be implemented on top of any extensible DBMS, e.g. as an extra component in between the Chabot system and the user application.  But, the process of content abstraction in multimedia databases imposes some constraints that are hard to handle effectively in existing database technology: most notably,

(1) The ever-increasing set of data types and operations on these types,

(2) A large number of is-a and has-a relationships between objects, content abstractions, and their components, and

(3) The necessity to index the multimedia objects incrementally.

Given the size, distribution and volatility of the data collections to be indexed, construction of a multimedia feature index is an ongoing activity. This complex environment motivates the introduction of an extra layer(Feature Database) in between the retrieval engine and an extensible DBMS.

  Database support:

 The underlying database system deployed here is Monet, a novel and powerful extensible DBMS. It supports a binary relational data model, using vertical decomposition to represent complex data; which is a particularly useful basis for the storage of the inhomogeneous data captured in the feature grammars. Monet also supports modular extension, a technique in line with data cartridges and data blades, which encapsulate the routines and data structures for a particular data type.  The system provides already modules that support image applications and GIS, and modules for the management and analysis of video data are currently under development.

Information access:

Our approach to the implementation of the retrieval engine is based on the architecture of information retrieval systems. The three fundamental issues in probabilistic information retrieval are:  representation of documents, query formulation, and a ranking function. These three issues are reflected in the three layers of the retrieval engine in the figure shown below:  the concept layer manages the basic `concepts' derived from multimedia objects, the evidential reasoning layer implements the ranking function, and the relevance feedback layer steers query formulation through a dialogue between the user and the system.

Multimedia  Databases  Concept   Abstraction   Component Relevance       feedback Evidential residential layer Concept layer         Data Abstraction  Component

Querying the digital image library now takes place as follows.  First, the user enters an initial (usually textual) query.  Next, we use the thesaurus to select clusters from the image content representations that are relevant to this initial query. The multimedia objects are ranked in the evidential reasoning layer using a standard probabilistic retrieval model, and the best results of this query are presented to the user.  The user may provide relevance feedback for these images; this relevance feedback is then used to improve the current query. A problem for the current retrieval system is that the thesaurus sometimes associates words in the annotations to irrelevant clusters, or the clustering process chooses clusters having little semantic value. To alleviate these problems, machine learning techniques are used to adapt the thesaurus and the clustering, by combining the relevance feedback across several query sessions.

Conclusions and future work:

One of the main challenges is to bridge the gap between the concepts in
the real-world environment of the end users and the low-level features that can be computed from the raw data of multimedia objects. The use of IR theory was presented for this purpose, and a novel architecture for multimedia database management systems that integrates these techniques in all levels of the system proposed.

    Future work in the information access research line consists of the application of machine learning techniques to improve the representations of multimedia objects in the concept layer using feedback across sessions. Also, significant improvements are expected in the quality of multimedia information retrieval, by obtaining finer grained information from the dialogue with the user.

11:14 AM