A SYNCHRONIZATION MODEL FOR PRESENTATION OF MULTIMEDIA OBJECTS - PDF document

Journal of the Chinese Institute of Engineers, Vol. 24, No. 2, pp. 173-186 (2001) 173 A SYNCHRONIZATION MODEL FOR PRESENTATION OF MULTIMEDIA OBJECTS In-Ho Lin, Bih-Hwang Lee* and Chwan-Chia Wu Department of Electrical Engineering National Taiwan University of Science and Technology Taipei, Taiwan 106, R.O.C. Key Words: multimedia synchronization, temporal relations, media group, resource scheduling. ABSTRACT To support the presentation requirements of distributed multime- dia information, synchronization of multimedia objects must be achieved. To this end, system resource scheduling and resource reservation for object pre-fetch, network bandwidth and buffer occupancy must be determined prior to the time the presentation is initiated. This paper proposes an object-oriented model to handle the temporal relationship for all of the multimedia objects at the pre- sentation platform and study the related problems of resource allocation. Synchronization of the composite media objects is achieved by ensur- ing that all objects presented in the upcoming “manageable” period must be ready for execution. To this end, the nature of overlap is first examined for various types of objects. The importance of critical over- lap and critical point that are vital to synchronization is addressed and taken into account in this research. The concept of manageable presentation interval and the irreducible media group are also introduced and defined. Analysis of resource allocation among pre-fetch time of media object, network bandwidth and buffer occu- pancy is also examined. Accordingly, a new model called group cascade object composition Petri-net (GCOCPN) is proposed and an algorithm to implement this temporal synchronization scheme is presented. I. INTRODUCTION graphics and so forth, over heterogeneous and distributed environments, to furnish a particular ap- With the advance of computer, network and plication service is essential and inevitable. Presen- multimedia technologies in the past decade, the de- tation of distributed multimedia information involves mands for multimedia information services from com- temporal organization, spatial organization, pre- plex environments are rapidly growing in different fetch, transformation and delivery of components, fields including education, entertainment, CAE, Web which compose the multimedia information for browsing, and internet telephony. It is an emerging the user and allow the user to interact with the pre- trend that the integration of various multimedia sentation sequence as well. As to the issues of tem- objects, which can include text, images, audio, video, poral synchronization and resource allocation, *Correspondence addressee

174 Journal of the Chinese Institute of Engineers, Vol. 24, No. 2 (2001) the media characteristics, temporal dependence and presented to cope with the problem of latency re- resource utilization must be clearly established to sulting from resource allocation, data generation, ensure proper scheduling of the synchronized packet assembly, network communication, etc. presentation. (Qazi et al ., 1993; Raghavan et al ., 1996). Media Multimedia data can be classified into two derived from different sources may introduce categories: discrete media and continuous media different delays to the data transmission path and (Allen, 1983; Blakowski and Stenmetz, 1996; Little thus result in lip-sync or delays jitters problems and Ghafoor, 1990; Nicolaou, 1990). Real-time data, during presentation. However, up to this moment, such as video, audio and animation that require time- none of the prior works presents a simple yet com- ordered presentation to the user are basically classi- prehensive model to resolve the aforementioned fied as continuous media. On the other hand, text, problems. images and graphics media are basically time-inde- In addition, resource allocation and scheduling pendent and are normally presented on a page or frame are essential to ensure achievement of the intra- and basis. They are therefore classified as discrete or inter- objects synchronization requirements for mul- static data. The specification of temporal composi- timedia presentation. To facilitate a smooth playback tion is required to describe these objects completely of media objects with satisfactory QoS, it is required by taking all of the temporal relationships into that objects should be presented to the memory be- account. fore they are delivered for playback. Synchronization in a multimedia presentation The purpose of this paper is to present an ob- system refers to the temporal relations between ject-oriented model that specifies and manipulates the media objects; it makes multimedia presentation temporal relationships of all multimedia objects at the take place in the desired time-ordered sequence ex- presentation platform. Synchronization of the com- actly at the predetermined starting and ending time posed media objects is achieved by ensuring that all instants. There are two basic types of synchroniza- of the media objects to be presented in the upcoming tion, intra-media and inter-media synchronization: “manageable” period must be available for execution. intra-media synchronization refers to the time rela- The basic concept is therefore to analyze the tempo- tions between various presentation units of one ral relationship and to partition the whole of the com- continuous media object; and a sequence of media posite media objects into several “manageable” units such as video frames should be played back groups that can be presented in sequence. To this continuously and smoothly for ensuring the serial end, we first investigate the nature of overlays for media synchroniza-tion. Inter-media synchroniza- various types of objects. Based on our observation, tion refers to the synchronization among different we define the manageable presentation interval and media objects, which may be retrieved and trans- introduce the concept of presentation groups. The ferred from different places and be presented in resource scheduling of each presentation group for parallel. In order to offer a better performance buffer occupancy versus pre-fetch time of media ob- and quality of services for multimedia applications, ject is also examined. We then develop a pre-fetch synchronization constraints for both the inter- scheme for describing the requirement of buffer oc- and intra- media objects must be specified and main- cupancy versus pre-fetch time. Accordingly, we tained. present a new model called group cascade object com- For supporting distributed multimedia position Petri-net (GCOCPN) to cope with the syn- applications, many researchers (Little and Ghafoor, chronization problems and an experimental result to 1990; Little and Ghafoor, 1993; Iino et al ., 1994; Qazi compare the required buffer occupancy from the origi- et al ., 1993; Raghavan et al ., 1996; Yang and Huang, nal multimedia system to the proposed GCOCPN 1996) adopt the concept of Petri-net (Murata, 1989) model. to construct a reference model for archiving multi- This paper is organized as follows: Section 2 media synchronization. The object composition outlines the related works in this field and the back- Petri-net (OCPN) model (Little and Ghafoor, 1990) ground of multimedia synchronization models; specifies temporal requirements at the presentation Section 3 introduces the concepts of multimedia in- level. Qazi et al . propose the XOCPN model (Qazi formation group and irreducible media group which et al ., 1993) that makes some improvement of are to be used in this paper; Section 4 provides the OCPN by introducing a synchronization interval unit analysis of system parameters which include buffer (SIU) to resolve the network communication delay occupancy, network bandwidth and pre-fetch time; problem. There also has been intensive research section 5 provides an experimental result by compar- on the subject of temporal synchronization (Qazi et ing the resource utilization from the original systems al ., 1993; Raghavan et al ., 1996; Yang and Huang, to the proposed GCOCPN model. Finally, conclu- 1996). Several models have been designed and sions are stated in section 6.