|
In this paper we describe a versatile new software system for the display of multimedia information. The software can be used for multimedia presentations, public information systems such as for tourism, shopping centres, banks, retail, hotels and airlines, and business information systems interacting with multimedia databases. The program can display computer images, animated graphics and text as well as audio and video information as desired. The user interacts with the program via either a mouse or a touch screen to access and display the information desired. Information is presented as a set of "pages" connected as a tree structure, so the user traverses from one page to the next, from more general to more specific information in a manner of his own choosing. The paper will describe the software and consider application environments, while focusing on the flexibility of the system and the power of the presentations which can be produced. Several examples will be included.
When affordable multimedia hardware became available, the authors realised that an early entry to this field was important, and that ownership of a portion of the technology was essential. To this end, a piece of computer software known as infoterm has been developed to display multimedia presentations. There were two main criteria driving its development; it had to be simple and attractive to use and it had to be simple to develop presentations for it.
Infoterm is the subject of this paper. Historical information which led to the development of the system, a description of the system itself and of its usage, its applications and future development will be discussed.
Still before the term multimedia was well known, Telecom Australia made another significant entry into the interactive information system environment, in partnership with the World Expo 88 Authority, by providing a system known as EXPO INFO (Clarke, 1988) to World Expo 88, held in Brisbane, Australia from April 30 to October 30, 1988. The theme of that Exposition was "Leisure in the Age of Technology". In keeping with that theme, EXPO INFO provided visitors with a computer screen which displayed text, graphics and high quality video. Visitors retrieved information simply by touching the screen of the terminal.
Through some 60 touch screen terminals in various locations around the Expo site, it provided visitors with information in both English and Japanese about pavilions, entertainment, food, drink, public facilities, tourism and transport, using computer generated graphics and animation, plus video action and sound from the laser disc player in each terminal. Information about the current day's events was transmitted via telephone lines to modems as required.
EXPO INFO proved very popular and was used by nearly one third of all visitors each day, with 91 percent of these reporting it as "easy" or "very easy" to use. It achieved around 46 million 'touches' during the six month period (Sheehan, 1989). Further, many companies expressed an interest in the equipment, so they could develop presentations for their own enterprises.
Unfortunately, much of the hardware that had been chosen for EXPO INFO was withdrawn from the marketplace. There was strong motivation to provide similar equipment, updated for new applications. Not only was the hardware purpose designed for EXPO 88, so too was the software, and of course the video. The need to redevelop the hardware gave us an opportunity to develop software which made it easy to produce new presentations.
Replacing the EXPO INFO product was only part of the motivation for developing infoterm. In addition, although there were no competing products in the marketplace at the time we were specifying the system, we knew that it would only be a matter of months, at the outside, before other, similar products would be available commercially. As a telecommunications company, our primary interest is in telecommunications network products and services; a major goal for infoterm has been to provide a multimedia front end for applications which utilise network resources. For this reason, ownership of the software, giving us the capability for developing it further as required, was essential. We also recognised that infoterm was of and in itself a useful means of providing stand alone demonstrations.
The user interface for infoterm comprises a single computer screen, overlaid by a touch screen. The computer screen displays full colour computer generated text animations, and images, and also displays video. The user is presented with screens full or "pages" of information, and describing choices for further pages, one of which they user may choose to select by touching the touch screen at a defined point. The full information database is collated as a tree structure; the user can move along branches, or jump from one branch to another as he or she chooses.
There is also provision for the gathering of statistics to provide feedback on how heavily an infoterm system is being used, and facilities for conditional execution of the information. For example, a map display for the "current location" of course depends on where specifically a system is located. A number of systems over say, a shopping centre, need to be given separate and distinct instruction as to which is the "current location" display page.
The software is quite generic and capable of displaying presentations of any size up to the memory storage limitations of the host computer. One can imagine complex information which could potentially run to thousands of pages. Finally, data can be updated and modified and incorporated into the presentation database while the system is operational.
Two computer platforms where chosen, one running Microsoft Windows 3 [1], and the other running the X Window System [2]. The motivation for this was to have a robust system that could run stand alone in the first instance, and to take advantage of the powerful networking capabilities of Unix [3] in the second instance.
No touch screen has been chosen for the X Windows on Unix version, for the Microsoft Version of infoterm there are a number of different touch screens, any of which are simply plugged in and can replace the mouse interface trivially. As a result the software has in actuality been developed using a mouse interface.
A standard off the shelf laserdisc video system was chosen as the video source; at the time of system specification this was the only video option that could be completely controlled by computer, such that an individual frame number can be given and the laserdisc will directly display that particular frame. Other video options, such as a VCR of a camera can also be used, but the same level of control over which particular frame is to be displayed is, of course, not possible.
To display video on a computer, a video board is required which can incorporate images from the video source with output from the computer's video buffer. At the time of system specification these boards were not widely available. A cost effective solution which was selected was the DVA-4000 video board (VideoLogic Inc, 1989) from VideoLogic Inc. This board is compatible with Microsoft Windows and VGA monitor. It stores the video input signal in a digital frame store and merges it with the output from the VGA board. The video can be located at any position on the display and be of any size. It can overlay computer generated graphics and simultaneously be overlaid by computer generated graphics. At this stage no video board has been selected for the X Windows alternative, although some are now available.
The complete absence of standards for the video display hardware is potentially a cause for concern. The system is currently locked to a single manufacturer. Should an alternative video board become attractive, at least with ownership of the software, and due to the fact that the video control section of the software has been written as a single module, a rewrite of this piece of software is not an insurmountable problem.
The infoterm software is configured as an all purpose software "engine" which reads a input a text database containing the complete "script" for a specific presentation. Change this text file and a completely different set of information is presented. The language of this text database was specified, at least provisionally, as the first step in the design process.
The language of infoterm basically consists of it is a set of "objects" which are read in the sequence they appear in the database. An object consists of other objects or of "attributes". For example, perhaps the most fundamental object is a "page", which consists of a "name" attribute so it can be referred to elsewhere in the script, and many other objects, such as a line, which has attributes of starting location, end location, thickness and colour, or an image, which has attributes of the filename where the image data is stored and a location and optional size where it is to be displayed on the "page".
Another important object is "touch" which has attributes of position and size on the screen, and a sequence of other objects. When a user presses a mouse button or touches the touch screen within this active touch area, the contained objects are executed. These may for instance draw a rectangle or flash a colour to simulate a button being pressed, or significantly direct that a new page be rendered on the screen.
A typical database really is that simple; it consists of pages which contain instructions for rendering objects on the screen as well as descriptions of touch sensitive areas which direct the software to parse a new page. The ordering of the pages in the database is irrelevant, but the ordering of objects within each page defines the order in which objects are drawn. This simple technique is very effective in execution, allowing quite sophisticated animations to be constructed.
{page
name myermenu
{image
name myers/myermenu.xwd
location 0 0
}
{touch
location 35 90
size 275 125
direct store
}
{touch
location 35 231
size 275 125
direct food
}
{touch
location 328 90
size 275 125
direct facils
}
{touch
location 328 231
size 275 125
direct help
}
}
Another object which needs to be explained is the "video" object. The DVA-4000 video board receives instructions as a set of text commands known by the board manufacturers as "MIC" commands. The video object is simply a sequence of these commands. An example MIC command would be "play 500,1250" which means to play the laserdisc from frame 500 to frame 1250. There are other commands to specify the size of the video window on the screen, to start and stop the video etc. One can specify which colours that have been generated by the computer VGA board are to be taken as transparent, so the generated video appears above them on the display, and which colours are to be taken as opaque, so items of those colours will overlay the video.
There are several other objects in the infoterm language, which enable it to render text, draw circles etc., or insert time delays for timing of animation sequences. There are also control structures, viz. the ability to set, change and compare variables then execute objects depending on the results and to allow indirect branching to pages which are specified as variables. These control structures make the infoterm language a fully functional object oriented programming language, albeit an extremely simple one.
Currently in the Unix version of infoterm, and planned for the Microsoft Windows version, is the ability to execute and communicate with any other program. These programs can typically access computer networks and query databases, then return the results in a format which can be read by the infoterm program. Ibis is another example of the great flexibility Of infoterm, achieved through simplicity. Rather than building complexity into the program by giving it the power to address every database imaginable, the language itself can be simply generated by other programs, and infoterm can simply provide text data to those programs. Armed with an infoterm executable one can make it as powerful as one wishes by virtue of its simple philosophy.
Infoterm is written in the 'C' language. The single piece of source code compiles to Unix or Microsoft Windows versions depending on the host machine. At a simple level its major function is to parse and store the objects in the database. It draws the first page on the screen then ultimately receives a touch event, whereat it draws any objects associated with that touch event and if necessary branches to a new page, rendering the objects associated with that page, continuing round that cycle indefinitely.
There is however much programming activity taking place which is not apparent to the user or even the presentation developer. Infoterm does not necessarily read the entire database initially; as has been stated it could in principle be thousands of pages in length. When first executed infoterm reads the database in a two pass operation, first building an index of the pages within the database, then reading in all the objects on a limited number of pages. It also reads the computer image files, if any, referred to by those pages (there usually are; most presentations developed to date have been based around computer generated images).
Figure 2: An example infoterm screen
Figure 3: An example infoterm screen.
Infoterm keeps track of the amount of memory it uses; if a page that is not resident in memory is encountered, it is read in; when the amount of memory passes a "high water mark", the pages that have not been displayed recently are discarded. If the database ever changes, the infoterm software notices, and rereads the relevant data. (This would happen if some data is updated while the program is running.)
Typical user behaviour is to encounter a new page, then look at that page, even if only for a second or two. Infoterm uses this time when there is no user activity to inspect the data it has stored. If, in the neighbourhood of the current location on the data tree there exist pages which infoterm does not have in memory, it reads them. If there is still time, and if necessary, stale pages are then discarded. So, normally, when a user requests a new page by touching the screen, the objects for the relevant page are in memory and the images are in memory. As soon as the screen is touched, infoterm is immediately ready to draw graphics as required.
Because of this philosophy of "looking ahead" for data that may be required soon, from a user's point of view infoterm is agreeably rapid in its response. Perhaps from the developers' point of view it is surprisingly rapid. It is a fascinating exercise to play the part of a hostile user touching the screen as fast as possible in an attempt to bring out any bug in the software. It doesn't fall over, but because it doesn't get any opportunity to look ahead, it does slow down visibly. The software simply won't let a user "push buttons" any quicker than he or she needs to.
The applications for which interactive multimedia systems like infoterm are limited only by the imaginations of the developers. Interest has been expressed in it from various diverse industry representatives including those in tourism, travel, advertising, retailing, education, banking, gambling, entertainment and sports events.
However, the true goal of infoterm is to access complex information across telecommunications networks, accessing remote multimedia databases. Already some fairly simple demonstrations showing infoterm accessing remote electronic telephone directories; this is but the tip of the iceberg.
It is intended to develop infoterm further to integrate it into network wide applications and demonstrations of the possibilities of multimedia technology. To achieve commercial applications of this nature will require substantial research into distributed multimedia data and methods for its access. Some of the possibilities to this end may be discovered in (Jennings et, al., 1992).
Further down the track, we can anticipate the introduction of broadband telecommunications networks (Biggar & Scott, 1992). When these eventuate, it will be possible to transmit high quality images in a fraction of a second, video will be able to be stored in distributed databases and transmitted to the user, completely under the control of that user, in real time.
At that point, interactive multimedia information will become an all pervasive technology, and part of our everyday lives. Infoterm is but a small beginning to a very exciting future.
Clarke, J. (1988). Expo Info - World Expo 88 (Telecom Australia's Interactive Information System). The Telecommunication Journal of Australia, 38(1), 63-68.
Davidson, R. I. and Jenkins, A. R. (1982). Videotex - an emerging public information service. The Telecommunication Journal of Australia, 32(2), 109-114.
Fedida, S. (1976). Viewdata - an interactive information medium for the general public using the telephone networks. 6th International Broadcasting Convention, 20-24 September.
Jennings, A., Flower, M. and Nicholson, P. (1992). A multimedia shop. In Promaco Conventions (Ed.), Proceedings of the International Interactive Multimedia Symposium, 573-583. Perth, Western Australia, 27-31 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1992/jennings.html
Sheehan, J. (1989). In for the long haul. AudioVision, 41.
VideoLogic Inc (1989). DVA-4000/ISA Specification sheet.
| Authors: P. Nicholson, A. Jennings & M. Flower Telecom Australia Research Laboratories 770 Blackburn Road, Clayton Victoria 3168, AUSTRALIA Please cite as: Nicholson, P., Jennings, A. and Flower, M. (1992). A multimedia information software system. In Promaco Conventions (Ed.), Proceedings of the International Interactive Multimedia Symposium, 383-391. Perth, Western Australia, 27-31 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1992/nicholson.html |