The hyperbase developer's toolkit

1. Introduction

The ability to effectively migrate paper based information into a database is crucial to hypermedia's commercial success [Glushko 89]. We are aware that many people are trying to address this difficulty. Most of these efforts attempt to automate this "text to hypertext" process [Raymond 87, Salton 88, Glushko 89]. To do this they use structural information to guide the conversion, ie. keying on markup languages such as SGML or undertaking layout analysis of the text [Frisse 88, Furuta 89, Niblett 89, Zobel et al. 1991]. These efforts have proven to be of limited value. We believe that at present, in order to produce high quality hypertext components, the human cannot be removed from the conversion process [Furuta 89, Frisse 88, Raymond 88, Bernstein 1990]. Thus the expense of constructing hyper-databases must presently include the price of the human expert's participation. By providing software tools to assist the human in this editorial process, we believe that both the overall cost and time involved can be reduced to a commercially acceptable level.

This paper reviews the nature of this "transformation" problem and discuss a solution which employs Hyperbase Developer's Toolkit (HDT), a software system which the KIN project team designed and developed to assist the hyper-editor in the hypermedia database creation operation. Section 2 give an overview of our definitions of hypersystem components and section 3 describes the text to hypertext conversion process. Section 4 presents the Hyperbase Developer's Toolkit in detail and we conclude with discussion of future work in section 5.

2. Hypersystem components

2.1 Nodes

2.2 Anchors and links

There are two classes of links: structural and referential. The main functions of structural links are for ordering nodes in the same sequence as they appear in the original document and for integrating nodes into a conceptual schema. A hierarchical index of the subject domain can be created and nodes pertinent to each point in the schema can be linked to that point i(Fig. 1). Referential links are generated to bind common concepts together within the set of nodes. As in figure 2, the mention of the Viscomte Francois Rene de Chateaubriand in a node discussing people whose names have become household words would have referential links to nodes containing elaborative information on this man.

Figure 1: Structural links

2.3 Index

3. Text to hypertext process

3. 1 Stage One: Conversion to an electronic medium

3.2 Stage Two: Text to hypertext transformation

3.3 Conversion methodologies

Figure 2: Referential links

For these reasons, the KIN Project is approaching this process from the perspective that the human must be provided with tools to assist them in the text to hypertext transformation process . The following sections give a description of the tool currently under development.

4. Description of the toolkit

4.1 Design principles

To ensure that the hyperbase creation tools are generic, it is important to adhere to standards. However, there is no commonly agreed upon standard for hyperbase system development at this moment. Therefore, we have designed our database schema in such a way that it can be easily modified to conform to emerging standards in the future. This schema is based on the notion that the knowledge in the hyperbase is make up of a collection of nodes or logical information modules (LIMs). These LIMs are arranged in a tree structure which reflect the logical relationship amongst the LIMs. For example, in one of our projects, the Australian Heritage Commission (AHQ Demonstrator, the flora information is arranged in genus-species structure where species are the children of a genus. For documents, this tree structure reflects the arrangement of sections and subsections within the document. In addition to the hierarchical links implied in the tree structure relationship, referential links are built for key phrases in the hyperbase. This design results in a network of linkages which provide the user with a mechanism to browse through the information space by following referential or hierarchical links.

4.2 Overview

There are four modules in the system:

1. Editorial Module --
   This module performs the transformation from raw to marked up data. Both batch and interactive mode of this module are implemented.

2. Hyperbase creation/loading Module -- is performed.

   In this module, transformation from marked up data to database storage is performed. Syntax errors in the mark up are flagged in this stage.

3. Maintenance Module --
   This module provides the facility to give selective dumps of the hyperbase content and statistical listing of data in the hyperbase.

4. Presentation Module --
   This is the front end of the system where the GUI is implemented to handle interactions between users and the hyperbase.

4.3 Editorial stage

Initially, the editorial stage is entirely decoupled from the hyperbase creation and loading stage. The user edits a text file to put in simple markers to denote nodes, anchors and various components in a hypertext. This process was greatly simplified by the fact that we are working with knowledge in a restricted domain (flora of Tasmania). Once the user has finished with marking up the file, a hyperbase loading program is invoked to parse the file and load the data into the hyperbase as described in the next section. This arrangement worked quite well but we expect an interactive environment should greatly enhance the productivity of the user and reduce error in the processed data. The following subsection describes this improved interactive editorial stage.

4.3.2 Interactive mode

The human uses a customised text editor to identify the various hypertext components contained within the linear document (Fig. 3). The markup language used for this stage includes symbols which define anchors, nodes, node titles, anchor types, node categories, key phrases and index values (Fig. 4). In addition, there is an interactive link-generation tool which allows the editor to choose the destinations of anchors by presenting a list of existing node tides as potential candidates (Fig. 5). Error correction at this stage is limited to ensuring names given to the nodes in the hyper-document are unique.

Figure 3: The Text Editor where nodes are identified

Figure 4(a): Mark a node in a document

Figure 4(b): Process a node individually

Figure 5(a): Window for anchor/link specification

Figure 5(b): Window for keyword/key phrase specification

4.4 Hyperbase creation and loading

With the initial design, the hyperbase is loaded in batch mode. The system reads in and parses all the specified marked up files, loads the hyperbase and generates some diagnostic comments regarding the status of the operation. Anchors without destination are also output as part of the diagnostic comments. The drawback of this batch mode loading is that we have to wait until the loading has finished before any correction can be made. Described in the following subsection is an improved version of hyperbase loading where this process is integrated into the editorial stage in an interactive environment.

4.4.2 Interactive mode

Once the user is satisfied with the markup of the raw data, a parser is invoked from the editor to read in the text file and to insert or update this new data into the hyperbase. At this stage, error checking is performed at the syntactic level. For example, anchors that have no associated link and links which do not have an origin or a destination anchor can be flagged for correction without delay. In addition, the text file with markups can be saved to the file system for backup purposes or future reference and update.

4.5 Maintenance stage

4.6 Presentation stage

4.6.1 The main display

This module consists of GUI which allows the user to access information in the hyperbase. There is a main window (Fig. 6) which consists of three areas:

1. Active information display (AID)
2. History and bookmark window
3. Function panel

Figure 6: The main window

If there are pictures associated with a LIM, a click on the Graphics button will give a pull down menu which consists of a list of picture titles. A picture is displayed by selecting a title from this menu. In addition, if there is a map associated with the LIM, a click on the Map button will instruct the system to retrieve the associated map from a remote GIS database [Abel et al. 1991]. There is a facility to save the content of a LIM into a text file. When the Save button is clicked, the user is prompted for a file name and the content of the currently displayed LIM will be saved into a disk file.

A floating popup menu is associated with this main window. Various hypertext utilities can be invoked by selecting an entry in this menu. Currently, the utilities supported are: Search by boolean query, Hierarchical index, Fish-eye index [Furnas, 861, and a simple editorial tool for editing a special type of LIMs which have a pre-defined format. The following subsections describes each of these in more details.

4.6.2 Hypertext navigation utilities

There are several ways in which the user can access the information in the hyperbase:

1. By following referential links:
   This is achieved by doubling clicking on an anchor (delimited by { and } in the AID.

2. By following hierarchical links:
   There are two index displays in this system. The hierarchical index (Fig. 7) gives a list of children's LIM title of the currently displayed LIM. Selecting an entry in this list will bring up the text of the selected child. The user can traverse up the hierarchical tree by clicking on the Parent button. Clicking on the Root button will bring the user back to the root of the tree.
   

   Figure 7: The hierarchial index

   The shortcomings of this index are that we cannot see more than one level of LIM titles at a time. The fish-eye index (Figure 8) [Furnas, 861 is provided to overcome this problem. This index display lets the user have a variable resolution view of the hierarchical tree structure by incorporating the concept of resolution order. With zero order, the tree consists of the ancestry list from the current LIM to the root of the tree (ie. root, ....., grandparent, parent, current LIM). With first order, the tree is augmented by including the immediate children of all LIMs in the zeroth order tree. This can be generalised recursively to any order greater than 0.
   

   Figure 8: The fish-eye index

3. By performing a search:
   A search window (Figure 9) is provided for the user to perform key words or key phrase search. Boolean search queries can be easily constructed by typing in words, clicking on boolean operators buttons or selecting from alphabetically listed key phrases in the hyperbase. The system then searches for LIMs which satisfy the search query and the results are displayed as a list of LIM tides. By selecting an entry in this result list, the corresponding LIM will be displayed in the AID.
   

   Figure 9: The search window

In the Australian Heritage Commission Demonstrator project, there is a special category of LIMs called Type Profile. Type profiles contain description of specific information types in the knowledge domain. This information is structured into a template format so that comparison amongst different profiles can be performed easily and uniformly. With this standard format, the author of type profiles need only to fill in a proforma relating to a specific class of types. A mechanism to create, update and delete type profiles is included in the system.

There is a main window for the type profile related functions (Fig. 10). It consists of a functions panel and a type profile active display area (TPAID). Two templates, which correspond to two classes of types in the system are provided in this version. Subsections in the template can be edited one at a time and the resulting type profile will be displayed in TPAID. While editing a subsection, the user can type in words and make link to LIM currently displayed in AID. Operation of cut and paste from AID into type profile is not provided because this will cause information redundancy in the system.

Figure 10: The type profile main window

5.0 Discussion, future work and conclusion

5.1 Discussion

5.2 Future directions

5.2 Conclusion

References

Bernstein, M. (1990). An apprentice that discovers hypertext links. In A. Rizk et al (eds), Hypertext: Concepts, Sytems and Applications. Cambridge University Press, Cambridge.

Clitherow, P., Riecken, D. and Miller, M. (1989). VISAR: A system for inference and navigation in hypertext. Hypertext '89, Pittsburgh, Nov 1989.

Frisse, M. (1988). From text to hypertext. Byte, October, 247-253.

Frisse, M. (1988). Searching for information in a hypertext medical handbook. Communications of the ACM, 31(7, July), 880-886.

Furnas, G. W. (1986). Generalised fish-eye views. CHI'86 Proceedings, 16-23. Boston, MA.

Furuta, R., Plaisant, C. and Schneidermann, B. (1989). A spectrum of automatic hypertext construction. Hypermedia, 1(2), 179-195.

Glushko, R. J. (1989). Design issues for multi-documents hypertexts. Hypertext '89, Pittsburgh, Nov 1989.

Hayes, P. and Pepper, J. (1989). Towards an integrated maintenance advisor. Hypertext '89, 119-127. Pittsburgh, Nov 1989.

ISO 8613 (1989). Information Processing - Text and Office Systems - Office Document Architecture (ODA) and Interchange Format, 1989.

ISO 8879 (1986). Information Processing - Tools and Office Systems - Standard Generalised Markup Language (SGML), 1986.

Lenat, D. B. (1986). CYC: Using common sense knowledge to overcome brittleness and knowledge acquisition bottlenecks. AI Magazine,, 6.

Lenat, D. B. and Guha, R. V. (1989). Building Large Knowledge Databases. Addison-Wesley.

Niblett, T. and van Hoff, A. (1989). Programmed hypertext and SGML. Sept 1989, The Turing Institute.

Parsaye, K., Chignell, M., Khoshafian, S. and Wong, H. (1989). Intelligent Databases: Object Oriented, Deductive, Hypermedia Technologies. New York: Wiley and Sons.

Raymond, D. and Tompa, F. (1987). Hypertext and the New Oxford Dictionary. Hypertext '87 Proceedings, 143-154. Chapel Hill, North Carolina.

Raymond, D. and Tompa, F. (1988). Hypertext and the New Oxford Dictionary. Communications of the ACM, 31(7), 871-879.

Salton, G. (1988). Automatic text indexing using complex identifiers. Proceedings ACM Conference on Document Processing Systems, Santa Fe, New Mexico.

Thom, J. A., Kent, A. J. and Sacks-Davis, R. (1990). A post-relational query language. Technical Report #17, Royal Melbourne Institute of Technology.

Zobel, J., Wilkinson, R., Thom, J. A., Mackie, E., Sacks-Davis, R., Kent, A. J. and Fuller, M. (1991). An architecture for hyperbase systems. Paper given at The First Australian Multimedia, Communications, Applications and Technology Workshop.

Authors: John Robertson and Kai Foong Knowledge and Information Navigation Project, Knowledge Based Systems Division of Information Technology, CSIRO, Australia Contact: John Robertson robertso@syd.dit.csiro.au Phone: (61) (02) 8879496 Fax: (61) (02) 8887787 Please cite as: Robertson, J. and Foong, K. (1992). The hyperbase developer's toolkit. In Promaco Conventions (Ed.), Proceedings of the International Interactive Multimedia Symposium, 393-407. Perth, Western Australia, 27-31 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1992/robertson.html