---

Background and Motivation

As a C++ programmer, I have developed many applications in which data needed to be stored in files. Commonly, such data was stored in a text, binary, or CSV format. These formats work well, but are quite rigid. Changes to the file structure within an application often involve writing a conversion program of some kind to transform the data. Even when doing something simple like adding a single field to a record.

XML-structured files, on the other hand, are quite flexible. Data is organized into nodes, and nodes may have named attributes. Adding a new field can be as simple as adding a new named attribute to an existing node.

Although XML is a standard designed for information interchange over the Internet, its flexibility makes it attractive as a file format for data storage in applications. The downside, however, is that processing XML files can be rather complex; parsing techniques must be used, and a general tree data structure is needed to hold the data. As such, a library is needed in order to effectively use XML files as a general storage mechanism in C++ applications.

There are existing C++ libraries for processing XML files. Some are proprietary. Some are open source. I was looking for an open source solution that I could use in my programs, whether those programs were open source or proprietary.

The open source libraries I found were quite cumbersome to use, in my opinion. To retrieve data from an XML file, one needed a series of function calls to navigate the node hierarchy. Then you needed a function call to retrieve the data from a node or a node's attribute. There were often several varieties of function calls, depending on whether you wanted to retrieve string data, integer data, or floating point data from the node. Then, if you wanted to change the data, another function call (with varieties for each type of data) was needed to put the new value into the file.

Don't get me wrong here. I don't envision one getting away from function calls to facilitate accessing XML files. But rather, I wanted the process for the programmer to be more intuitive and less cumbersome. Given that, I set out to write my own library.

In developing xdb, I had these goals in mind:

• The XML file should be viewed as a container of data, and one should be able to access data in this container directly. One should not have to retrieve data from the file using a function call, then use another function call to replace the data in the file.
• As all XML data is considered string data, conversions between data types should be easy and automatic. If a data item you are storing is being used as an integer, then simply use it like an integer. If you are using a data item as a string, reference it in the same way, but use it like a string. There may be lots of data that needs to be converted - I didn't want to have to deal with it all over the place.
• Make use of operator overloading so that one can navigate the data in an intuitive way. For example, one should be able to reference the nodes in an XML file at a given level using array notation. One should be able to use array notation with string indexes to reference the named attributes of a node.
• One should be able to easily navigate nodes based on their tag values, and should be able to easily find nodes with a given tag and attribute/value combination. In xdb, the function call operator is overloaded to easily facilitate these kinds of searches.
• The library should be able to perform sophisticated and complex searches if needed. xdb provides base "predicate" classes on which complex searching and sorting can be done.
• The library should work in both Linux and Windows environments, with a variety of compilers. Further, using it in any of these environments should not involve complex installation and/or usage procedures.
• One should be able to use the library to build proprietary software. (I included this requirement because I, personally, may want to use the library in this way.)

With these goals in mind, I built a prototype library in early 2002 that I called xdb version 0.1. This library's interface relied heavily on C++ STL. As such, the programmer needed to know how to use STL iteraters. Although the library worked well, I found the use of iterators to access the data to be still quite cumbersome. As a professor of computer science, I was not comfortable releasing the library in this state to students who may not yet be familiar with pointer notation.

In mid-summer 2002, I took what I learned from version 0.1 and replaced the library's interface with one that I felt was more intuitive. Although STL is used quite heavily to implement the library, STL iterators are no longer used as part of the library's interface. In only one part of the interface is any kind of pointer notation used, and one need not be comfortable with pointer notation to use it. This work became version 1.0-beta.

In the fall 2002 semester, some of my software engineering students used version 1.0-beta in their projects. The projects were quite successful, but during the course of the semester I identified many areas for improvement in the library - particularly in the area of data conversions. During the 2002-2003 winter break, I modified the library based on this experience. The result was the first "public" release of xdb, version 1.0-beta-2.

I presented version 1.0-beta-2 to my advanced programming class in the spring of 2003. In this class, students were required to implement an independent term project. Some of my students chose to use the library as part of their project. I found that the students who used the library did so with very little difficulty, and built some very successful projects with it. At this point, I felt that xdb had reached a point where I could consider it to be a mature library.

During the summer of 2003, I set out to release the first "official" version of xdb; version 1.0, which included only a few changes from version 1.0-beta-2. That is the work represented here and available today.

---