Various methods exist to commence with ontology development. Top-down ontology development has the underlying assumption that by using a foundational ontology, one can speed up ontology development and improve quality and interoperability of the domain ontology. Opponents complain that the foundational ontologies are too abstract, too expressive, too comprehensive for ‘simple’ or domain ontologies, and that it takes too much time to understand them in sufficient detail anyway. Informal assessment of these assumptions reveals ambiguous results either way, which are not only open to different interpretations but also such that foundational ontology usage is not foreseen in most methodologies.
So, what should an ontology developer do? The title of this blog post already suggests the answer to this question. It is fair to ask some follow-up questions: why should the developer use a foundational ontology and how and where does it make a difference?
Thus far, there were only theoretical answers to these questions, but no hard evidence to back it up. Sure, the OBO Foundry experiment is ongoing, but, as Ben already mentioned in a previous blog post comment, it is known that there is no empirical proof that validates the assumption. I tried to validate it in one way last year and only recently took the time to write it down. I had set up a controlled experiment with participants of the three ontology engineering courses I taught at the University of Havana and University of Computer Science, in Cuba, and CSIR Meraka in South Africa in 2010. 52 people participated in the experiment who developed 18 ontologies in groups, which were analysed on basic statistics (how many classes, relations, axioms were added), errors, language used, and other observations were noted. The raw data, i.e., the non-anonymised ontologies and some notes, are available upon request.
After a lecture about DOLCE, BFO, and part-whole relations, the lab session with the experiment started with the task to develop a computer ontology (for the time span of 24 hours to hand it in), with as options to start from scratch, to use DOLCE, BFO or GFO, and/or to use the taxonomy of part-whole relations, all made available in their OWLized versions.
One-third chose to start domain ontology development with an OWLized foundational ontology (6 out of 18 ontologies), being either DOLCE or the part-whole relations.
On average, those who commenced with a foundational ontology added more new classes and class axioms, and significantly less object properties than those who started from scratch. No one made the well-known novice error of confusing is-a with part-of, though the ones who did not use a foundational ontology spent some time inventing their own naming scheme for part-whole relations. There were some errors, but none had anything to do with using a foundational ontology or not (e.g., due to OWL 2 DL intricacies). None of the ontologies had the same definition for computer, but the ones that used a foundational ontology were obviously easier to harmonise.
More details—characterisation of participants, description of results, and discussion—are described in the technical report An empirical assessment of the use of foundational ontologies in ontology development [1] (a shorter version is under review at the moment). The comprehensive results show that the ‘cost’ incurred in spending time getting acquainted with a foundational ontology—in casu, DOLCE and a taxonomy of part-whole relations—compared to starting from scratch was more than made up for in size, understandability, and interoperability already within the limited time frame of the experiment.
Aside from further experimentation, the next step is to contemplate how this can be incorporated in extant ontology development methodologies, both in ‘high-level’ methodologies, such as the NeOn Methodology, and in ‘low level’ methodologies that focus on adding the entities, such as OntoSpec and along the line of the example for the African Wildlife Ontology tutorial ontology.
References
[1] C.M. Keet. An empirical assessment of the use of foundational ontologies in ontology development. KRDB Research Centre Technical Report KRDB10-6, Faculty of Computer Science, Free University of Bozen-Bolzano, Italy. December 2010.
Keet blog 