Outcome of the empirical assessment on the use of foundational ontologies in ontology development

In an earlier post, I described briefly an experiment I had carried out with 52 (novice) ontology developers who had developed 18 ontologies, 1/3 of whom had use a foundational ontology voluntarily, and whose ontologies were better than those who did not use a foundational ontology in domain ontology development. It being the first empirical experiment on this matter, the slightly shorter version of the tech report mentioned in that earlier blog post has been accepted as full paper at the 8th Extended Semantic Web Conference (ESWC’11).

The informal summary with some details were already introduced in the earlier post, so I will include only the abstract of the paper The use of foundational ontologies in ontology development: an empirical assessment here:

There is an assumption that ontology developers will use a top-down approach by using a foundational ontology, because it purportedly speeds up ontology development and improves quality and interoperability of the domain ontology. Informal assessment of these assumptions reveals ambiguous results that are not only open to different interpretations but also such that foundational ontology usage is not foreseen in most methodologies. Therefore, we investigated these assumptions in a controlled experiment. After a lecture about DOLCE, BFO, and part-whole relations, one-third chose to start domain ontology development with an OWLized foundational ontology. On average, those who commenced with a foundational ontology added more new classes and class axioms, and significantly less object properties than those who started from scratch. No ontology contained errors regarding part-of vs. is-a.

The comprehensive results show that the ‘cost’ incurred spending time getting acquainted with a foundational ontology compared to starting from scratch was more than made up for in size, understandability, and interoperability already within the limited time frame of the experiment.

The last thing has not been said about it though. E.g., is 1/3 few or a lot? It remains unclear why the participants preferred reusing DOLCE over BFO, and what the outcome will be if also much larger ontologies, such as Cyc or SUMO, were to be added to the options in a controlled experiment. Also, it may be interesting to see similar experiments with other lecturers and other types of participants, such as with non-computing domain experts with experience in modeling, or a longer time period than used for this experiment. Further, only preliminary suggestions were made how one may want to include the use of foundational ontologies in ontology development, which should be done both at the high-level steps in the development process—none includes something about that now—as well as methods for the actual modeling, where only OntoSpec makes a first attempt in that direction.

References

[1] Keet, C.M. The use of foundational ontologies in ontology development: an empirical assessment. 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS (in print).

Advertisement