The TDDonto tool to try out TDD for ontology authoring

Last month I wrote about Test-Driven Development for ontologies, which is described in more detail in the ESWC’16 paper I co-authored with Agnieszka Lawrynowicz [1]. That paper does not describe much about the actual tool implementing the tests, TDDonto, although we have it and used it for the performance evaluation. Some more detail on its design and more experimental results are described in the paper “The TDDonto Tool for Test-Driven Development of DL Knowledge Bases” [2] that has just been published in the proceedings of the 29th International Workshop on Description Logics, which will take place next weekend in Cape Town (22-25 April 2016).

What we couldn’t include there in [2] is multiple screenshots to show how it works, but a blog is a fine medium for that, so I’ll illustrate the tool with some examples in the remainder of the post. It’s an alpha version that works. No usability and HCI evaluations have been done, but at least it’s a Protégé plugin rather than command line :).

First, you need to download the plugin from Agnieszka’s ARISTOTELES project page and place the jar file in the plugins folder of Protégé 5.0. You can then go to the Protégé menu bar, select Windows – Views – Evaluation views – TDDOnto, and place it somewhere on the screen and start using it. For the examples here, I used the African Wildlife Ontology tutorial ontology (AWO v1) from my ontology engineering course.

Make sure to have selected an automated reasoner, and classify your ontology. Now, type a new test in the “New test” field at the top, e.g. carnivore DisjointWith: herbivore, click “Add test”, select the checkbox of the test to execute, and click the “Execute test”: the status will be returned, as shown in the screenshot below. In this case, the “OK” says that the disjointness is already asserted or entailed in the ontology.

Now let’s do a TDD test that is going to fail (you won’t know upfront, of course); e.g., testing whether impalas are herbivores:

The TDD test failed because the subsumption is neither asserted nor entailed in the ontology. One can then click “add to ontology”, which updates the ontology:

Note that the reasoner has to be run again after a change in the ontology.

Lets do two more: testing whether lion is a carnivore and that flower is a plan part. The output of the tests is as follows:

It returns “OK” for the lion, because it is entailed in the ontology: a carnivore is an entity that eats only animals or parts thereof, and lions eat only herbivore and eats some impala (which are animals). The other one, Flower SubClassOf: PlantParts fails as “undefined”, because Flower is not in the ontology.

Ontologies do not have only subsumption and disjointness axioms, so let’s assume that impalas eat leaves and we want check whether that is in the ontology, as well as whether lions eat animals:

The former failed because there are no properties for the impala in the AWO v1, the latter passed, because a lion eats impala, and impala is an animal. Or: the TDDOnto tool indeed behaves as expected.

Currently, only a subset of all the specified tests have been implemented, due to some limitations of existing tools, but we’re working on implementing those as well.

If you have any feedback on TDDOnto, please don’t hesitate to tell us. I hope to be seeing you later in the week at DL’16, where I’ll be presenting the paper on Sunday afternoon (24th) and I also can give a live demo any time during the workshop (or afterwards, if you stay for KR’16).

 

References

[1] Keet, C.M., Lawrynowicz, A. Test-Driven Development of Ontologies. 13th Extended Semantic Web Conference (ESWC’16). Springer LNCS. 29 May – 2 June, 2016, Crete, Greece. (in print)

[2] Lawrynowicz, A., Keet, C.M. The TDDonto Tool for Test-Driven Development of DL Knowledge bases. 29th International Workshop on Description Logics (DL’16). April 22-25, Cape Town, South Africa. CEUR WS vol. 1577.

Advertisement

72010 SemWebTech lecture 7: Dealing with uncertainty and vagueness

The third advanced ontology engineering topic concerns how to cope with uncertainty and vagueness in ontology languages and their reasoners—and what we can gain from all the extra effort.

For instance, consider information retrieval: to which degree is a web site, a page, a text passage, an image, or a video segment relevant to the information need and an acceptable answer to what the user was searching for? In the context of ontology alignment, one would want to know (automatically) to which degree the focal concepts of two or more ontologies represent the same thing, or are sufficiently overlapping. In an electronic health record system, one may want to classify patients based on their symptoms, such as throwing up often, having a high blood pressure, and yellowish eye colour. How can software agents do the negotiation for your holiday travel plans that are specified imprecisely, alike “I am looking for a package holiday of preferably less than 1000 euro, but really no more that 1150 euro, for about 12 days in a warm country”?

The main problem to solve, then, is what and how to incorporate such vague or uncertain knowledge in OWL and its reasoners. To clarify these two terms upfront:

• Uncertainty: statements are true or false, but due to lack of knowledge we can only estimate to which probability / possibility / necessity degree they are true or false;
• Vagueness: statements involve concepts for which there is no exact definition (such as tall, small, close, far, cheap, expensive), which are then true to some degree, taken from a truth space.

The two principal approaches regarding uncertainty and the semantic web are probabilistic and possibilistic languages, ontologies, and reasoning services, where the former way of dealing with uncertainty receives a lot more attention than the latter. The two principal approaches regarding vagueness and the semantic web are fuzzy and rough extensions, where fuzzy receives more attention compared to the rough approach. The lecture will cover all four approaches to a greater (probabilistic, fuzzy) and lesser (possibilistic, rough) extent.

None of the extant languages and automated reasoners that can cope with vague or uncertain knowledge have made it into ‘mainstream’ Semantic Web tools yet. There was a W3C incubator group on uncertainty, but it remained at that. This has not stopped research in this area; on the contrary. There are two principle strands in these endeavours: one with respect to extending DL languages and its reasoners, such as Pronto that combines the pellet reasoner with a probabilistic extension and FuzzyDL that is a reasoner for fuzzy SHIF(D), and another strand to uses different techniques underneath OWL, such as Bayesian networks and constraint programming-based reasoning for probabilistic ontologies (e.g., PR-OWL), and Mixed Integer Logic Programming for fuzzy ontologies. Within the former approach, one can make a further distinction between extensions of tableaux algorithms and rewritings to a non-uncertain/non-vague standard OWL language so that one of the generic DL reasoners can be used. For each of these branches, there are differences as to which aspects of probabilistic/possibilistic/fuzzy/rough are actually included—just like we saw in the previous lecture about temporal logics.

We shall not cover all such permutations in the lecture, but instead focus on general aspects of the languages and tools. A good introductory overview can be found in [1] (which also has a very long list of references to start delving into the topics [you may skip the DLP section]).  Depending on your background education and the degree programme you are studying now, you may find the more technical overview [2] of interest as well. To get an idea of one of the more recent results on rough DL-based ontologies, you might want to glance over [3]. Last, I assume you have a basic knowledge of probability theory and fuzzy sets; if there are many people who do not, I will adjust the lecture somewhat, but you are warmly advised to look it up before the lecture if you do not know about it (even if it is only the respective Wikipedia entry here and here).

References

[1] Umberto Straccia. Managing Uncertainty and Vagueness in Description Logics, Logic Programs and Description Logic Programs. In Reasoning Web, 4th International Summer School, 2008.
[2] Thomas Lukasiewicz and Umberto Straccia. 2008. Managing Uncertainty and Vagueness in Description Logics for the Semantic Web. Journal of Web Semantics, 6:291-308.
[3] Jiang, Y., Wang, J., Tang, S., and Xiao, B. 2009. Reasoning with rough description logics: An approximate concepts approach. Information Sciences, 179:600-612.

Note: references 1 or 2 is mandatory reading, 3 optional.

Lecture notes: lecture 7 – Uncertainty and vagueness

Course website