The TDDonto tool to try out TDD for ontology authoring

Last month I wrote about Test-Driven Development for ontologies, which is described in more detail in the ESWC’16 paper I co-authored with Agnieszka Lawrynowicz [1]. That paper does not describe much about the actual tool implementing the tests, TDDonto, although we have it and used it for the performance evaluation. Some more detail on its design and more experimental results are described in the paper “The TDDonto Tool for Test-Driven Development of DL Knowledge Bases” [2] that has just been published in the proceedings of the 29th International Workshop on Description Logics, which will take place next weekend in Cape Town (22-25 April 2016).

What we couldn’t include there in [2] is multiple screenshots to show how it works, but a blog is a fine medium for that, so I’ll illustrate the tool with some examples in the remainder of the post. It’s an alpha version that works. No usability and HCI evaluations have been done, but at least it’s a Protégé plugin rather than command line :).

First, you need to download the plugin from Agnieszka’s ARISTOTELES project page and place the jar file in the plugins folder of Protégé 5.0. You can then go to the Protégé menu bar, select Windows – Views – Evaluation views – TDDOnto, and place it somewhere on the screen and start using it. For the examples here, I used the African Wildlife Ontology tutorial ontology (AWO v1) from my ontology engineering course.

Make sure to have selected an automated reasoner, and classify your ontology. Now, type a new test in the “New test” field at the top, e.g. carnivore DisjointWith: herbivore, click “Add test”, select the checkbox of the test to execute, and click the “Execute test”: the status will be returned, as shown in the screenshot below. In this case, the “OK” says that the disjointness is already asserted or entailed in the ontology.

Now let’s do a TDD test that is going to fail (you won’t know upfront, of course); e.g., testing whether impalas are herbivores:

The TDD test failed because the subsumption is neither asserted nor entailed in the ontology. One can then click “add to ontology”, which updates the ontology:

Note that the reasoner has to be run again after a change in the ontology.

Lets do two more: testing whether lion is a carnivore and that flower is a plan part. The output of the tests is as follows:

It returns “OK” for the lion, because it is entailed in the ontology: a carnivore is an entity that eats only animals or parts thereof, and lions eat only herbivore and eats some impala (which are animals). The other one, Flower SubClassOf: PlantParts fails as “undefined”, because Flower is not in the ontology.

Ontologies do not have only subsumption and disjointness axioms, so let’s assume that impalas eat leaves and we want check whether that is in the ontology, as well as whether lions eat animals:

The former failed because there are no properties for the impala in the AWO v1, the latter passed, because a lion eats impala, and impala is an animal. Or: the TDDOnto tool indeed behaves as expected.

Currently, only a subset of all the specified tests have been implemented, due to some limitations of existing tools, but we’re working on implementing those as well.

If you have any feedback on TDDOnto, please don’t hesitate to tell us. I hope to be seeing you later in the week at DL’16, where I’ll be presenting the paper on Sunday afternoon (24th) and I also can give a live demo any time during the workshop (or afterwards, if you stay for KR’16).

 

References

[1] Keet, C.M., Lawrynowicz, A. Test-Driven Development of Ontologies. 13th Extended Semantic Web Conference (ESWC’16). Springer LNCS. 29 May – 2 June, 2016, Crete, Greece. (in print)

[2] Lawrynowicz, A., Keet, C.M. The TDDonto Tool for Test-Driven Development of DL Knowledge bases. 29th International Workshop on Description Logics (DL’16). April 22-25, Cape Town, South Africa. CEUR WS vol. 1577.

Advertisement

This blog is now 10 years old

Writing the title of this post does make me wonder how it happened. That blogs are still being read, WordPress that hosts it is still around, and I’m still in academia writing about research and other topics. Honestly, when I started dabbling in writing blog posts, I didn’t expect to last it this long, nor when it celebrated 5 years that another 5 would be added. Nor did I expect to end up persistently receiving typically over 1000 visitors/month, which is fairly popular given the blog’s topics, and surpass the 100 000th visitor some time last year. Admitted, there are not a lot of comments, but then, nor do I comment a lot on other blogs (uncontrollable digital footprint and all that). So, I sat down and wrote a few reflections, which might be of use to someone thinking of starting a (science-oriented) blog or having a dip in posting.

Some pros

Having a blog is useful for learning to try to simplify one’s own research papers into a roughly presentable ‘sound byte’ that can be read in a few minutes. (I don’t get a lot of click-throughs to my papers though, so it may not help with getting people to actually read your papers.)

It is useful to push oneself to take notes during conferences and read those papers, and therewith also reflect on the conferences and workshops one attended (Which papers were actually interesting? Which ones might be useful for your own research? Did someone present a cool solution to a problem you knew of but hadn’t had [at all/enough] time for to solve but are happy someone did?).

If you don’t know what to write about: present/discuss a paper you found interesting, or disagreed with. This helped me at the start to post and not let the blog fizzle out. Admittedly, I have plenty of accepted papers now so as to space it out to ‘market’ one per month. Nevertheless, giving ‘airtime’ to others has its merits, especially when they are in the area of your research, for it shows you actually do read other papers, critically. The offline thank-yous are just a bonus.

That said, it is nice to get feedback from other researchers and readers on the posts; and it really doesn’t matter whether that is left as a comment on the blog, by email, or in person. While they may be pleased I mentioned their paper in a post, I’m happy to know someone used up some of that extremely precious resource—time—to read what I wrote.

I like to think that my writing has improved over the years. If not that, then at least it now takes less time to write at that very same level.

 

Some caveats

That much for the good side that I could come up with. What about the negative side? Mainly, it does take up quite a lot of time to write up the posts. Writing one evening, reading and revising it the next day, and all the layouting of the post can take up several hours. Those hours could have been spent differently.

You won’t know upfront which posts will be ‘popular’; some posts that I liked aren’t popular at all, yet some that I thought were minor, are. I haven’t deciphered a correlation between effort put in to write a post and its popularity either. I write what I fancy writing about and then hope for the best. Looking at the statistics of page visits, popular-science posts have many, many more visits than posts about my research.

Even when trying to be polite when writing, reading some of the posts years hence, some things seem to be formulated harsher than intended. There is the danger to piss off someone, and therewith feeding the rumour that blogs may be harmful.

Related to the latter, is that there are some things I wanted, or even craved, to write about, but couldn’t due to the decision to have a non-anonymous blog. That said, even anonymous blogs can be ‘revealed’ (e.g., fsp), and some things just fester better through the grapevine.

You may have trolls. I did have them. This depended on the topic (such as positive posts about Cuba), but may also be immature students or a colleague with an axe (of the xenophobic, racist, and/or sexist type) to grind. Ignore them.

 

Final remarks

Does doing this blogging actually have any impact on the level of my ‘popularity’ or ‘standing’ (good or bad) in the research community? I don’t have the faintest idea.

Will I go on for another 10 years? I don’t know. For now, I still try to write at least 2 posts per month. I hope you stay with me, but I also know that interests change, so I will not hold abandoning keetblog against you J. In fact, I am grateful you have taken, take, and/or will take the time to read the blog, and I hope you consider it time well spent, not wasted, or that it may have been effective for your structured procrastination.