Automatically finding the feasible object property

Late last month I wrote about the updated taxonomy of part-whole relations and claimed it wasn’t such a big deal during the modeling process to have that many relations to choose from. Here I’ll back up that claim. Primarily, it is thanks to the ‘Foundational Ontology and Reasoner enhanced axiomatiZAtion’ (FORZA) approach which includes the Guided ENtity reuse and class Expression geneRATOR (GENERATOR) method that was implemented in the OntoPartS-2 tool [1]. The general idea of the GENERATOR method is depicted in the figure below, which outlines two scenarios: one in which the experts perform the authoring of their domain ontology with the help of a foundational ontology, and the other one without a foundational ontology.

generator

I think the pictures are clearer than the following text, but some prefer text, so here goes the explanation attempt. Let’s start with scenario A on the left-hand side of the figure: a modeller has a domain ontology and a foundational ontology and she wants to relate class two domain classes (indicated with C and D) and thus needs to select some object property. The first step is, indeed, selecting C and D (e.g., Human and Heart in an anatomy ontology); this is step (1) in the Figure.

Then (step 2) there are those long red arrows, which indicate that somehow there has to be a way to deal with the alignment of Human and of Heart to the relevant categories in the foundational ontology. This ‘somehow’ can be either of the following three options: (i) the domain ontology was already aligned to the foundational ontology, so that step (2) is executed automatically in the background and the modeler need not to worry, (ii) she manually carries out the alignment (assuming she knows the foundational ontology well enough), or, more likely, (iii) she chooses to be guided by a decision diagram that is specific to the selected foundational ontology. In case of option (ii) or (iii), she can choose to save it permanently or just use it for the duration of the application of the method. Step (3) is an automated process that moves up in the taxonomy to find the possible object properties. Here is where an automated reasoner comes into the equation, which can step-wise retrieve the parent class, en passant relying on taxonomic classification that offers the most up-to-date class hierarchy (i.e., including implicit subsumptions) and therewith avoiding spurious candidates. From a modeller’s viewpoint, one thus only has to select which classes to relate, and, optionally, align the ontology, so that the software will do the rest, as each time it finds a domain and range axiom of a relationship in which the parents of C and D participate, it is marked as a candidate property to be used in the class expression. Finally, the candidate object properties are returned to the user (step 4).

While the figure shows only one foundational ontology, one equally well can use a separate relation ontology, like PW or PWMT, which is just an implementation variant of scenario A: the relation ontology is also traversed upwards and on each iteration, the base ontology class is matched against relational ontology to find relations where the (parent of the) class is defined in a domain and range axiom, also until the top is reached before returning candidate relations.

The second scenario with a domain ontology only is a simplified version of option A, where the alignment step is omitted. In Figure-B above, GENERATOR would return object properties W and R as options to choose from, which, when used, would not generate an inconsistency (in this part of the ontology, at least). Without this guidance, a modeler could, erroneously, select, say, object property S, which, if the branches are disjoint, would result in an inconsistency, and if not declared disjoint, move class C from the left-hand branch to the one in the middle, which may be an undesirable deduction.

For the Heart and Human example, these entities are, in DOLCE terminology, physical objects, so that it will return structural parthood or plain parthood, if the PW ontology is used as well. If, on the other hand, say, Vase and Clay would have been the classes selected from the domain ontology, then a constitution relation would be proposed (be this with DOLCE, PW, or, say, GFO), for Vase is a physical object and Clay an amount of matter. Or with Limpopo and South Africa, a tangential proper parthood would be proposed, because they are both geographic entities.

The approach without the reasoner and without the foundational ontology decision diagram was tested with users, and showed that such a tool (OntoPartS) made the ontology authoring more efficient and accurate [2], and that aligning to DOLCE was the main hurdle for not seeing even more impressive differences. This is addressed with OntoPartS-2, so it ought to work better. What still remains to be done, admittedly, is that larger usability study with the updated version OntoPartS-2. In the meantime: if you use it, please let us know your opinion.

 

References

[1] Keet, C.M., Khan, M.T., Ghidini, C. Ontology Authoring with FORZA. 22nd ACM International Conference on Information and Knowledge Management (CIKM’13). ACM proceedings, pp569-578. Oct. 27 – Nov. 1, 2013, San Francisco, USA.

[2] Keet, C.M., Fernandez-Reyes, F.C., Morales-Gonzalez, A. Representing mereotopological relations in OWL ontologies with OntoPartS. 9th Extended Semantic Web Conference (ESWC’12), Simperl et al. (eds.), 27-31 May 2012, Heraklion, Crete, Greece. Springer, LNCS 7295, 240-254.

A few refreshing feminist articles—to point out and fix bugs in the game

Most articles on gender issues and feminism regurgitate the same old story and arguments, or are reports on more data and experiments with similar results popping up. Some articles or blog posts do bring something relatively new to the table, or apply a feminist analysis to something else, or explain things in a novel way that resonates better in this day and age. Upfront, to those who think gender issues and feminism is mostly rubbish, please read the parable by John Scalzi about the computer game, which is set at the lowest difficulty setting in the Game of the Real World for the Straight White Male; then read ‘those feminazi articles’ as one of pointing out bugs in the code, and of suggesting bug fixes or of a slight rewriting of the game logic to level the playing field. So, here are a few links to some such articles that otherwise may be snowed-under by the online articles on women in STEM, IT, management etc.

The feminist appraisal of Dirty Dancing over at Jezebel’s blog, or, as another one puts it “It’s the feminist sleeper agent of chick flicks” (and some class issues); after reading this, you won’t see the movie the same anymore. (Yes, I did watch the movie again, and the points made in the articles are valid, which, honestly, had escaped me when I watched it in the 80s.)

The many shortcomings of (old) white men futurology, who have a rather limited set of imaginations (fantasies?) in prognosticating. Maybe people in that (non-STEM) discipline already know about the issues and limitations, but I’m in another field of research, so it was new to me. Obviously, if futurology is a science, then it should not make a difference whether men or women do it, but that’s another discussion.

The Super-exploitation of women by Marlene Dixon on capitalism and patriarchy in cahoots to keep women as their unpaid servants and labour-producers wives. I did search for more recent analyses, but they don’t compare in content and clarity to this one.

I did not manage to find again the recent fine rant on feminist issues in Africa that are, at least in part, different from ‘the [white middle-class] feminism in the West’, but these will do on scope as well: feminism here on the continent driven by African women who really do have lots of agency (e.g., all the way up to presidents/prime ministers and Nobel Peace Prize winners) and where certain types of ‘help’ from the outside is counterproductive for it enforces dependence. An example of a currently hot topic here (and, afaik, never was in Europe) is the need for free sanitary pads for girls whose family cannot afford them, so that they can keep going to school to learn rather than miss out on it for a few days each month.

Finally, a slightly crudely formulated article that discusses a whining “pick-up artist” who is “cockblocked by redistribution” in Denmark, a socialist-like and feminist-friendly country. Squeezed between the chatter are notes on flaws on evolutionary psychology and the criticism on feminism as an individual pursuit (e.g., ‘lean in’) versus as a collective goal. Even the pick-up artist eventually notes “we can’t fulfill basic human rights for all without viewing everyone as equal”.

New OWL files for the (extended) taxonomy of part-whole relations

Once upon a time (surely >6 years ago) I made an OWL file of the taxonomy of part-whole relations [1], which contains several parthood relations and a few meronyic-only ones that in natural language are considered ‘part’ but are not so according to mereology (like participation, membership). Some of these relations were defined with a specific domain and range that was a DOLCE category (it could just as well have been, say, GFO). Looking at it recently, I noticed it was actually a bit scruffy (but I’ll leave it here nonetheless), and more has happened in this area over the years. So, it was time for an update on contents and on design.

For the record on how it’s done and to serve, perhaps, as a comparison exercise on modeling, here’s what I did. First of all, I started over, so as to properly type the relations to DOLCE categories, with the DOLCE IRIs rather than duplicated as DOLCE-category-with-my-IRI. As DOLCE is way too big and slows down reasoning, I made a module of DOLCE, called DOLCEmini, mainly by removing the irrelevant object properties, though re-adding the SOB, APO and NAPO that’s in D18 but not in DOLCE-lite from DLP3791. This reduced the file from DOLCE-lite’s 534 axioms, 37 classes, 70 OPs, in SHI to DOLCEmini’s 388 axioms, 40 classes, 43 OPs, also in SHI, and I changed the ontology IRI to where DOLCEmini will be put online.

Then I created a new ontology, PW.owl, imported DOLCEmini, added the taxonomy of part-whole relations from [1] right under owl:topObjectProperty, with domain and range axioms using the DOLCE categories as in the definitions, under part-whole. This was then extended with the respective inverses under whole-part, all the relevant proper part versions of them (with inverses), transitivity added for all (as the reasoner isn’t doing it [2]) annotations added, and then aligned to some DOLCE properties with equivalences. This makes it to 524 axioms and 79 object properties.

I deprecated subquantityOf (annotated with ‘deprecated’ and subsumed by a new property ‘deprecated’). Several new stuff relations and their inverses were added (such as portions), and annotated them. This made it to the PW ontology of 574 axioms (356 logical axioms) and 92 object properties (effectively, for part-whole relations: 92 – 40 from dolce – 3 for deprecated = 49).

As we made an extension with mereotopology [3] (and also that file wasn’t great, though did the job nevertheless [4]), but one that not everybody may want to put up with, yet a new file was created, PWMT. PWMT imports PW (and thus also DOLCEmini) and was extended with the main mereotopological relations from [3], and relevant annotations were added. I skipped property disjointness axioms, because they don’t go well with transitivity, which I assumed to be more important. This makes PWMT into one of 605 (380 logical) axioms and 103 object properties, with, effectively, for parts: 103 – 40 from dolce – 3 for deprecated – 1 connection = 59 object properties.

That’s a lot of part-whole relations, but fear not. The ‘Foundational Ontology and Reasoner enhanced axiomatiZAtion’ (FORZA) and its tool that incorporates with the Guided ENtity reuse and class Expression geneRATOR (GENERATOR) method [4] describes a usable approach how that can work out well and has a tool for the earlier version of the owl file. FORZA uses an optional decision diagram for the DOLCE categories as well as the automated reasoner so that it can select and propose to you those relations that, if used in an axiom, is guaranteed not to lead to an inconsistency that would be due to the object property hierarchy or its domain and range axioms. (I’ll write more about it in the next post.)

Ah well, even if the OWL files are not used, it was still a useful exercise in design, and at least I’ll have a sample case for next year’s ontology engineering course on ‘before’ and ‘after’ about questionable implementation and (relatively) good implementation without the need to resorting to criticizing other owl files… (hey, even the good and widely used ontologies have a bunch of pitfalls, whose amount is not statistically significantly different from ontologies made by novices [5]).

 

References

[1] Keet, C.M., Artale, A. Representing and Reasoning over a Taxonomy of Part-Whole Relations. Applied Ontology, 2008, 3(1-2):91-110.

[2] Keet, C.M. Detecting and Revising Flaws in OWL Object Property Expressions. 18th International Conference on Knowledge Engineering and Knowledge Management (EKAW’12), Oct 8-12, Galway, Ireland. Springer, LNAI 7603, 252-266.

[3] Keet, C.M., Fernandez-Reyes, F.C., Morales-Gonzalez, A. Representing mereotopological relations in OWL ontologies with OntoPartS. 9th Extended Semantic Web Conference (ESWC’12), Simperl et al. (eds.), 27-31 May 2012, Heraklion, Crete, Greece. Springer, LNCS 7295, 240-254.

[4] Keet, C.M., Khan, M.T., Ghidini, C. Ontology Authoring with FORZA. 22nd ACM International Conference on Information and Knowledge Management (CIKM’13). ACM proceedings, pp569-578. Oct. 27 – Nov. 1, 2013, San Francisco, USA.

[5] Keet, C.M., Suarez-Figueroa, M.C., Poveda-Villalon, M. Pitfalls in Ontologies and TIPS to Prevent Them. In: Knowledge Discovery, Knowledge Engineering and Knowledge Management: IC3K 2013 Selected papers. Fred, A., Dietz, J.L.G., Liu, K., Filipe, J. (Eds.). Springer, CCIS 454, pp. 115-131. 2015.

An exhaustive OWL species classifier

Students enrolled in my ontology engineering course have to do a “mini-project” on a particular topic, chosen from a list of topics, such as on ontology quality, verbalisations, or language features, and may be theoretical or software development-oriented. In terms of papers, the most impressive result was OntoPartS that resulted in an ESWC2012 paper with the two postgraduate students [1], but also quite some other useful results have come out of it over the past 7 years that I’m teaching it in one form or another. This year’s top project in terms of understanding the theory, creativity to do something with it that hasn’t been done before, and working software using Semantic Web technologies was the “OWL Classifier” by Aashiq Parker, Brian Mc George, and Muhummad Patel.

The OWL classifier classifies an OWL ontology in any of its ‘species’, which can be any of the 8 specified in the standard, i.e., the 3 OWL 1 ones and the 5 OWL 2 ones. It also gives information on the DL ‘alphabet soup’—which axioms use which language feature with which letter, and an explanation of the letters—and reports on which axioms are the ones that violate a particular species. An example is shown in the following screenshot, with an exercise ontology on phone points:

phonePoints

The students’ motivation to develop it was because they had to learn about DLs and the OWL species, but Protégé 4.x and 5.x don’t tell you the species and the interfaces have only a basic, generic, explanation for the DL expressivity. I concur. And is has gotten worse with Protégé 5.0: if an ontology is outside OWL 2 DL, it still says the ‘old’ DL expressivity plus an easy-to-overlook tiny red triangle in the top-right corner once the reasoner was invoked (using Hermit 1.3.8) or a cryptic “internal reasoner error” message (Pellet), whereas with Protégé 4.x you at least got a pop-up box complaining about the ‘non-simple role…’ issues. Compare that with the neat feedback like this:

t15and16

It is also very ‘sensitive’—more so than one would be with Protégé alone. Any remote ontology imports have to be available at the location specified with the IRI. Violations due to wrong datatype usage is a known issue with the OWL Reasoner Evaluation set of ontologies, and which we’ve bumped into with the TDD testing as well. The tool doesn’t accept the invalid ones (wrong datatypes—one can select any XML data type in Protégé, but the OWL standard doesn’t support them all). In addition, a language such as OWL 2 QL has further restrictions on types of datatypes. (It is also not trivial to figure out manually whether some ontology is suitable for OBDA or not.) So I tried one from the Ontop website’s examples, presumably in OWL 2 QL:

fishdelish

Strictly speaking, it isn’t in OWL 2 QL! The OWL 2 QL profile does have xsd:integer as datatype [2], not xsd:int, as, and I quote the standard, “the intersection of the value spaces of any set of these datatypes [including xsd:integer but not xsd:int, mk] is either empty or infinite, which is necessary to obtain the desired computational properties”. [UPDATE 24-6, thanks to Martin Rezk:] The main toolset for OWL 2 QL, Ontop, actually does support xsd:int and a few other datatypes beyond the standard (e.g.: also float and boolean). There is similar syntax fun to be had with the pizza ontology: the original one is indeed in OWL DL, but if you open the file in Protégé 5 and save it, it is not in OWL DL anymore but in OWL 2 DL, for the save operation snuck in an owl#NamedIndividual. Click on the thumbnails below to see the before-and-after in the OWL classifier. This is not an increase in expressiveness—both are in SHOIN—just syntax and tooling.

pizzaOldpizzaP5

 

 

 

 

 

The OWL Classifier can thus classify both OWL 1 and OWL 2 ontologies, which it does through a careful orchestration of two OWL APIs: v1.4.3 was the last one to support OWL 1 species checking, whereas for the OWL 2 ontologies, the latest version is used (v4.2.3). The jar file and the source code are freely available on github for anyone to use and to take further. Turning it into a Protégé plugin very likely will make at least next year’s ontology engineering students happy. Comments, questions, and suggestion are welcome!

 

References

[1] Keet, C.M., Fernandez-Reyes, F.C., Morales-Gonzalez, A. Representing mereotopological relations in OWL ontologies with OntoPartS. 9th Extended Semantic Web Conference (ESWC’12), Simperl et al. (eds.), 27-31 May 2012, Heraklion, Crete, Greece. Springer, LNCS 7295, 240-254.

[2] Boris Motik, Bernardo Cuenca Grau, Ian Horrocks, Zhe Wu, Achille Fokoue, Carsten Lutz, eds. OWL 2 Web Ontology Language: Profiles. W3C Recommendation, 11 December 2012 (2nd ed.).

Reflections on ESWC 2016: where are the ontologies papers?

Although I did make notes of the presentations I attended at the 13th Extended Semantic Web Conference a fortnight ago, with the best intentions to write a conference report, it’s going to be an opinion piece of some sort, on ontology engineering, or, more precisely: the lack thereof at ESWC2016.

That there isn’t much on ontology research at ISWC over the past several years, I already knew, both from looking at the accepted papers and the grapevine, but ESWC was still known to be welcoming to ontology engineering. ESWC 2016, however, had only one “vocabularies, schemas, and ontologies” [yes, in that order] session (and one on reasoning), with only the paper by Agnieszka and me solidly in the ‘ontologies’/ontology engineering bracket, with new theory, a tool implementing it, experiments, and a methodology sketch [1]. The other two papers were more on using ontologies, in annotating documents and in question answering. My initial thought was: “ah, hm, bummer, so ESWC also shifted focus”. There also were few ontologists at the conference, so I wondered whether the others moved on to a non-LD related field, alike I did shift focus a bit thanks/due to funded projects in adjacent fields (I did try to get funds for ontology engineering projects, though).

To my surprise, however, it appeared that a whopping 27 papers had been submitted to the “vocabularies, schemas, and ontologies” track. It was just that only three had made it through the review process. Asking around a bit, the comments were sort of like when I was co-chair of the track for ESWC 2014: ‘meh’, not research (e.g., just developing a domain ontology), minor delta, need/relevance unclear. And looking again at my reviews for 2015 and 2016, in addition to those reasons: failing to consider relevant related work, or a lacking a comparison with related work (needed to demonstrate improvement), and/or some issues with the theory (formal stuff). So, are we to blame and ‘suicidal’ or become complacent and lazy? It’s not like the main problems have been solved and developing an ontology has become a piece of cake now, compared to, say, 10 years ago. And while it is somewhat tempting to do some paper/presentation bashing, I won’t go into specifics, other than that at two presentations I attended, where they did show a section of an ontology, there was even the novice error of confusing classes with instances.

Anyway, there used to be more ontology papers in earlier ESWCs. To check that subjective impression, I did a quick-and-dirty check of the previous 12 editions as well, of which 11 had named sessions. Here’s the overview of the number of ontology papers over the years (minus the first one as it did not have named sections):

ontoPap

The aggregates are a bit ‘dirty’ as the 2010 increase grouped ontologies together with reasoning (if done for 2016, we’d have made it to 6), as was 2007 a bit flexible on that, and 2015 had 3 ontologies papers + 3 ontology matching & summarization, so stretching it a bit in that direction, as was the case in 2013. The number of papers in 2006 is indeed that much, with sessions on ontology engineering (3 papers), ontology evaluation (3), ontology alignment (5), ontology evolution (3), and ontology learning (3). So, there is indeed a somewhat downward trend.

Admitted, ‘ontologies’ is over the initial hype and it probably now requires more preparation and work to come up with something sufficiently new than it was 10 years ago. Looking at the proceedings of 5 years ago rather, the 7 ontologies papers were definitely not trivial, and I still remember the one on removing redundancies [2], the introduction of two new matching evaluation measures and comparison with other methods [3], and automatically detecting related ontology versions [4]. Five ontology papers then had new theory and some experiments, and two had extensive experiments [5,6]. 2012 had 6 ontologies papers, some interesting, but something like the ‘SKOS survey’ is a dated thing (nice, but ESWC-level?) and ISOcat isn’t great (but I’m biased here, as I don’t like it that noun classes aren’t in there, and it is hard to access).

Now what? Work more/harder on ontology engineering if you don’t want to have it vanish from ESWC. That’s easier said than done, though. But I suppose it’s fair to say to not discard the ESWC venue as being ‘not an ontology venue anymore’, and instead use these six months to the deadline to work hard enough. Yet, who knows, maybe we are harder to ourselves when reviewing papers compared to other tracks. Either way, it is something to reflect upon, as an 11% acceptance rate for a track, like this year, isn’t great. ESWC16 in general had good papers and interesting discussions. While the parties don’t seem to be as big as they used to be, there sure is a good time to be had as well.

 

p.s.: Cretan village, where I stayed for the first time, was good and had a nice short walk on the beach to the conference hotel, but beware that the mosquitos absent from Knossos Hotel all flock to that place.

 

References

[1] Keet, C.M., Lawrynowicz, A. Test-Driven Development of Ontologies. In: Proceedings of the 13th Extended Semantic Web Conference (ESWC’16). Springer LNCS 9678, 642-657. 29 May – 2 June, 2016, Crete, Greece.

[2] Stephan Grimm and Jens Wissmann. Elimination of redundancy in ontologies. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 260-274.

[3] Xing Niu, Haofen Wang, GangWu, Guilin Qi, and Yong Yu. Evaluating the Stability and Credibility of Ontology Matching Methods. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 275-289.

[4] Carlo Alocca. Automatic Identification of Ontology Versions Using Machine Learning Techniques. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 275-289.

[5] Keet, C.M. The use of foundational ontologies in ontology development: an empirical assessment. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 321-335.

[6] Wei Hu, Jianfeng Chen, Hang Zhang, and Yuzhong Qu. How Matchable Are Four Thousand Ontologies on the Semantic Web. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 290-304.

Preliminary promising results on a data-driven spellchecker for isiZulu

While developing a spellchecker for isiZulu is not new, the one for Open Office v3 doesn’t work with the more up-to-date versions of Open Office, it was of limited quality, the other papers on isiZulu spellcheckers have no tools for it, and the techniques used were tailored to the language. The demand for it has not decreased, however. Last year’s honours student I co-supervised, Balone Ndaba, set out to address this using n-grams and several corpora, so if this were to work, then this essentially language-independent solution may be ported to other Bantu languages. The data-driven approach worked reasonably well (up to 89% accuracy), so it all ended up as a paper entitled “The effects of a corpus on isiZulu spellcheckers based on n-grams” [1], co-authored with Balone Ndaba, Hussein Suleman, and Langa Khumalo. It has been accepted at the IST Africa’16 conference that will take place in less than two weeks in Durban, South Africa. The material can be downloaded from Balone’s honours project page.

In a nutshell, well less than one page cf. the paper’s 9 pages, the paper describes the design, implementation, and experimental results of a statistics-based spellchecker. One option in a data-driven approach is to make long wordlists to look up the word to be spellchecked, indeed, but this is not doable due to the agglutination in the language (i.e., way too many options), and there are limited curated resources anyhow. Instead, we let it ‘learn’ using a statistical language model using n-grams created from a corpus. For instance, take the word yebo ‘yes’, then its bigrams are ye, eb, and bo, its trigrams are yeb and ebo, and quadrigram (i.e., 4 characters) yebo. Feeding the algorithm a lot of text generates a lot of n-grams, some of which occur much more often than others, whereas the very rare ones were probably typos or a loan word in the original text. The hope is then that when it is fed a word that it has not been trained on, it can recognize that it is (very probably) still a valid word in the language, as the sequence of the characters in the string are deemed common enough, or, conversely, that it is very likely to be misspelled.

It was not clear upfront which of the variables will affect the quality most, and which combination leads to the best spellchecker performance. This could be the threshold for when to include the n-gram, how big the n-gram should be, and whether the training corpus had any effect. So, all this was tested (see paper for details).

Trigrams worked better than quadrigrams, and a threshold of 0.003 worked better than the other thresholds. This is the ‘easy’ part, in a way. What complicated matters were the corpora, where one was much better to learn from or test on than the other. This is summarised in the table below. UC stands for the Ukwabelana corpus [2] that consists of the bible and a few copyright-expired novels, S. INC for a sample of the isiZulu National Corpus that is being developed at the University of KwaZulu-Natal [3], and INC is a small corpus of news items from the Isolezwe and news24 in isiZulu news websites (extended from the earlier playing with some of those articles). The INC and NIC work well on each other, but not at all with UC, and if trained with UC then only mediocre on the more recent texts of NIC and INC. So, the training corpus has a rather large effect on the spellchecker’s performance.

 

Lexical recall with n-grams: 10-fold cross-validation (i.e., with itself) and tested against the other corpora (t. = test).

10-fold Train UC Train NIC Train S. INC
3-gr. 4-gr. t. NIC t. INC t. INC t. UC t. UC t. NIC
UC 85 80 30 41
S. INC 66 63 54 89
NIC 86 79 70 89

 

We did look into the corpora a little more, but there’s more to analyse. Notably, the number of unique words in the corpus seems to matter more than its size, and the datedness and quality of the text suggest room for additional evaluation. UC contains some archaic terms that wouldn’t be used in modern-day isiZulu, and there have been some errors introduced in the words, assumed to be due to OCR (words with three successive vowels are a no-no in isiZulu, verbs missing the final vowel).

Overall, though, a lexical recall and an accuracy of 89% is quite alright for a first-pass, meriting resources for fine-tuning for isiZulu and it being promising as approach for related languages.

Balone will present the paper at IST Africa, and Hussein and Langa will also participate, so you can ask them more details in person at the conference. (I’ll be holding a final training for the team representing UCT in the ACM ICPC World Finals and pack my bags to join them as coach in Phuket in Thailand.)

 

References

[1] Ndaba, B., Suleman, H., Keet, C.M., Khumalo, L. The Effects of a Corpus on isiZulu Spellcheckers based on N-grams. IST-Africa 2016. May 11-13, 2016, Durban, South Africa.

[2] S. Spiegler, A. van der Spuy, P. Flach. Ukwabelana – An Open-source Morphological Zulu Corpus. Proceedings of the 23rd International Conference on Computational Linguistics (COLING, 2010). ACL. 1020-1028.

[3] Khumalo. Advances in developing corpora in African languages. Kuwala, 2015, 1(2): 21-30.

The TDDonto tool to try out TDD for ontology authoring

Last month I wrote about Test-Driven Development for ontologies, which is described in more detail in the ESWC’16 paper I co-authored with Agnieszka Lawrynowicz [1]. That paper does not describe much about the actual tool implementing the tests, TDDonto, although we have it and used it for the performance evaluation. Some more detail on its design and more experimental results are described in the paper “The TDDonto Tool for Test-Driven Development of DL Knowledge Bases” [2] that has just been published in the proceedings of the 29th International Workshop on Description Logics, which will take place next weekend in Cape Town (22-25 April 2016).

What we couldn’t include there in [2] is multiple screenshots to show how it works, but a blog is a fine medium for that, so I’ll illustrate the tool with some examples in the remainder of the post. It’s an alpha version that works. No usability and HCI evaluations have been done, but at least it’s a Protégé plugin rather than command line🙂.

First, you need to download the plugin from Agnieszka’s ARISTOTELES project page and place the jar file in the plugins folder of Protégé 5.0. You can then go to the Protégé menu bar, select Windows – Views – Evaluation views – TDDOnto, and place it somewhere on the screen and start using it. For the examples here, I used the African Wildlife Ontology tutorial ontology (AWO v1) from my ontology engineering course.

Make sure to have selected an automated reasoner, and classify your ontology. Now, type a new test in the “New test” field at the top, e.g. carnivore DisjointWith: herbivore, click “Add test”, select the checkbox of the test to execute, and click the “Execute test”: the status will be returned, as shown in the screenshot below. In this case, the “OK” says that the disjointness is already asserted or entailed in the ontology.

cdisjh

Now let’s do a TDD test that is going to fail (you won’t know upfront, of course); e.g., testing whether impalas are herbivores:

impalaFail

The TDD test failed because the subsumption is neither asserted nor entailed in the ontology. One can then click “add to ontology”, which updates the ontology:

impalaAdd

Note that the reasoner has to be run again after a change in the ontology.

Lets do two more: testing whether lion is a carnivore and that flower is a plan part. The output of the tests is as follows:

lionflower

It returns “OK” for the lion, because it is entailed in the ontology: a carnivore is an entity that eats only animals or parts thereof, and lions eat only herbivore and eats some impala (which are animals). The other one, Flower SubClassOf: PlantParts fails as “undefined”, because Flower is not in the ontology.

Ontologies do not have only subsumption and disjointness axioms, so let’s assume that impalas eat leaves and we want check whether that is in the ontology, as well as whether lions eat animals:

lionImpalaEats

The former failed because there are no properties for the impala in the AWO v1, the latter passed, because a lion eats impala, and impala is an animal. Or: the TDDOnto tool indeed behaves as expected.

Currently, only a subset of all the specified tests have been implemented, due to some limitations of existing tools, but we’re working on implementing those as well.

If you have any feedback on TDDOnto, please don’t hesitate to tell us. I hope to be seeing you later in the week at DL’16, where I’ll be presenting the paper on Sunday afternoon (24th) and I also can give a live demo any time during the workshop (or afterwards, if you stay for KR’16).

 

References

[1] Keet, C.M., Lawrynowicz, A. Test-Driven Development of Ontologies. 13th Extended Semantic Web Conference (ESWC’16). Springer LNCS. 29 May – 2 June, 2016, Crete, Greece. (in print)

[2] Lawrynowicz, A., Keet, C.M. The TDDonto Tool for Test-Driven Development of DL Knowledge bases. 29th International Workshop on Description Logics (DL’16). April 22-25, Cape Town, South Africa. CEUR WS vol. 1577.