Although I did make notes of the presentations I attended at the 13th Extended Semantic Web Conference a fortnight ago, with the best intentions to write a conference report, it’s going to be an opinion piece of some sort, on ontology engineering, or, more precisely: the lack thereof at ESWC2016.
That there isn’t much on ontology research at ISWC over the past several years, I already knew, both from looking at the accepted papers and the grapevine, but ESWC was still known to be welcoming to ontology engineering. ESWC 2016, however, had only one “vocabularies, schemas, and ontologies” [yes, in that order] session (and one on reasoning), with only the paper by Agnieszka and me solidly in the ‘ontologies’/ontology engineering bracket, with new theory, a tool implementing it, experiments, and a methodology sketch [1]. The other two papers were more on using ontologies, in annotating documents and in question answering. My initial thought was: “ah, hm, bummer, so ESWC also shifted focus”. There also were few ontologists at the conference, so I wondered whether the others moved on to a non-LD related field, alike I did shift focus a bit thanks/due to funded projects in adjacent fields (I did try to get funds for ontology engineering projects, though).
To my surprise, however, it appeared that a whopping 27 papers had been submitted to the “vocabularies, schemas, and ontologies” track. It was just that only three had made it through the review process. Asking around a bit, the comments were sort of like when I was co-chair of the track for ESWC 2014: ‘meh’, not research (e.g., just developing a domain ontology), minor delta, need/relevance unclear. And looking again at my reviews for 2015 and 2016, in addition to those reasons: failing to consider relevant related work, or a lacking a comparison with related work (needed to demonstrate improvement), and/or some issues with the theory (formal stuff). So, are we to blame and ‘suicidal’ or become complacent and lazy? It’s not like the main problems have been solved and developing an ontology has become a piece of cake now, compared to, say, 10 years ago. And while it is somewhat tempting to do some paper/presentation bashing, I won’t go into specifics, other than that at two presentations I attended, where they did show a section of an ontology, there was even the novice error of confusing classes with instances.
Anyway, there used to be more ontology papers in earlier ESWCs. To check that subjective impression, I did a quick-and-dirty check of the previous 12 editions as well, of which 11 had named sessions. Here’s the overview of the number of ontology papers over the years (minus the first one as it did not have named sections):
The aggregates are a bit ‘dirty’ as the 2010 increase grouped ontologies together with reasoning (if done for 2016, we’d have made it to 6), as was 2007 a bit flexible on that, and 2015 had 3 ontologies papers + 3 ontology matching & summarization, so stretching it a bit in that direction, as was the case in 2013. The number of papers in 2006 is indeed that much, with sessions on ontology engineering (3 papers), ontology evaluation (3), ontology alignment (5), ontology evolution (3), and ontology learning (3). So, there is indeed a somewhat downward trend.
Admitted, ‘ontologies’ is over the initial hype and it probably now requires more preparation and work to come up with something sufficiently new than it was 10 years ago. Looking at the proceedings of 5 years ago rather, the 7 ontologies papers were definitely not trivial, and I still remember the one on removing redundancies [2], the introduction of two new matching evaluation measures and comparison with other methods [3], and automatically detecting related ontology versions [4]. Five ontology papers then had new theory and some experiments, and two had extensive experiments [5,6]. 2012 had 6 ontologies papers, some interesting, but something like the ‘SKOS survey’ is a dated thing (nice, but ESWC-level?) and ISOcat isn’t great (but I’m biased here, as I don’t like it that noun classes aren’t in there, and it is hard to access).
Now what? Work more/harder on ontology engineering if you don’t want to have it vanish from ESWC. That’s easier said than done, though. But I suppose it’s fair to say to not discard the ESWC venue as being ‘not an ontology venue anymore’, and instead use these six months to the deadline to work hard enough. Yet, who knows, maybe we are harder to ourselves when reviewing papers compared to other tracks. Either way, it is something to reflect upon, as an 11% acceptance rate for a track, like this year, isn’t great. ESWC16 in general had good papers and interesting discussions. While the parties don’t seem to be as big as they used to be, there sure is a good time to be had as well.
p.s.: Cretan village, where I stayed for the first time, was good and had a nice short walk on the beach to the conference hotel, but beware that the mosquitos absent from Knossos Hotel all flock to that place.
References
[1] Keet, C.M., Lawrynowicz, A. Test-Driven Development of Ontologies. In: Proceedings of the 13th Extended Semantic Web Conference (ESWC’16). Springer LNCS 9678, 642-657. 29 May – 2 June, 2016, Crete, Greece.
[2] Stephan Grimm and Jens Wissmann. Elimination of redundancy in ontologies. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 260-274.
[3] Xing Niu, Haofen Wang, GangWu, Guilin Qi, and Yong Yu. Evaluating the Stability and Credibility of Ontology Matching Methods. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 275-289.
[4] Carlo Alocca. Automatic Identification of Ontology Versions Using Machine Learning Techniques. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 275-289.
[5] Keet, C.M. The use of foundational ontologies in ontology development: an empirical assessment. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 321-335.
[6] Wei Hu, Jianfeng Chen, Hang Zhang, and Yuzhong Qu. How Matchable Are Four Thousand Ontologies on the Semantic Web. In: Proceedings of the 8th Extended Semantic Web Conference (ESWC’11). Heraklion, Crete, Greece, 29 May – 2 June 2011. Springer LNCS 6643, 290-304.
Keet blog 