Moral responsibility in the Computing era (SEP entry)

The Stanford Encyclopedia of Philosophy intermittently has new entries that have to do with computing, like on the philosophy of computer science about which I blogged before, ethics of, among others, Internet research, and now Computing and Moral Responsibility by Merel Noorman [1]. The remainder of this post is about the latter entry that was added on July 18, 2012. Overall, the entry is fine, but I had expected more from it, which may well be due to that the ‘computing and moral responsibility’ topic needs some more work to mature and then maybe will give me the answers I was hoping to find already.

Computing—be this the hardware, firmware, software, or IT themes—interferes with the general notion of moral responsibility, hence, affects every ICT user at least to some extent, and the computer scientists, programmers etc who develop the artifacts may themselves be morally responsible, and perhaps even the produced artifacts, too. This area of philosophical inquiry deals with questions such as “Who is accountable when electronic records are lost or when they contain errors? To what extent and for what period of time are developers of computer technologies accountable for untoward consequences of their products? And as computer technologies become more complex and behave increasingly autonomous can or should humans still be held responsible for the behavior of these technologies?”. To this end, the entry has three main sections, covering moral responsibility, the question whether computers can be more agents, and the notion of (and the need for) rethinking the concept of moral responsibility.

First, it reiterates the general stuff about moral responsibility without the computing dimension, like that it has to do with the actions of humans and its consequences: “generally speaking, a person or group of people is morally responsible when their voluntary actions have morally significant outcomes that would make it appropriate to praise or blame them”, where the SEP entry dwells primarily on the blaming. Philosophers roughly agree that the following three conditions have to be met regarding being morally responsible (copied from the entry):

 1. There should be a causal connection between the person and the outcome of actions. A person is usually only held responsible if she had some control over the outcome of events.

2. The subject has to have knowledge of and be able to consider the possible consequences of her actions. We tend to excuse someone from blame if they could not have known that their actions would lead to a harmful event.

3. The subject has to be able to freely choose to act in certain way. That is, it does not make sense to hold someone responsible for a harmful event if her actions were completely determined by outside forces.

But how are these to be applied? Few case examples of the difficulty to apply it in praxis are given; e.g., the malfunctioning Therac-25 radiation machine (three people died caused by overdoses of radiation, primarily due to issues regarding the software), the Aegis software system that misidentified an Iranian civilian aircraft in 1988 as an attacking military aircraft and the US military decided to shoot it down (contrary to two other systems that had identified it correctly) and having killed all 209 passengers on board, the software to manage remote-controlled drones, and perhaps even the ‘filter bubble’. Who is to blame, if at all? These examples, and others I can easily think of, are vastly different scenarios, but they have not been identified, categorized, and treated as such. But if we do, then perhaps at least some general patters can emerge and even rules regarding moral responsibility in the context of computing. Here’s my initial list of different kinds of cases:

1. The hardware/software was intended for purpose X but is used for purpose Y, with X not being inherently harmful, whereas Y is; e.g., the technology of an internet filter for preventing kids to access adult-material sites is used to make a blacklist of sites that do not support government policy and subsequently the users vote for harmful policies, or, as simpler one: using mobile phones to detonate bombs.
2. The hardware/software is designed for malicious intents; ranging from so-called cyber warfare (e.g., certain computer viruses, denial-of-service attacks) to computing for physical war to developing and using shadow-accounting software for tax evasion.
3. The hardware/software has errors (‘bugs’):
   1. The specification was wrong with respect to the intentionally understated or mis-formulated intentions, and the error is simply a knock-on effect;
   2. The specification was correct, but a part of the subject domain is intentionally wrongly represented (e.g., the decision tree may be correctly implemented given the wrong representation of the subject domain);
   3. The specification was correct, the subject domain represented correctly, but there’s a conceptual error in the algorithm (e.g., the decision tree was built wrongly);
   4. The program code is scruffy and doesn’t do what the algorithm says it is supposed to do;
4. The software is correct, but has the rules implemented as alethic or hard constraints versus deontic or soft constraints (not being allowed to manually override a default rule), effectively replacing human-bureaucrats with software-bureaucrats;
5. Bad interface design to make the software difficult to use, resulting in wrong use and/or overlooking essential features;
6. No or insufficient training of the users how to use the hardware/software;
7. Insufficient maintenance of the IT system that causes the system to malfunction;
8. Overconfidence in the reliability of the hardware/software;
   1. The correctness of the software, pretending that it always gives the right answer when it may not; e.g., assuming that the pattern matching algorithm for fingerprint matching is 100% reliable when it is actually only, say, 85%;
   2. Assuming (extreme) high availability, when no extreme high availability system is in place; e.g., relying solely on electronic health records in a remote area whereas the system may be down right when it is crucial to access it in the hospital information system.
9. Overconfidence in the information provided by or through the software; this is partially alike 8-i, or the first example in item 1, and, e.g., willfully believing that everything published on the Internet is true despite the so-called ‘information warfare’ regarding the spreading of disinformation.

Where the moral responsibility lies can be vastly different depending on the case, and even within the case, it may require further analysis. For instance (and my opinions follow, not what is written in the SEP entry), regarding maintenance: a database for the electronic health records outgrows it prospective size or the new version of the RDBMS actually requires more hardware resources than the server has, with as consequence that querying the database becomes too slow in a critical case (say, to check whether patient A is allergic to medicine B that needs to be administered immediately): perhaps the system designer should have foreseen this, or perhaps management didn’t sign off on a purchase for a new server, but I think that the answer to the question of where the moral responsibility lies can be found. For mission-critical software, formal methods can be used, and if, as engineer, you didn’t and something goes wrong, then you are to blame. One cannot be held responsible for a misunderstanding, but when the domain expert says X of the subject domain and you have some political conviction that you prefer Y and build that into the software and that, then, results in something harmful, then you can be held morally responsible (item 3-ii). On human vs. software bureaucrat (item 4), the blame can be narrowed down when things go wrong: was it the engineer who didn’t bother with the possibility of exceptions, was there a/no technological solution for it at the time of development (and knowingly ignore it), or was it the client who willfully was happy avoiding such pesky individual exceptions to the rule? Or, another example, as the SEP entry questions (an example of item 1): can one hold the mobile phone companies responsible for having designed cell phones that also can be used to detonate bombs? In my opinion: no. Just in case you want to look for guidance, or even answers, in the SEP entry regarding such kind of questions and/or cases: don’t bother, there are none.

More generally, the SEP entry mentions two problems for attributing blame and responsibility: the so-called problem of ‘many hands’ and the problem with physical and temporal distance. The former concerns the issue that there are many people developing the software, training the users, etc., and it is difficult to identify the individual, or even the group of individuals, who ultimately did the thing that caused the harmful effect. It is true that this is a problem, and especially when the computing hardware or software is complex and developed by hundreds or even thousands of people. The latter concerns the problem that the distance can blur the causal connection between action and event, which “can reduce the sense of responsibility”. But, in my opinion, just because someone doesn’t reflect much on her actions and may be willfully narrow-minded to (not) accept that, yes, indeed, those people celebrating a wedding in a tent in far-away Afghanistan are (well, were) humans, too, does not absolve one from the responsibility—neither the hardware developer, nor the software developer, nor the one who pushed the button—as distance does not reduce responsibility. One could argue it is only the one who pushed the button who made the judgment error, but the drone/fighter jet/etc. computer hardware and software are made for harmful purposes in the first place. Its purpose is to do harm to other entities—be this bombing humans or, say, a water purification plant such that the residents have no clean water—and all developers involved very well know this; hence, one is morally responsible from day one that one is involved in its development and/or use.

I’ll skip the entry’s section on computers as agents (AI software, robots), and whether they can be held morally responsible, just responsible, or merely accountable, or none of them, except for the final remark of that section, credited to Bruno Latour (emphasis mine):

[Latour] suggests that in all forms of human action there are three forms of agency at work: 1) the agency of the human performing the action; 2) the agency of the designer who helped shaped the mediating role of the artifacts and 3) the artifact mediating human action. The agency of artifacts is inextricably linked to the agency of its designers and users, but it cannot be reduced to either of them. For him, then, a subject that acts or makes moral decisions is a composite of human and technological components. Moral agency is not merely located in a human being, but in a complex blend of humans and technologies.

Given the issues with assigning moral responsibility with respect to computing, some philosophers ponder about doing away with it, and replace it with a better framework. This is the topic of the third section of the SEP entry, which relies substantially on Gotterbarn’s work on it. He notes that computing is ethically not a neutral practice, and that the “design and use of technological artifacts is a moral activity” (because the choice of one design and implementation over another does have consequences). Moreover, and more interesting, is that, according to the SEP entry, he introduces the notions of negative responsibility and positive responsibility. The former “places the focus on that which exempts one from blame and liability”, whereas the latter “focuses on what ought to be done”, and entails to “strive to minimize foreseeable undesirable events”. Computing professionals, according to Gotterbarn, should adopt the notion of positive responsibility. Later on in the section, there’s a clue that there’s some way to go before achieving that. Accepting accountability is more rudimentary than taking moral responsibility, or at least a first step toward moral responsibility. Nissenbaum (paraphrased in the SEP entry) has identified four barriers to accountability in society (at least back in 1997 when she wrote it): the above-mentioned problem of many hands, the acceptance of ‘bugs’ as an inherent element of large software applications, using the computer as scapegoat, and claiming ownership without accepting liability (read any software license if you doubt the latter). Perhaps that needs to be addressed before going on to the moral responsibility, or one reinforces the other? Dijkstra vents his irritation in one of his writings about software ‘bugs’—the cute euphemism dating back to the ‘50s—and instead proposes to use one of its correct terms: they are errors. Perhaps users should not be lenient with errors, which might compel developers to deliver a better/error-free product, and/or we have to instill in the students more about the positive responsibility and reduce their tolerance for errors. And/or what about re-writing the license agreements a bit, like accepting responsibility provided it is used in one of the prescribed and tested ways? We already had that when I was working for Eurologic more than 10 years ago: the storage enclosure was supposed to work in certain ways and was tested in a variety of configurations, and that we signed off on for our customers. If it was faulty in one of the tested system configurations after all, then that was our problem, and we’d incur the associated costs to fix it. To some extent, that was also with our suppliers. Indeed, for software, that is slightly harder, but one could include in the license something along the line of ‘X works on a clean machine and when common other packages w, y, and z are installed, but we can’t guarantee it when you’ve downloaded weird stuff from the Internet’; not perfect, but it is a step in the right direction. Anyone has better ideas?

Last, the closing sentence is a useful observation, effectively stretching the standard  notion of moral responsibility thanks to computing (emphasis added): “[it] is, thus, not only about how the actions of a person or a group of people affect others in a morally significant way; it is also about how their actions are shaped by technology.”. But, as said, the details are yet to be thought through and worked out in some detail and general guidelines that can be applied.

References

[1] Merel Noorman. (forthcoming in 2012). Computing and Moral Responsibility. Stanford Encyclopedia of Philosophy (Fall 2012 Edition), Zalta, E.N. (ed.).  Stable URL: http://plato.stanford.edu/archives/fall2012/entries/computing-responsibility/.

Advertisement

A new version of ONSET and more technical details are now available

After the first release of the foundational ONtology Selection and Explanation Tool ONSET half a year ago, we—Zubeida Khan and I—continued its development by adding SUMO, conducting a user evaluation, and we wrote a paper about it, which was recently accepted [1] at the 18th International Conference on Knowledge Engineering and Knowledge Management (EKAW’12).

There are theoretical and practical reasons why using a foundational ontology improves the quality and interoperability of the domain ontology, be this by means of reusing DOLCE, BFO, GFO, SUMO, YAMATO, or another one, in part or in whole (see, e.g., [2,3] for some motivations). But as a domain ontology developer, and those who are potentially interested in using a foundational ontology in particular, do ask: which one of them would be best to use for the task at hand? That is not an easy question to answer, and hitherto required from a developer to pore over all the documentation, weighing the pros and cons for the scenario, make an informed decision, know exactly why, and be able to communicate that. This bottleneck has been solved with the ONSET tool. Or, at least: we claim it does, and the user evaluation supports this claim.

In short, ONSET, the foundational ONtology Selection and Explanation Tool helps the domain ontology developer in this task. Upon answering one or more questions and, optionally, adding any scaling to indicate some criteria are more important to you than others, it computes the most suitable foundational ontology for that scenario and explains why this is so, including reporting any conflicting answers (if applicable). The questions themselves are divided into five different categories—Ontology, representation language, software engineering properties, applications, and subject domain—and there are “explain” buttons to clarify terms that may not be immediately clear to the domain ontology developer. (There are a few screenshots at the end of this post.)

Behind the scenes is a detailed comparison of the features of DOLCE, BFO, GFO, and SUMO, and an efficient algorithm. The latter and the main interesting aspects of the former are included in the paper; the complete set of criteria is available in a file on the ONSET webpage. You can play with ONSET using your real or a fictitious ontology development scenario after downloading the jar file. If you don’t have a scenario and can’t come up with one: try one of the scenarios we used for the user evaluation (also online). The user evaluation consisted of 5 scenarios/problems that the 18 participants had to solve, half of them used ONSET and half of them did not. On average, the ‘accuracy’ (computed from selecting the appropriate foundatinal ontology and explaining why) was 3 times higher for those who used ONSET compared to those who did not. The ONSET users also did it slightly faster.

Thus, ONSET greatly facilitates in selecting a foundational ontology. However, I concede that from the Ontology (philosophy) viewpoint, the real research component is, perhaps, only beginning. Among others, what is the real effect of the differences between those foundational ontolgoies for ontology development, if any? Is one category of criteria, or individual criterion, always deemed more important than others? Is there one or more ‘typical’ combination of criteria, and if so, is there a single particular foundational ontology suitable, and if not, where/why are the current ones insufficient? In the case of conflicts, which criteria do they typically involve? ONSET clearly can be a useful aid investigating these questions, but answering them is left to future works. Either way, ONSET contributes to taking a scientific approach to comparing and using a foundational ontology in ontology development, and provides the hard arguments why.

We’d be happy to hear your feedback on ONSET, be this on the tool itself or when you have used it for a domain ontology development project. Also, the tool is very easy to extend thanks to the way it is programmed, so if you have your own pet foundational ontology that is not yet included in the tool, you may like to provide us with the values for the criteria so that we can include it.

Here are a few screenshots: of the start page, questions and an explanation, other questions, and the result (of a fictitious example):

Startpage of ONSET, where you select inclusion of additional questions that don’t make any difference right now, and where you can apply scaling to the five categories.

Section of the questions about ontological commitments and a pop-up screen once the related “Explain” button is clicked.

Another tab with questions. In this case, the user selected “yes” to modularity, upon which the tool expanded the question so that a way of modularisation can be selected.

Section of the results tab, after having clicked “calculate results” (in this case, of a fictitious scenario). Conflicting results, if any, will be shown here as well, and upon scrolling down, relevant literature is shown.

References

[1] Khan, Z., Keet, C.M. ONSET: Automated Foundational Ontology Selection and Explanation. 18th International Conference on Knowledge Engineering and Knowledge Management (EKAW’12). Oct 8-12, Galway, Ireland. Springer, LNAI, 15p. (accepted)

[2] Keet, C.M. The use of foundational ontologies in ontology development: an empirical assessment. 8th Extended Semantic Web Conference (ESWC’11), G. Antoniou et al (Eds.), Heraklion, Crete, Greece, 29 May-2 June, 2011. Springer, Lecture Notes in Computer Science LNCS 6643, 321-335.

[3] Borgo, S., Lesmo, L. The attractiveness of foundational ontologies in industry. In: Proc. of FOMI’08, Amsterdam, The Netherlands, IOS Press (2008), 1-9.

Theory of Computation topics mostly as elective in the planned ACM/IEEE CS2013 curriculum

As part of the intermittent research I have taken up on teaching Theory of Computation, I have been reading a few papers on curriculum development for Computer Science. As it happens, there is currently a draft—the so-called ‘Strawman draft’—up for discussion until July 15 in this forum. Upon community input, that draft may be updated and it will become the new version of ACM curriculum guidelines for CS (and its variants) in 2013; see [1] for a 2-page context.

One thing that may surprise the CS researcher and professor who is in the system since a while, is that since the Computing Curricula CC2001 version, and carried over in the Strawman draft, most of automata is elective, only a little complexity is core, and, e.g., Turing machines is elective—which was unlike the seminal first version that described computing as a discipline and principal curriculum topics in 1989 [2] that did have formal languages, automata, Turing machines, complexity and computability in the core. More precisely, the CS2013 proposal is as follows (layout edited):

AL/Basic Automata Computability and Complexity

[3 Core-Tier1 hours, 3 Core-Tier2 hours]

Topics:

[Core-Tier1] Finite-state machines, Regular expressions, The halting problem

[Core-Tier2] Context-free grammars (cross-reference PL/Syntax Analysis), P vs NP (tractable and intractable problems), Definition of P, NP, and NP-complete, Exemplary NP-complete problems (e.g., SAT, Knapsack)

AL/Advanced Computational Complexity

[Elective]

Topics: Review definitions of the classes P and NP; introduce EXP, NP-completeness (Cook’s theorem), Classic NP-complete problems, Reduction Techniques,

AL/Advanced Automata Theory and Computability

[Elective]

Topics: Sets and languages, Regular languages (Review of deterministic finite automata (DFAs), Nondeterministic finite automata (NFAs), Equivalence of DFAs and NFAs, Review of regular expressions; their equivalence to finite automata, Closure properties, Proving languages non-regular via the pumping lemma or alternative means), Context-free languages (Push-down automata (PDAs), Relationship of PDAs and context-free grammars, Properties of context-free languages), Turing machines, or an equivalent formal model of universal computation, Nondeterministic Turing machines, Chomsky hierarchy, The Church-Turing thesis, Computability, Rice’s Theorem, Examples of uncomputable functions, Implications of uncomputability,

Aside from pondering how one can do DFAs and regular expressions but not also NFAs (to name but one), this list contrasts quite markedly with the one from the survey I conducted earlier, which I blogged about and whose results are online. One of the questions of the survey was whether something like a theory of computation course should be part of a CS curriculum, which was responded with a near-unanimous “yes”, in contrast with the CC2001 and the Strawman report. Another question was about which topics should be covered, where a list of 46 topics was provided and respondents could add more. For each topic, they had to select it as being ‘essential’, suitable for an ‘extended version’ of a ToC course, or as ‘peripheral/may be skipped’. Summarizing the full computed preferences of the topics, we see a different picture than the categorization in the Strawman Draft, above; the top-20 topics with their corresponding percentage where the respondents answered with ‘essential’ are:

Regular expressions 98.55, Deterministic Finite Automata (DFA) 95.71, The Turing Machine (basics) 94.37, Context-free Grammars (definition, ambiguity, simplification, derivations) 94.20, Non-Deterministic Finite Automata (NFA, epsilon-NFA) 88.41, Equivalences & conversion RE and automata 85.29, Problems a computer cannot solve 85.29, Halting problem 82.81, Properties of Regular languages 80.30, Regular grammars (RG) 80.00, Examples of some undecidable problems/languages 78.13, Church-Turing thesis 77.94, Computability and decidability 76.81, Equivalences & conversion DFA, NFA, epsilon-NFA 73.85, P, NP, NP-complete, NP-hard, co-NP 73.53, Universal Turing Machine 72.06, Undecidability 68.57, Pumping lemma for Regular languages 68.18, Some well-known NP problems (e.g., TSP, SAT, Node cover) 68.18, Chomsky normal form, hierarchy 67.16.

I understand that not everything can go in the curriculum, as there are more and more topics that ought to be in a CS curriculum whilst the study programme time remains the same, but the rationale behind the Strawman Draft’s lists on ToC topics eludes me. Meanwhile, my COMP314 students will get a full plate of ToC in the upcoming semester (including, among others, Turing Machines, and they will be reminded of Turing’s centenary this year [as an aside: even Nature finally recognized something from Computer Science [3], honouring it with a special, too]).

References

[1] Mehran Sahami, Ernesto Cuadros-Vargas, Steve Roach, David Reed. Computer Science Curriculum 2013: Reviewing the Strawman Report from the ACM/IEEE-CS Task Force. SIGCSE’12, February 29–March 3, 2012, Raleigh, North Carolina, USA.

[2] Peter J. Denning, Douglas E. Comer, David Gries, Michael C. Mulder, Allen Tucker, A. Joe Turner, and Paul R. Young. Computing as a Discipline. Communications of the ACM, 1989, 32(1): 9-23.

[3] Chouard, T. Turing at 100: Legacy of a universal mind. Nature, 2012, 482 (7386), 455-455.