Published on Tue Oct 30 2018

The Responsibility Quantification (ResQu) Model of Human Interaction with Automation

Nir Douer, Joachim Meyer

Intelligent systems and advanced automation are involved in information collection and evaluation. In such systems, human responsibility becomes equivocal. Using Information Theory, we develop a responsibility quantification (ResQu) model of human involvement in intelligent automated systems.

0
0
0
Abstract

Intelligent systems and advanced automation are involved in information collection and evaluation, in decision-making and in the implementation of chosen actions. In such systems, human responsibility becomes equivocal. Understanding human casual responsibility is particularly important when intelligent autonomous systems can harm people, as with autonomous vehicles or, most notably, with autonomous weapon systems (AWS). Using Information Theory, we develop a responsibility quantification (ResQu) model of human involvement in intelligent automated systems and demonstrate its applications on decisions regarding AWS. The analysis reveals that human comparative responsibility to outcomes is often low, even when major functions are allocated to the human. Thus, broadly stated policies of keeping humans in the loop and having meaningful human control are misleading and cannot truly direct decisions on how to involve humans in intelligent systems and advanced automation. The current model is an initial step in the complex goal to create a comprehensive responsibility model, that will enable quantification of human causal responsibility. It assumes stationarity, full knowledge regarding the characteristic of the human and automation and ignores temporal aspects. Despite these limitations, it can aid in the analysis of systems designs alternatives and policy decisions regarding human responsibility in intelligent systems and advanced automation.

Wed Jul 22 2020
Artificial Intelligence
Regulating human control over autonomous systems
In recent years, many sectors have experienced significant progress in automation. It is argued that the use of increasingly autonomous systems should be guided by the policy of human control. In the military sector there is a fear that AS could mean that humans lose control over life and death decisions.
0
0
0
Thu Oct 31 2019
Artificial Intelligence
Extending Causal Models from Machines into Humans
Causal Models are increasingly suggested as a means to reason about the behavior of cyber-physical systems in socio-technical contexts. We show how a technical causal model can be extended with models of human behavior to cover the complexity and interplay between humans and technical systems.
0
0
0
Mon Jul 08 2013
Artificial Intelligence
A Knowledge-based Treatment of Human-Automation Systems
In a supervisory control system the human agent knowledge of past, current, and future system behavior is critical for system performance. Being able to purposefullyreason about that knowledge in a precise and structured manner is central to effective system design.
0
0
0
Tue Oct 27 2020
Artificial Intelligence
Assured Autonomy: Path Toward Living With Autonomous Systems We Can Trust
Autonomy is a capability that enables systems to behave without direct control by a human operator. The challenge of establishing assurance in autonomy is rapidly attracting increasing interest in the industry, government, and academia. A series of three workshops aimed to help create a unified understanding of the goals for assured autonomy.
0
0
0
Fri Jul 26 2013
Artificial Intelligence
Man and Machine: Questions of Risk, Trust and Accountability in Today's AI Technology
Artificial Intelligence has grown into a discipline that is deeply entwined with commerce and society. Today's AI technology poses some difficult questions of risk, trust and accountability.
0
0
0
Mon Oct 12 2020
Artificial Intelligence
Thinking Fast and Slow in AI
This paper proposes a research direction to advance AI which draws inspiration from cognitive theories of human decision making. The premise is that if we gain insights about the causes of some human capabilities that are still lacking in AI, we may obtain similar capabilities in an AI system.
1
0
0