The SAS project consists of six work packages, split into different categories. The Science and Technology (S&T) work packages are WP1-WP3 and training is WP4. In WP5 all activities related to Exploitation, Dissemination and Communication are bundled and all management activities are grouped under WP6.
WP1: Designing Inherintly Safe Autonomous Systems
Methodologies and techniques to integrate safety directly into the architecture/design of the autonomous system.
- ESR 1 – Development of a Generic Framework to Monitor and Handle Safety of Autonomous Systems during Run-Time
- ESR 2 – Development of an Adaptive Platform for Resilient Autonomous Systems based on a MAPE-K Cycle
- ESR 3 – Dynamic Safety Handling of Autonomous Systems-Of-Systems With Run-Time Safety Contracts
- ESR 4 – Creating Software Design Guidelines and Testing Specifications for Non-Functional Requirements in Safety-critical Autonomous Systems
- ESR 5 – Making Connectivity Work Reliably in a diverse Range of Environments
WP1 involves 5 ESRs and tackles the actual safety-aware design, i.e., making safety inherently part of the design process, of resilient autonomous systems, with 3 ESRs focusing on generic frameworks and methodologies to guarantee by-design safe behaviour during run-time and 2 ESRs focusing on specific hardware and software techniques-and-measures to achieve fault-tolerant – or even fail-operational – behaviour.
ESR1 and ESR2 take up the challenge of developing generic frameworks to monitor and handle the safety of autonomous systems during run-time. No complex system can be considered fault-free and this is particularly true for autonomous systems having non-deterministic decision-making capabilities. The role of such a safety monitoring-and-handling framework is to observe the system and its environment and to trigger interventions that maintain the system’s safety, so-called safety rules. As for non-autonomous systems, a human operator takes a significant role in this fault-monitoring and, definitely, in the fault-handling, only a limited set of safety rules had to be considered in the past. In contrast, versatile autonomous systems will have to deal with a much richer set of safety rules. Moreover, these safety rules have to take into account the wide application of machine learning in autonomous systems, causing them to evolve over time, and the a-priori largely unknown open-context in which autonomous systems will be applied. Current fault-monitoring and fault-tolerance mechanisms, which are fixed prior to run-time, will no longer be sufficient. Dynamic adaptation of fault-detection and fault-handling will be a key ability for safe autonomous systems. ESR1 and ESR2 will be working on related, but complementary projects, with ESR1 having the task to extend current safety-monitoring frameworks such that they cover the whole chain from safety-constraint definition to the actual autonomous reactions to avoid a possible hazard, while ESR2 starts from a MAPE-K cycle (i.e., Monitoring, Analyse, Plan and Execute based on Knowledge represented in run-time models) to enable real-time adaptations of functionality, structure, and fault-tolerance mechanisms in order to assure the run-time resilience of autonomous systems.
ESR3 will run in parallel with ESR1 and ESR2 and go one step further and integrate dynamic safety handling of autonomous systems-of-systems through run-time safety contracts into the adaptive safety monitoring and handling framework. Driven by trends like ubiquitous computing and cyber-physical systems, new application domains for autonomous systems-of-systems have emerged. Cooperative agricultural vehicles such as harvesters and tractors that are combined into autonomous harvesting fleets to optimize harvesting in the field, car-to-car interactions that help to prevent accidents at intersections or optimize cruising speed, or plug-and-play emergency rooms supporting the rapid, on-demand (re-)configuration of surgical equipment are only a few promising examples. In such systems-of-systems, different devices are combined during run-time to fulfil higher-level emergent functionalities in a collaboration that cannot be provided by one of the involved systems on its own. Of course, the safety of such an autonomous system-of-systems must be guaranteed. However, classic safety assurance relies heavily on a complete understanding of the structure and behaviour, which is not available at design-time for an autonomous system-of-systems. It is therefore more reasonable to use the idea safety contracts between the different subsystems. Safety contracts are an effective way to conditionally describe the safety guarantees that a component should fulfil in order to make sure that the overall system-of-systems remains safe. Up until now, the use of safety contracts has mainly been limited to static, non-evolving systems. ESR3 will extend this approach to dynamic, modular safety contracts. In addition, these safety contracts will be a key element in the run-time safety-assurance strategies of ESRs 10 and 11 (WP3), covering two flavours of executable assurance.
ESRs 4 and 5 will work on effective techniques and measures that assure by-design that even under fault conditions the autonomous system remains safe without any human intervention. When autonomy increases, so does the software complexity and thus the likelihood that it contains faults. Therefore, ESR4 focuses on software design guidelines and testing specifications for non-functional requirements in safety-critical autonomous systems. Future applications of autonomous systems will rely heavily on different communication technologies to connect and interact with other devices, infrastructure, the “cloud”, etc. Although adding connectivity has its benefits, it also adds challenges, among which are most definitely its robustness and resilience. ESR5 focuses on more hardware-oriented design and testing specifications, which make connectivity work reliably under a diverse range of environments. This takes into account a combination of stresses, including electromagnetic interference, temperature and vibrations, aging, etc.
WP2: Providing Evidence for Autonomous Systems
Effective model-based system analysis techniques to provide evidence that the behaviour of an autonomous system remains safe under all conditions.
- ESR 6 – Virtual Worlds Generation for Testing Autonomous Robots in Simulation
- ESR 7 – Rigorous Design and Evaluation of Situation Coverage Testing for Autonomous Vehicles
- ESR 8 – Model-based System Analysis Techniques to Determine Propagation Paths of Functional Insufficiencies in Software-intensive Systems
- ESR 9 – Model-based System Analysis of the Robustness of Autonomous Systems against ElectroMagnetic Interference
WP2 targets novel methodologies that allow us to evaluate, validate and verify the safety-aware design (WP1), meaning that safety can be guaranteed given the complex environment and extremely varied use-case scenarios that autonomous systems will be subjected to. This challenge cannot be underestimated. Just recently, Michael Bolle, President of Bosch, Corporate Research said in a speech: “We have looked at what it takes to physically validate autonomous driving, and the time needed was estimated at 100,000 years. We need breakthrough solutions from the research community.” As physical testing is too costly and too time consuming, we must turn to virtual, i.e., simulation- and model-based, testing.
ESRs 6 and 7 will collaborate to achieve a breakthrough with respect to the overall coverage of the model-based safety analysis. ESR6 will address the issue of the virtual-worlds generation and will apply this to autonomous robots. In other words, ESR6 answers the question “which operational situations and environments should be tested in the virtual world?” and starts from a criticality analysis. Once the most critical virtual worlds have been generated, ESR7 will evaluate and maximize the situation coverage of each of the virtual worlds. Exploiting combinatorial testing techniques, ESR 7 will determine exactly which simulation runs should be performed to maximally challenge robot’s ability to cope with the features of its environment.
Whereas a classic model-based safety analysis often limits itself to failures of one or multiple components, the open-context nature of autonomous systems forces us to also consider the safety-applications of functional insufficiencies. A typical example being a camera in a self-driving car that should prevent a collision with a human being, but only detects a human correctly in 99.9% of the cases. Therefore, ESR8 is going to look at model-based system-analysis techniques to determine propagation paths of functional insufficiencies in software-intensive systems and will use probabilistic ways to model the uncertainties.
Complementary to ESR5 (WP1), ESR9 will also take up the challenge of the strong reliance of autonomous systems on wireless communication and will perform a model-based system analysis of the robustness of autonomous systems against electromagnetic interference. Combining efficient statistical electromagnetic modelling with behavioural modelling, the resulting behaviour of an autonomous system upon electromagnetic disturbances will be forecasted and evaluated.
WP3: Providing Assurance Strategies
To develop novel safety assurance strategies which combine the architectural/design measures with the evidence in order to allow us to have trust in the autonomous system.
- ESR 10 – From Static Assurance Cases at Design-Time to Executable Assurance Cases at Run-Time
- ESR 11 – Assurance Case Structures for Machine Learning in the Decision Making of Highly Autonomous Systems
- ESR 12 – Assuring Autonomous Sailing from A to B while Minimizing Operational Costs
- ESR 13 – Safety assurance for Clinical Conversational Bot
- ESR 14 – Dependability Assurance for Vehicle Autonomy
- ESR 15 – Between Safety and Liability: Towards a Liability Allocation Framework for Safe Autonomous Systems
The S&T WPs conclude with WP3, which pilots novel safety-assurance strategies, combining the previous 2 research WPs, thereby allowing us to put trust in the safe behaviour of autonomous systems. In total this WP involves 6 ESRs and, besides safety, also covers other design constraints such as security, reliability, availability and liability.
ESRs 10 and 11 both focus on dedicated assurance cases for autonomous systems. Existing standards, processes and practices place a great emphasis on how safety can be certified throughout the design and development stages. However, there is little guidance on how safety assurance should be maintained throughout the system’s operational life. Many assumptions about the environment and the system performance and use, particularly for complex and novel autonomous systems, that are made during the design and development stages might turn out to be incorrect during operation. From a safety point of view, this can threaten the validity of the safety case and weaken confidence in the actual safety of the system. Within SAS, two complementary approaches will be pursued to tackle this. On the one hand, ESR10 aims at making the transition from static assurance cases during design-time to executable assurance cases during run-time. Here, a safety-assurance case is structured argumentation, supported by evidence (WP2), intended to justify that the system is designed (WP1) such that its behaviour is acceptably safe when being put into service. While for non-autonomous systems, the whole safety case is traditionally developed, documented and accepted prior to operation, the safety case for some autonomous systems may instead need to be posed with residual obligations that are only satisfied during run-time. For example, the vast number of possible inter-vehicle and infrastructure configurations that an autonomous vehicle may encounter may require run-time verification of safety properties (such as end-to-end response times, or the integrity of received data) to sustain the safety case.Therefore, ESR10 will establish a new way of working with an executable set of claims that will be sustained and maintained during run-time. On the other hand, ESR11 will study assurance-case structures for machine learning in the decision making of highly autonomous systems. Currently, the use of machine learning or any other artificial intelligence technique, is not recommended for safety-critical tasks. However, many autonomous systems will rely on machine learning and ways to address this are urgently needed.
ESRs 12, 13, and 14 all start from a specific application scenario, i.e., autonomous vessels, clinical conversational bots and autonomous vehicles, respectively. In addition, they always consider other, possibly conflicting, design constraints, as is the case in industrial practice. ESR12 wants to assure safe autonomous sailing from A to B while minimizing operational costs by combining a cost-optimization algorithm, a collision-avoidance algorithm and situational awareness. ESR13 looks at the safety assurance for clinical conversational bots by combining safety engineering with typical clinical processes. ESR14 will cover the whole dependability assurance for autonomous vehicles, covering, besides safety, also reliability, availability and cyber-security.
Last, but certainly not least, ESR15 will take up the emerging challenge of the liability aspects of autonomous systems in safety-critical domains. ESR15 will propose a liability allocation framework for safe autonomous systems that explores new avenues for the allocation of liability for autonomous systems in a way that strikes a balance between the commercial interests of operators and manufacturers and the safety and fair compensation of the general public.
Create an innovative European Doctoral/Graduate School on safety of autonomous systems; Develop a structural training programme for current and future generation of researchers/engineers/innovators active in Europe; Change the mind-set of European researchers and safety engineers from static safety assurance to dynamic safety assurance.
This WP applies to all ESRs.
WP5: Exploitation, Dissemination and Communication
Disseminate the results of SAS to the academic, teaching, industrial and public communities in Europe. Establish and deploy a solid exploitation plan for the future market introduction of safe and trustworthy autonomous systems.
This WP applies to all ESRs.
Successful and efficient project execution according the work plan, maintenance of relations between Participants and ESRs, communication management and reporting.
This WP applies to all ESRs.