SafeML based reliability assessment
Coordinated by
IESE
In earlier work, a statistical distance-based measure (SafeML) is proposed for machine learning components. In BIECO project, we propose extension of it with the use of Statistical Distance Dissimilarity across time series to obtain SDD based reliability and robustness estimate (StadRE and StadRO).
Description
Describe the innovation content of the result:
Obtain the statistical distance dissimilarity based reliability assessment which takes into account the distance between the data distributions and the performance.
Who will be the customer?
The customers are researchers, industry that want to assess the reliability of their ML component.
What benefit will it bring to the customers?
Runtime prediction of the ML component reliability.
When is the expected date of achievement in the project (Mth/yr)?
Methodology concept in 09/2022, SafeML based reliability 08/2024
When is the time to market (Mth/yr)?
May need some more maturity after the end of Project
What are the costs to be incurred after the project and before exploitation?
SafeML based reliability assessment will be ready for certain use cases without further investment after finishing BIECO but further generalization and research based on it will need.
What is the approximate price range of this result/price of licences?
Still in discussion-
What are the market size in Millions € for this result and relevant trend?
ML market size was estimated to be ~4 Billion in 2021. Expected to grow upto 120 billion by 2030.
How will this result rank against competing products in terms of price/performance?
The addresses the use of ML in safety/financial critical system where a runtime reliability estimate is needed..
Who are the competitors for this result?
National and international public and proprietary evaluators that implement a runtime simulation-based approach for assuring the safety of self-adaptive systems..
How fast and in what ways will the competition respond to this result?
Creation of a similar solution will take at least three years..
Who are the partners involved in the result?
The concept was evaluated on the use case by 7Bulls
Who are the industrial partners interested in the result (partners, sponsors, etc.)?
7Bulls is interested in the results.
Have you protected or will you protect this result? How? When?
Research publications are provided along the way of the concept development and its validation.
Other results
Ontology Manager Tool
Ontology Manager is a Framework responsible for managing the Core Ontology used in BIECO, called DAEMON. It aims to support organizing concepts and their relationships related to System of Systems (SoS), Internet of Things (IoT), and System Components management and Monitoring.
Vulnerabilities Forecasting Tool
The Vulnerabilities Forecasting Tool (VFT) provides historical vulnerability data and projections for time intervals of 1, 2, 3, 6, and 12 months for several major software components.
Failure Prediction Tool
The Failure Prediction Tool (FPT) performs failure predictions by monitoring the logs of the applications that make up a system. It has a REST interface through which it receives in real time the log messages from the monitored applications.
safeTbox
The pre-existing tool safeTbox (www.safetbox.de) has been extended to support interoperation with the ResilBlockly tool for combined safety and security analysis.
Conditional Safety Certificates for ICT
Conditional Safety Certificates (ConSerts) have been applied to support resiliency of ICT infrastructures. Support for deployment and execution of ConSerts in ICT infrastructure according to use case needs was provided additionally.
