SafeML based reliability assessment
Coordinated by
IESE
In earlier work, a statistical distance-based measure (SafeML) is proposed for machine learning components. In BIECO project, we propose extension of it with the use of Statistical Distance Dissimilarity across time series to obtain SDD based reliability and robustness estimate (StadRE and StadRO).
Description
Describe the innovation content of the result:
Obtain the statistical distance dissimilarity based reliability assessment which takes into account the distance between the data distributions and the performance.
Who will be the customer?
The customers are researchers, industry that want to assess the reliability of their ML component.
What benefit will it bring to the customers?
Runtime prediction of the ML component reliability.
When is the expected date of achievement in the project (Mth/yr)?
Methodology concept in 09/2022, SafeML based reliability 08/2024
When is the time to market (Mth/yr)?
May need some more maturity after the end of Project
What are the costs to be incurred after the project and before exploitation?
SafeML based reliability assessment will be ready for certain use cases without further investment after finishing BIECO but further generalization and research based on it will need.
What is the approximate price range of this result/price of licences?
Still in discussion-
What are the market size in Millions € for this result and relevant trend?
ML market size was estimated to be ~4 Billion in 2021. Expected to grow upto 120 billion by 2030.
How will this result rank against competing products in terms of price/performance?
The addresses the use of ML in safety/financial critical system where a runtime reliability estimate is needed..
Who are the competitors for this result?
National and international public and proprietary evaluators that implement a runtime simulation-based approach for assuring the safety of self-adaptive systems..
How fast and in what ways will the competition respond to this result?
Creation of a similar solution will take at least three years..
Who are the partners involved in the result?
The concept was evaluated on the use case by 7Bulls
Who are the industrial partners interested in the result (partners, sponsors, etc.)?
7Bulls is interested in the results.
Have you protected or will you protect this result? How? When?
Research publications are provided along the way of the concept development and its validation.
Other results
BIECO Integrated Platform
BIECO Integrated Platform will integrate the tools in a loosely coupled way.
Data Collection Tool
Data Collection Tool (DCT) stores information from relevant vulnerability related datasets, providing a single access point to information required by the vulnerability detection and forecasting tools developed in T3.3, as well as for the failure prediction tools developed in T4.2.
Vulnerability Detection Tool
Vulnerability Detection Tool will detect existing vulnerabilities within the source code which may lead to the successful execution of an attack.
Vulnerability Exploitability Forecasting Tool
Vulnerability Exploitability Forecasting Tool will estimate the probability of a vulnerability to be exploited in the next 3, 6 or 12 months.
Vulnerability Propagation Tool
Vulnerability Propagation Tool will calculate and offer the paths affected by a vulnerability in the source code.
Fuzzing Tool
Fuzzing Tool will test System Under Test (SUT) security vulnerabilities or inputs not contemplated that could compromise the system; as a black-box process, by using unintended or incorrect inputs and monitoring their corresponding outputs.
