Artificial intelligence (AI) is currently hailed as a silver bullet against almost everything. So it's no wonder that OT security is also expected to benefit from a self-learning system. Together with Rhebo and other technology partners, the Fraunhofer IOSB has investigated the possibilities in the AICAS project. The automation of AI-triggered mitigation mechanisms is not (yet) recommended in OT.

The goal of the joint project Autonomous Industrial Cybersecurity Assistance System (AICAS for short) was to develop an assistance system for the intelligent, autonomous detection of cybersecurity incidents. The two-year research project aimed to answer two core questions:

1) how can the precision of cyber incident detection in Operational Technology (OT) be increased and thus the probability of false positive alerts be reduced?

2) How can the automation and quality of incident response be improved?

Both questions have a special significance in OT:

1. OT networks, systems and components are fundamentally insecure. It must therefore be assumed that the majority of industrial infrastructures are under threat from novel attack techniques and previously unknown vulnerabilities. As a result, classic signature-based logic from IT security cannot simply be transferred to OT. Of course, tools such as firewalls at the network perimeter still form an important first line of defense against known attack patterns. However, they require a second "line of defense" within the networks to ensure detection of successful network penetration.
2. Defensive operations based on false positives can have a strong impact on the stability and availability of industrial processes. Unplanned downtimes can lead to significant costs if, for example, the internal (and external) supply chain gets affected or equipment has to be recalibrated or serviced after the shutdown.

Project structure

The project was informed by the internationally established MITRE ATT&CK for ICS framework when identifying relevant use cases. Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB designed one testbed for energy companies and one for industrial companies, respectively. The OT monitoring and anomaly detection system Rhebo Industrial Protector and a system of the second technology partner were used as industrial intrusion detection systems.

The project was divided in three phases:

1. Utilizing the MITRE ATT&CK as well as the System Model Processing framework (SyMP) a concept was designed that would build the basis for the envisioned assistance system.
2. In a second step, machine-learning methods were used to enrich alert notifications of industrial intrusion detection systems to increase the quality of alerts and their respective root causes. This capability is crucial for the assessment of security alerts in OT environments and the (automated) decision on mitigation measures.
3. In the final phase, machine-learning methods were implemented using network log data in industrial anomaly and intrusion detection systems to test the accuracy of OT security automation capabiities.

Result 1: Optimized root cause analysis

The SyMP framework allowed for the configuration and implementation of complex analysis. This, amongst others, included automated intrusion detection and correlation, analysis of threats, configurations, vulnerabilities and conformity. The high degree of automation of the framework showed great potential to both amend and support existing approaches.

The enrichment of alerts also showed great potential for industrial use. »Utilizing supervised machine learning enabled the attribution of alerts from Rhebo Industrial Protector to specific attack techniques according to MITRE ATT&CK. This allowed us to assess the root cause of alerts and attach a quality score to the outcome,« Markus Karch of Fraunhofer IOSB explains. »The quality score of an anomaly notification gives operators and security managers an indication on the relevance of an alert and the probability that it is part of malicious activity. The lower the quality score of an anomaly notification the higher the risk of a false-positive alert.« 

The machine-learning supported enrichment resulted in increased quality scores for several alert categories. The used quality scores correlates to the Technology Readiness Level (TRL) as defined by the EU. One example of optimized alert TRL is the eavesdropping on industrial communications using ARP spoofing where the TRL increased from a 4 to 8.

Result 2: OT security needs humans

However, it also became apparent that a large number of anomaly notifications could not be assigned and evaluated unambiguously by automated procedures if they were preceded by an insufficient learning phase. Thus, unpredictable challenges, such as misconfigured DNS, NTP or update servers, can lead to problems when using machine learning methods in OT security.

»We still rely on insider expert knowledge who share intelligence on specific configurations, policies, presumptions and requirements to assess anomaly notifications,« Karch warns. Adding to that the ever-changing threat landscape, makes AI-supported security automation that actively blocks communication or stops processes in the OT prone to errors. »Context is everything. And context can vary heavily from network to network. In OT, AI or an intelligent algorithm can function as a VORAB filter only which internal experts can use to assess the notification. In this regard, the project also proofed that heuristic methods still provide very effective results compared to AI. In both cases, the decision on mitigation measures needs to come from the people working daily within the infrastructure.« 

Automated security mechanisms might have both overlooked real attack activities or disrupted industrial process based on false-positive alerts.

For Karch, the results of the study confirm what has also been apparent for some time under 'real-world' conditions: »By working with Rhebo, we were able to determine that the majority of cyberattacks can be detected by conventional whitelisting methods used in anomaly detection. Thus, when evaluating machine learning techniques for network-based anomaly detection, future scientific work should always analyze whether they provide any advantage at all over conventional techniques.«

Furthermore, the research done within the AICAS project can be utilized to further optimize industrial intrusion detection systems. Rhebo uses the project’s results in its OT monitoring with anomaly detection Rhebo Industrial Detector:

1. Its general approach and methodology to detect and document security-related alerts is already based on the MITRE ATT&CK frameworks. 
2. Algorithms that produced significantly high TRLs were incorporated in the product to enable a faster assessment of anomaly notifications. 

The quality of TRLs for anomalies and combined attack vectors is set to increase in the future. Rhebo has already included this to its long-term product road map.