The challenge and relevance of OT security services

Klaus Mochalski

Hello and welcome to a new episode of the Rhebo podcast. I'm Klaus Mochalski, founder of Rhebo. My guest today is Thomas Menze. He is with the ARC Advisory Group and he is today with me talking about the challenge of services in OT security. But before we get into this topic, a few words of introduction about yourself Thomas.

Thomas Menze

Of course. So thank you Klaus, for this warm welcome and it's an honor for me to be here. So ARC Advisory Group stands for Automation Research Corporation. So we are a market research company, but we have a very strong focus on automation markets and trends and technologies and we have really a long lasting experience. So we were founded 1990 in North America, in Boston, and so we have now 30 years of insight into the automation industry, into the disruptions. And I belong to the European team. We have several offices in Europe, so we cover here from Germany, Western Europe, Eastern Europe, North Africa and Middle East. And we are talking to automation suppliers and end users at the same time. And I think we are going to discuss today some special aspects of cybersecurity. And I'm really looking forward to this kind of discussion.

Klaus Mochalski

Yeah. So this podcast is called OT Security Made Simple. So in your research, what have been topics where you touch the field of OT security recently?

Thomas Menze

Okay, we see a constant change in OT security for years now. Really for years. Remember, everything started with IT security, with IT virus scanner software, with IT patch management and so on. And 20 to 25 years ago all the OT technology was really developed and designed to be air gapped from the internet. So that means cybersecurity in OT environments was not really on the agenda of the developers. And because of that the cybersecurity in OT application was quite low. So this has been changed dramatically because from my point of view, in the last ten years we see really an increased attack activity in the OT environment as well. And that has been changed. Everything and that has been changed, let's say the behavior of end users, let's say big chemical associations in Western Europe, they recommend to their companies, to their partners to do things differently, to be more secure, especially regarding digitalization and the digital transformation. And even the EU Parliament in Brussels, they recommend now to the so called critical OT infrastructure to do things differently regarding OT security. So I would say this whole industry is in a constant change and all the security parameters are definitely rising.

Klaus Mochalski

That's an interesting point you raised. So when you mentioned the European Commission in Brussels. So what made this change happening? Was it the increased awareness, insight on the asset owner side? Or was it legislation that forced many of the critical infrastructure operators into thinking about this topic more seriously in the past and then actually doing something about it?

Thomas Menze

Yeah, good question, really good question. So I would say everything comes from bad experienced. So let's say many, many end user reported an increased number of attacks and really increased damages. And damages goes hand in hand with costs. So it costs a lot of money to solve it out after this successful attack and this was now really realized by the European Parliament in Brussels. And they divided the industry in two big sectors - the critical infrastructure and the normal infrastructure, so just the industry sector. And due to the increased tension coming from the geopolitical situation, so we see really more and more attacks really targeting the critical infrastructure. And critical infrastructure is to keep it short. It's the energy supply, it's the water and waste industry, it's maybe oil and gas. So it's everything which is important to keep our, let's say our European countries here up and running, that provides sustainable services to the people who live here. And these critical infrastructure need a special protection because we have seen ransomware attacks, we have seen phishing attacks who are really targeting especially this industry. And they had two objectives. One objective was of course to gain money, to get revenue out of a successful attack, but secondly, to stop operation.

Thomas Menze

And now imagine if we attack a water and wastewater application and there is no fresh water, no potable water in big cities available that ends up in a disaster. Same belongs to energy supply and so on and so on. So the European Parliament really said, hey, critical infrastructure need special protection and because of that they raised these new legislations.

Klaus Mochalski

Okay, so you raise a couple of points here. So the risk is there and potentially increasing. We all agree on the consequence. So it's serious consequence if a utility company fails to operate water, wastewater, also electricity. So this has serious implications. But you also mentioned that we have seen an increase of targeted attack activities against these providers. If I look at our customer base, we are today mostly serving critical infrastructure customers and we have a lot of energy companies, utilities, distribution system operators, transport system operators, as customers and we provide a lot of service doing regular cyber risk assessments with these customers. And we have done this over the past five years. And so we can also compare the current situation against previous years. And here I have to say that we are not really observing a significant increase in actual attack activity, admittedly on the OT side of the business. So we are only sitting, let's say in the electrical substation. We're monitoring the traffic to and from the control system. So at the very critical operational technology instances, not so much on the OT side. So you wouldn't really expect to see a ransomware attack.

And we know there has been a search here. But the targeted attacks, they would ultimately show up here. So we are observing a lot of vulnerabilities, suspicious communication like activities with the Internet that shouldn't be there. But rarely we see targeted cyberattacks in these protected environments. So what does it mean from your perspective? Does it mean that all of these companies are doing just a tremendously good job in keeping these attacks at bay? Or is it that we're not yet seeing the real wave of attacks that could be here, for instance, if the geological situation that you mentioned becomes worse than it is today already?

Thomas Menze

Yeah. Okay, many questions here. Let's start at the beginning. I think, first of all, your observation is absolutely right. We haven't seen too many successful attacks in the critical infrastructure because we are using cybersecurity appliances since years. So cybersecurity in OT critical infrastructure is mandatory for, I would say, 15 years. So everybody is using that. And now a new trends, let's say ten years ago, a new trends came on the horizon, and that was the required efficiency increase because most of the OT critical infrastructure processes are a bit old. They are very reliable, but they have less digital technology embedded. And we know from other industry, for example, automotive production, that with digital services we can increase the efficiency tremendously. Think of the clever use of robots in automotive manufacturing. And now, of course, the process guys, in the critical infrastructure, they said, hey, we need something similar. We want to use digitalization to enable more efficiency in our processes. But what does that mean? If they use more digitalization, they use more IoT devices. An IoT device or IoT stands for Internet of Things. It means they have small digital appliances embedded at the edge of the operational system to measure here and there, to establish connectivity and to give some guidance, to really take data driven decisions.

I think the era of experienced skilled operators ends because we know we have more or less not enough really experienced workers. They become too old, they are going into retirement and so on and so on. So we have new workers. They are highly motivated, but they have not the same level of experience with the use of digitalization. We can deliver really them the subset of data they can take decisions on and take the right decision to improve the processes. But, and that is a big but. All these little IoT devices, they rely on software and firmware. And we all know these software and firmware is not bug free. So, if I remember, there exists a statistic from the BSI, that is the German Association for IT Security. They see more and more vulnerabilities every year. That means it is just a statistical question when the next successful cyberattack will occur. So your observation is right. Right now, the balance between the used cybersecurity appliances and the associated risk in the critical infrastructure is balanced. That's the reason why you not see too many of these successful cyberattacks. But if we are using more and more digital components in future, it is really just a question of statistic when the next successful cyber attack will happen.

And remember, we have seen something last year that was really bad, the complete stop of a pipeline system in the US. And I think this pipeline was for Colonial Pipeline. And that was out of operation for seven days. And I think nobody from US will see something similar here in Western Europe.

Klaus Mochalski

But also in this case, I recently read again aftermath study on this and the attack didn't really penetrate the OT part, but the company management actually decided to take down OT as precaution so that it wouldn't get hit by the ongoing attack against the IT infrastructure. So here probably if we had our OT monitoring system in place, we probably would not have seen the attack because it was still happening on the IT side mostly.

Thomas Menze

Yes, but it was a precaution. And you say the UN Security of the management regarding cybersecurity, even if they don't see the attack directly in the automation system, they stop everything because nobody knows exactly what's happening.

Klaus Mochalski

And you can never know what's actually going on, even if you don't see anything..

Thomas Menze

And to answer your question, and this is exactly the point where the European Union reacts now and they required from May this year, active attack detection systems in OT environments to raise an alarm if an active attack happens, to take the appropriate action to stop the attack or maybe stop operation of one part of the plant. And I think that is exactly the answer to your question. So it is correct at this moment in time. All these cybersecurity methods we are using is in balance. But due to the heavily use of digital communication, of digital tools, we have to do more to keep this balance for the future.

Klaus Mochalski

Right! So it's not so much that security on a persistent level gets worse, but we simply have a higher quantity of systems. They all have the number of vulnerabilities that these systems tend to have naturally and had always have had historically. And we just see an increase of vulnerabilities that can of course be used by attackers to gain foothold of these infrastructures.

Thomas Menze

Exactly. And if you discuss this with BSI from 2010 to 2024, as I did, the prediction for next year as well, you see the vulnerabilities. This increase is not linear, it's an exponential increase. So we have to take action and we cannot ignore that.

Klaus Mochalski

Right, so this is of course something that we have been noticing at our customers. Many of our customers have worked towards this deadline deploying OT monitoring systems with the goal to detect cyberattacks early, to be able to stop them. A challenge that we hear from our customers quite often is the problem with sufficient experienced staff. So the talent shortage that the entire industry has worldwide, I think it's a worldwide problem. We have this here in Europe specifically, also in Germany, and this is something that well, we all know and have acknowledged that a problem exists, but also, unfortunately, there is no simple and no quick solution to it. What would be a proper countermeasure to basically do something against the challenge? What role can specifically service play in this part? Because especially if we talk about smaller customers, smaller entities, well, we have seen this in the past that they simply asked their IT administration department to take care of OT security as well. We have seen this failing. We believe quite often that's not a good idea per se. You have to invest into educating the IT staff with the specific OT requirements.

So what do you think, what do you observe? What role will service play has played historically and what is the development over the next years going to show?

Thomas Menze

Yeah, so your observation is quite right. Because in IT, I think if you study it, part of your study is really the cybersecurity part. And as I said, 15 years to 20 years ago, in OT cybersecurity does not exist because nobody had the attention to connect the systems to the Internet or to bring additional interfaces in. So because of that, the OT equipment lives in a quite healthy environment and in many cases the IT department of large companies, they take control regarding OT cybersecurity. There was not too much to do, full stop. So this has been changed and today we have three really mature services. This is really the assessment service to assess if the cybersecurity measures are in place and what is the network topology and so on and so on. Then maybe as outcome of the assessment service, you have implementation services. So if there's any corrective action required, then it is implemented by the implementation service or configuration of firewalls, network segmentation and this kind of stuff. And last but not least, they exist managed services. And in managed services you can really scan your network for the communication references, who is communicating with whom, you can do some anomaly detection and you can see what is the patch level of the components and so on.

So this was the traditional service environment in OT cybersecurity. And now on the horizon we see a fourth cyber service domain. And I would call them cybersecurity as a service. Maybe it's limited to the critical infrastructure, but you have to meet these regulatory requirements coming from the European Union. And these cybersecurity as a service is something which is part of the managed service, but it is not one single service. It's maybe a service over the lifecycle of the plant where you constantly monitor the criticality of the application and the deployed cybersecurity measures. And you do frequently animal lead detection scans and this kind of stuff. And you report at the same time to the BSI, to this association in your country who is responsible to receive these cybersecurity reports. And I think these kind of services they become more and more important. Because the IT people have not enough know how to cover the OT cybersecurity because they are using different protocols, they have different objectives compared to the IT. And the OT automation engineers are really a little bit overloaded with IT information. They have the ability to configure their automation system, they have the ability to control really the processes.

But now they have to use really sophisticated IT components to do root cause and anomaly detection. And that is a little bit too much. So I feel we will see in the future dedicated cybersecurity as a service provider who are working especially with mid size and smaller companies, who have not enough resources to provide these kind of services on their own. Maybe the global giants, they have enough resources to do these kind of service themselves. But again, mid sized companies, smaller companies, they have to think different.

Klaus Mochalski

So the problem is somewhat new experience is missing. It's rather complex with the specific challenges of OT and the IT side and then the talent shortage. So this also means that we have to share resources here and here service becomes in a sense also resource sharing for smaller asset owners who still need to have the same or similar security level as their larger peers who can afford having their own teams.

Thomas Menze

Right. Because you should not forget if you say we need experienced talents yes, if they are coming, if they're finishing their university degree, yes, they are experienced and educated. But on the attacker side, let's say they constantly improve their know how, they constantly improve their cyber attack capabilities. And that means we have to educate even our talents of today constantly. And that costs a lot of money. I feel it makes more sense maybe to at least consider a cybersecurity as a service provider because he has the latest technology, he has experienced people and he can share, let's say, the attack analysis from different application. And don't forget at the end, let's say a critical infrastructure provider has a core competence, let's keep this example fresh water supply. The core competence of an OT company is really to supply fresh water. Yes, they can do a decent level of automation to automate the processes. But there are no cybersecurity experts, especially no expert on sophisticated cybersecurity route analysis.

Klaus Mochalski

And they shouldn't have to become experts. So from your perspective, as an advice to our listeners, especially the asset owners, smaller companies looking to deploy security solutions, I know from personal experience that quite often the selection process is centered around picking the right tools. But from your perspective, what's more important, picking the right tools? It's really just one? So if you look for just one class of tools, you're probably already doing a mistake. So picking the right tools for your specific use case or picking the right service to operate these tools, what's the more important decision?

Thomas Menze

I think picking really the right services and understand the criticality of your plant. So let's say the service you are picking must be in line with the criticality of your plant. It is definitely a difference if you are a process owner of a nuclear power plant or a municipal water supply station. But in all cases you need really service providers who give you maybe an independent view on your criticality and then come up with a shortlist what are the best tools and why, and at the end you are in the decision making process and the user has to pick the right components and software tools.

Klaus Mochalski

Okay, so clear advice to our listeners service first, you should spend at least as much time looking for the right service provider than you spend at looking at the proper tools. If you don't do this, you're likely missing out on the best possible protection for your infrastructure.

Thomas Menze

Yeah, so good summary. I agree entirely.

Klaus Mochalski

Yeah. Okay, so that would be a nice finish. But there has been an observation that I made and many of us made over the past weeks that I quickly wanted to touch discuss with you. So there have been a significant amount of layoffs while starting any general IT industry. So there was Google and some of the large companies, but this was also followed by IT security company layoffs and lately also by some specific OT security company layoffs. What's your take on this? I mean, so far in the news have been mostly well funded VC backed North American companies. Is this a general trend in the market or is it just the regular ups and downs of the markets? What's your take on this?

Thomas Menze

So I think this belongs really to the regular ups and downs. These are financial impacts. At this moment in time we have all these geopolitical issues and one outcome of the geopolitical issue we discussed before is increased number of cyber attacks. Another outcome of the geopolitical issues, the global geopolitical issues is really sometimes flat business and now all these companies, they react. And let's say if the huge companies like Microsoft or Google react, it is hard for smaller service providers not to react and explain that to their financial investors. I think the outcome could be if there are more experienced cybersecurity talents now available in the market, maybe they establish their own service companies and they become competitors to their mother companies. But that needs to be observed at this moment in time. If I see this is a normal up and down, it doesn't indicate anything that the cyberattack activity is lower right now than in earlier periods or something like that. Is this really a normal up and down.

Klaus Mochalski

It's quite the opposite. It's a bit counterintuitive. Everybody talks about the current situation, how it's worse than ever before the geopolitical situation and still there are expert layoffs and so this is an interesting move. So I guess what we as the security industry may have to do better than in the past is you mentioned that it may be a decision driven by investors and they are looking at their margins, at profitability. And we all believe that we do cybersecurity to protect these margins and actually to save money in the long run by preventing outages from cyberattacks. And maybe we altogether have to have to do a better job in explaining not our immediate customers but their investors, why what we are doing actually helps them in the long run.

Thomas Menze

Yeah, I think we need to change the perspective, but what you described is really the short term perspective if we look from quarter to quarter. So again, we discussed previously the lifecycle of the plant and this could be easily in OT cybersecurity 20 or 30 years. So we have to really maintain the profits generated by these plants over a longer period of time than just a couple of quarters. And that is the discussion we have to do with our financial investors to explain in why cybersecurity is crucial to really sustain the environment. Even in the digital transformation, there are no short term profits, but longer term we increase our competitiveness and this will generate then really sustainable profits in future.

Klaus Mochalski

Okay, I think that's a nice final word. So not just to take away for our listeners, but also for us as service providers that we need to all do a better job here in explaining not just what we do, but why we do it. And this will help everyone because then security will just increase over time.

Thomas Menze

Yes, I agree. And it was really an honor for me to be here even on this very warm day here in Germany.

Klaus Mochalski

So we're all happy that we are getting to the close of this because it's indeed very hot here. Thank you, Thomas, for being here.

Thomas Menze

Klaus, thank you very much for our conversation today.

‍