Klaus Mochalski
Hello and welcome to a new episode of the OT Security Made Simple podcast. I'm Klaus Mochalski, founder of Rhebo. My guest today is Max Weidele. Max and I have known each other for many years. Max is probably known to many as the operator of the [German]website Sichere Industrie (Secure Industry). Please introduce yourself briefly.
Max Weidele
Yes, with pleasure, Klaus. My name is Max Weidele. I am the founder and managing director of the [web platform] www.sichere-industrie.de. On the one hand, we are an engineering consultancy for security and, on the other, we also run the Secure Industry knowledge platform. I think we now have just over 100 specialist articles, where we simply share specialist content from the projects. Where we also promote the community idea a little and make our small knowledge contribution to publicize the know-how a little more.
Klaus Mochalski
I think everyone really appreciates what you have done for the community over the years in terms of, shall I say, sharing knowledge. OT security was a rather obscure topic for a long time and you started investing in this information transfer relatively early on. I think that was very helpful for many people out there. I'm interested in what you've seen in the last few years when you've been observing this market – from a slightly higher vantage point and from various aspects. What are the core developments that you have noticed over the last two or three years?
Max Weidele
It is particularly exciting because when we started back then, certain search terms simply did not exist and still don't. So you can't even optimize your website with content for certain terms. Of course, there are the very familiar terms such as KRITIS (in Germany short for critical infrastructure), IT Security Act (German implementation of EU directive NIS1) and so on. But beyond that, it gets very, very thin. This is also very exciting because we naturally also look at the content that is doing particularly well. Or we also talk about our projects and topics that have naturally gained momentum, especially in the last year or two. I mean the classic NIS2, the IT Security Act, KRITIS, the Maschinenverordnung (Machinery Order: German implementation of EU directive 2006/42/EG or Machinery Directive) and all that sort of thing. Everything that is ultimately pushed from this compliance corner.
But if you then look more closely at the technical measures, asset management is very much on the rise, and all you can really say is: Oh, thank you at last! Everyone has been talking about it for 20 years: We can only protect what we know. And the solution is asset inventory, asset management, and very few people have implemented it. Not even in IT.
And I would say that the topic that comes after that is something that is also very close to my heart personally, but which many people don't mention so much, but rather when you're involved in conversations. And that is the whole topic of organization. And it's actually about the fact that we have a shadow organization in OT simply because of these established structures. A classic example: the maintenance technician who actually does maintenance now also does network technology in production. But he was never actively told this. It's also not in his job profile, and it's not included in his annual planning or that he even has time for it.
Klaus Mochalski
That's very exciting. You mention the topic of OT organization, which has also been a frequent topic in our podcast here. I actually recognise that as a pattern, that the whole market is becoming more mature, that it's less about technical solutions. They will certainly play a role, they will always play a role. But the topic of organization, i.e. how do I integrate this into my entire company organization, is becoming increasingly important. Especially as companies' processes mature and OT security becomes more integrated into regular processes.
Let's take another look at that in a moment. But before we do that, I'd like to go back to one thing you mentioned. I've come across it very, very often. But I've also been following an interesting discussion recently: You can only protect what you know. Sounds like a commonplace at first. Dale Peterson (OT security expert from the USA and organizer of the S4 event series) once wrote an article saying that this can't actually be right if you look at banks and safe deposit boxes. They do protect things that you don't know. And if you transfer that to IT, of course, a firewall also protects things that it doesn't know.
It protects the perimeter, controls who can get in and out. But the assets on the inside don't really matter for the time being. Is that true? Do we need to know what we have in the OT infrastructure in order to be able to secure it?
Max Weidele
I think there are two classic approaches. One is this approach of looking from the outside in. Moving from the perimeter to the inside. And the other approach is the nasty word zero-trust approach, where we go from the inside to the outside. Where I start with the highest possible quality of protective measures for my individual assets. And I think it always depends a bit on that. The zero-trust approach would be absolutely desirable if you have the time and the means to proceed in this way.
Klaus Mochalski
But of course you have to know the assets, because in the end everyone has to authenticate themselves and I think that's actually an appealing approach. And we also talked about zero trust in OT a few episodes ago. In other words, this approach is definitely gaining momentum here too.
Max Weidele
Yes, absolutely. And I think, if you stick with the comparison of the safe deposit box, the question you have to ask yourself is: How do they communicate? At the end of the day, the person who goes to the safe deposit box and takes something in and out is the one who communicates. And in technology, these are simply automated communication flows that we have permanently. In other words, it's actually talking all the time. And then of course you can say: maybe I don't need to know exactly what the asset is sometimes, but I do need to know what the communication looks like and what can be done there in terms of function and process.
You can certainly put every asset behind a firewall, it's quick and you don't have to worry about the asset. It becomes problematic when the asset has to talk to another asset, and I think that's a bit of a challenge. In other words, before I need to know the detailed asset, I need to know what communication I have and what this communication needs to achieve with each other. And ultimately, I think that's what I'm actually doing. I try to map the communication flows that I need for my functional processes as securely as possible. Yes.
Klaus Mochalski
In other words, it really doesn't work without knowledge. And I believe this is also an observation that all market participants have been able to make for many years. That asset management is becoming increasingly important and is actually being implemented. And that people are not just talking about how nice it would be to have it. In this respect, I believe it is a very important aspect.
Max Weidele
In the end, it’s also an introduction to responsibility. The question of Who owns the asset?
Klaus Mochalski
Asset management is only secondarily a technical solution. It is primarily about organization, responsibilities, structure and hierarchies. And that brings us back to what you actually wanted to emphasize as an observation over the last few years. Namely, that there are very clear developments in the organization. You also do consulting, you said. If you now think about a specific project, what are the things that you have really been able to observe happening there?
Max Weidele
I think everyone is also familiar with this issue of IT and OT having to talk to each other. There has to be a translation. And you very quickly realize that there are no contact persons. Or certain people have never spoken to each other. But there are people in the company who do certain tasks. Ultimately, this is what we call the OT organization.
And what we actually see there is that it is not clear who owns certain assets, who is responsible? You can imagine it a bit like this... I'm sure some people are familiar with the Purdue reference model. In other words, we have our layer model, and we have an enterprise zone or similar at the very top, which can be clearly assigned to an IT department. Or if we start from the other side, from the bottom, we know that these are assets, these are controls, these are perhaps sensors, they are in the e-technology in the plant or something like that. What is actually interesting is exactly what lies between these two layers.
It is usually very easy to determine the responsibilities of the top layer and those at the bottom. And now you can ask yourself the question: Who owns the production firewall? Is it an IT asset or is it an OT asset or is it an asset of the plant? And there is no easy answer to this question. Because presumably IT provides the asset, perhaps even operates it. But they can't implement the right firewall rules and the like because they don't know what the process is like. This means that engineering may have to provide precisely this information. And suddenly I have this meeting of these two worlds. And that's exactly what it's all about.
The companies are standing there trying to figure out who is responsible for this? Is this an activity that lies in maintenance? Does it perhaps also make sense... Let me say I have six plants. In each of these plants, I have one person in maintenance, one person in electrical engineering, etc. in a wide variety of related departments. But they all do the OT networks on site. Does it perhaps make sense to formally give them the title of OT admin and also group them together in a cross-plant team so that they benefit from each other?
These are questions that companies are currently asking themselves in order to simply, I always say, organize IT operations or the operation of IT-heavy systems in production. And this is growing together step by step. Here's a good example: we already have an IT department that provides a PDA, a production data acquisition system. However, it is ultimately operated by OT or the plant. And that's actually the tension that [companies] are experiencing, which is why we often make a distinction between IT, OT and the plant. For us, it's not just OT. For us, it really is this intermediate layer, this unregulated intermediate layer is ultimately the OT. And that's where [the companies] are right now. Trying to establish a service catalog. What services do we actually have? Can IT perhaps also provide the VMware administrator? Can IT provide us with the servers? How do the right [industrial] applications get onto these servers? In principle, we are trying to professionalize this operation, just as we have professionalized IT operations in recent years.
Klaus Mochalski
If you think about where the topic of OT security belongs and take two steps back from day-to-day business, isn't it obvious that...
So I'm thinking of two things now. Firstly, I also spoke to someone in a previous podcast about the fact that the OT security risk has to be considered like any other risk for the operation of my company.
Max Weidele
Yes.
Klaus Mochalski
And that fits into this list of risks. This risk assessment has always existed, and it simply has to be included. This risk assessment is ultimately a management task. It then goes down the hierarchy from management. That's where I need the relevant experts. But it is an overall corporate or overall organizational issue that must not be distributed across individual pillars. There must be no silos.
On the other hand, I'm thinking of a discussion I've often had with customers. They keep asking: Why should I use the security solution from Rhebo, for example, and not rely on the security solution from my vendor XY [who also supplies me with the control technology]? Let's call them Siemens or Honeywell. Whoever they are. And I think that's always [a view point] very easy to argue against. Safety, security, IT, OT security, no matter in which area, is a cross-cutting issue. It is vendor-independent. It affects my infrastructure, so to speak, regardless of where the components come from, who supplied the components, which vendor they are from. And if I rely on every single vendor, I end up with a patchwork quilt that doesn't fit together at all from an overall organizational perspective. And just as I am convinced that this is true [in IT] – and that is why there are also many pure IT security providers in the IT security industry that operate independently of the vendors – this overall approach is also always needed in the OT security organization. It is passed down from the management of the organization through the entire hierarchy and is applied there in the individual specialist departments. Of course, feedback is always provided in order to assess risk and then propagate appropriate decisions from the top down.
Max Weidele
What perhaps also needs to be separated at this point are the topics of OT security responsibility and OT organization. Just like you separate IT security responsibility and IT organization. Because that's exactly what it's all about. Ideally, I don't have an IT security responsibility like the CISO as a team member in my IT, but rather as an auditing position which sets the requirements that IT simply has to fulfill from a security perspective.
And that's exactly what it's all about. Creating that for our OT too. That's exactly what the organization is all about for me. The OT and plant world. In principle, companies now have to set up IT in the OT. Just as they have built an IT organization, they now have to build an OT organization. And then they give a CISO, for example, the additional requirement to also take responsibility for OT security. The CISO then passes on their IT security requirements to the IT department and their OT security requirements to the OT departments.
The CISOs often have no one to talk to. They have no one to whom they can say: “Please run your Siemens world like this, like this, like this and like this”. Because there is no one there. It is precisely this contact person, this shadow organization, that now needs a face. And that is what actually needs to be developed. And that's what companies do with service catalogs. Or they create a product owner for OT networks and ultimately give it a face. Then you have operational people who can also take care of any requirements. But they don't write them themselves. The CISO does that.
Klaus Mochalski
Do I understand correctly that, despite the technical expertise that exists in the OT departments, from your perspective the central function of OT security should be located in the IT department as a new specialist area under the leadership of the CISO?
Max Weidele
So, I think a company needs a main responsibility who ultimately takes a holistic view of IT and OT security together. Because in the meantime, they have become so intertwined that there can only be one person who subsequently passes this on to the organization. What is happening at IT department and OT department level is that companies are now asking themselves the question: “Is a maintenance technician who manages OT networks all day still appropriate in maintenance? Or is this actually an OT network admin and should perhaps join the Global IT network team?”
This does not mean that global IT can remain exactly as it is right now. They have to change enormously in order to meet the requirements. But these are questions that are currently being asked. Someone who spends the whole day preparing servers in production, but is actually an e-technician, should they do the job, give it up or switch to another part of the organization with this package of functions? Because quite often this is this shadow OT. None of them have this in their job profile. No one has even looked at the effort that goes into it. Sometimes you don't even know that all this is already...
In other words, every company [already] has an OT organization. If it didn't have one, nobody would be deploying any servers there right now. Nobody would put HMIs into operation. Nobody would be doing any work in production there. In other words, the companies have something. It's just often not organized. And that eliminates all the synergy effects. For example, why do I have to have a VMware specialist in my operations engineering department when I have an established VMware specialist group in IT that I only have to explain to: “Hey, when you provide me with a virtualization server for Siemens, please pay attention to 1, 2, 3 and 4.” And then they say: “No problem at all! Here you go!”
And that is actually what is being recognized more and more and is happening more and more. Because it's just not that different in some areas. I think we're talking about the last 20% of fine-tuning in many areas, and I have to tell IT what to look out for. And after that, it's regular IT operations for the time being. Especially if you're in the engineering environment and so on.
Klaus Mochalski
And if I understand you correctly, you see the last 20% primarily as an organizational challenge, because most of the functions already exist in most companies. However, these functions, these executive bodies, are often wrongly organized. Take the electrical engineer who installs servers in production, for example, and not only perhaps senselessly uses up resources that could be obtained more efficiently elsewhere, i.e. in the IT department. On the other hand, maybe not. Over the years, the IT department may have defined standards for what certain installations should look like, including in terms of security configuration. But today, these standards do not extend to the IT systems that are installed in OT. And that is of course a vulnerability. In other words, these units need to be merged organizationally.
Max Weidele
Absolutely. How many times in projects have I simply taken existing documents from an IT organization and added one or two smart chapters to these documents and processes and inserted them into this overall process in order to do justice to OT security. You often don't write it from scratch. So there are already so many topics in place.
I would say that this is actually the topic of IT/OT convergence in its purest form. I bring together the OT areas of responsibility with those that IT is perhaps already doing on the other side, and learn from both sides. Ultimately, the OT world is the customer and in case of doubt, IT is the supplier. And they have to adapt to their customers and build up skill sets. This does not mean – and this is very important – that IT can remain as it is. They will have OT experts in their team. A former maintenance engineer may be a member of IT at some point, and it is simply a Global IT/OT. Nothing else really makes sense in many companies.
When I look at how, for example, SAP now communicates across the company, how my PDA-MES communicates. This is becoming so intertwined that we are already getting into a corner where you sometimes have to ask yourself the question: Can someone do OT security without understanding IT security? Or can someone only do IT security without also understanding OT security? For certain topics, because the systems go across all models, i.e. layers, you can achieve a lot of synergy effects if they simply work more closely together.
Klaus Mochalski
Yes, I agree. I've also been talking to customers for years about how difficult it is to find experts who fully understand OT security, because they have to understand completely different aspects of the problem – the OT problems on the one hand, but also the IT security problems on the other. And I often don't find that in the same head or in the same person. And that's why the key here is collaboration.
And that's why I think it's fitting that you say that OT security has to move to the IT department as a function, that the IT department has to change and become bigger. On the one hand, in terms of the scope of expertise, but also in terms of reaching into the different areas of a company, so to speak. And this organization, and ultimately the CISO as the head of the organization, will have to face up to this change.
Max Weidele
Exactly. And ultimately, of course, a management team where, let's say, a Managing Director of Finance and a Managing Director of Engineering say: Yes, we're handing over four maintenance staff from three plants to IT, for example. And where IT is also very much expanding its supplier obligations to the plants. Also the topic of SLAs (service level agreements) and the like. A plant naturally wants to continue to receive its service in an appropriate quality, ideally better. Yes, at the end of the day, that's what it's all about.
Klaus Mochalski
Yes, I think we have a very clear recommendation and a forecast of how the organizational structures will develop.
Now for the last question. From your specific observation, when I embark on this journey as a company with an OT infrastructure… I have these specialists that you describe, the OT technicians, who have somehow also taken on security responsibility because they have grown into it. I have my IT department that works over there and sometimes even co-governs, but it's not really structured yet. And now I want to embark on this journey as a company and set up a clean structure like the one we've just described. How much time and investment am I talking about?
Max Weidele
Yes, I usually say that this journey takes 2 to 4 years with varying degrees of knowledge. For some people, it is already clear that this will sometimes be the goal. Others need an intermediate step, a whole lot is happening there.
I think the important thing is to simply get started. And that's relatively easy to do. Very few companies sit down and say: What is OT for us? What is an OT asset? And [if someone then] says: Yes, we have defined it here, these are the production topics and so on, you can ask the next question: What about logistics? What about building automation, etc.? Very few companies have defined OT for themselves. Every company that has OT and wants to protect it should do this for themselves.
The second step is then to go into the corner of a so-called OT service catalog. You simply write down with the people – also from OT and IT – which activities, which processes, which systems, which services do we actually have that we provide in production? This includes maintaining asset management. This includes the exchange of HMIs. This includes implementing the cybersecurity requirements of corporate IT. This includes the provision of production data acquisition or a SCADA system, etc. In other words, all these IT-heavy services in production.
Then you have a list and take a look at who is currently doing that. And for some topics, IT is quickly involved. And IT also says: Yes, everything is just right, everything is running as it should. For other issues, it depends on which network it is. This is not so often the case now, but networking has been an issue for the last two years, where the operation of the OT network came up and IT said: We'll do the network! Then technology said: Yes, well, this means SLA 24/7. [To which IT responded:] No, we don't do that. We don't do that at all. Then you come to exactly this layer in between, where you have to separate and ask: Well, how do we handle this then? And you also have to redefine and say: When we talk about OT, we're not talking about the main switches in production. We are talking about the switches to which the control systems are connected. Initially, IT doesn't even have that on its screen.
Klaus Mochalski
Exactly.
Max Weidele
And this service catalog leads me to the discussion about responsibilities. I can determine a plan/actual and then start to trace my service map and look at “Who do I already have? Who is already doing this?”. And then Mr. Smith pops up again, because he is the maintenance engineer who is actually the network administrator in Plant 5. But nobody knows about it.
Klaus Mochalski
In other words, to summarize, we can say that this journey, that a project like this has a certain scope and is quite complex, with challenges in the details. And that's why we estimate that it will take two to four years. But on the other hand, you can certainly make rapid gains with the first steps. You also make progress in the first few months. In other words, it is worth starting as soon as possible because you will then see successive improvements.
Max Weidele
The main lever, whether OT security or great new stuff of the future – Industry 4.0, digitalization, etc. – lies precisely in this. I need an organization that is capable of putting my great new technological things on the road and also meeting my security requirements.
Ultimately, we are talking about an organizational change. This means that people may move to another department. It may be that a works council has to be informed. Tariff agreements may change and so on. We're changing an organization and that doesn't just happen at the click of a button or with a signature from the managing director, it's simply a change process that you have to [manage] over a certain period of time, where you have to take people with you so that you don't lose anyone. And that's usually 2 to 4 years, depending on the organization.
Klaus Mochalski
Very good closing words. So nothing to be afraid of, but definitely a challenge. But the most important thing is to get started so that you can immediately improve overall security in IT and OT. Max, thank you very much for the interesting discussion. It was a lot of fun again, and we'll be happy to do it again when we get the chance. Thank you very much for being here.
Max Weidele
My pleasure. Thank you for having me.
Klaus Mochalski
Bye.
Max Weidele
Ciao.
