How do you secure the smart grid infrastructure?

Klaus Mochalski

Hello, and welcome to a new episode of OT Security Made Simple. I'm Klaus Mochalski, founder of Rhebo. My guest today is Zeek Muratovic. He is Director of Security Solutions at Landis+Gyr. Zeek, please introduce yourself to our guests and tell us a little bit about what you're doing at Landis+Gyr, and then we'll get to what you've experienced lately with some of the customers that you visited.

Zeek Muratovic

Yeah, absolutely. Yeah. So thanks for having me today. My name is Zeek Muratovic. I've been with Landis+Gyr for a little bit over a year now, but my background is strictly in cybersecurity. So, for the past 17 years, I was involved across many different industry verticals, helping customers identify the threat landscape that they have, see where the security holes are, build vulnerability management programs, SIEM implementations, et cetera, et cetera.

And now, I'm very heavily focused in the utility space with Landis+Gyr. For the past year, I've been doing a lot of traveling, talking to a lot of customers, helping them assess their security posture and how well-prepared they are to receive the threats that we see every day in the news, ransomware attacks, anything like that. It's definitely been a very, very busy past year.

Klaus Mochalski

That's very interesting. It seems, if this is correct to say, that you transitioned a bit from dealing mostly with IT security into the field of OT security.

Zeek Muratovic

Yeah.

Klaus Mochalski

What are some of the changes that you have observed over the past months while doing that?

Zeek Muratovic

What I've observed is that OT now is where IT was about 15, 16 years ago. Very reluctant to invest into security, they don't know much about security. Talking about the cloud is a complete NO for a lot of our utility customers. But there are also quite a few of them who are warming up to the idea.

There's definitely a shift that is coming slowly to OT where they are realizing that security needs to be taken very seriously. That there are a lot of threats out there currently targeting utilities because the bad guys are aware that the OT industry isn't really investing into security as much as it should.

Klaus Mochalski

But when you talk to customers, especially in the utility field... I guess everybody today knows that they have to deal with security, specifically with OT security, and they're doing this. What are the things that you're seeing? How do they usually start? What's their level of awareness, and what are they asking for from their vendors?

Zeek Muratovic

In the US, this past year, NERC CIP has been a heavy push with regulations, penalties, and monetary penalties [which] are driving [customers] to get more into security, look at products, talk to vendors. So NERC CIP has been a big driver. There are certain companies that have IT departments that somewhat overlook their OT side, but most of them don't.

We have OT guys that really don't have much IT experience starting to slowly getting into security, though they really don't understand it. They're asking questions, how can we gain visibility into our network? What do we need to do? They're looking at us, at Landis+Gyr, for example, and Rhebo, to guide them through this entire process. They just don't have anybody with experience that can tell and point them in the right direction. To them, it's... We do have customers that really don't know how to start, where to start, even with the basics. And then we have some customers, again, as I mentioned, they have a security IT team that is guiding them.

Klaus Mochalski

But they're generally still at the beginning of their security journey, and they still need strong guidance, I assume.

Zeek Muratovic

Right. Very early stages.

Klaus Mochalski

When we look at the specific field that Landis+Gyr is serving, we're talking about utility customers. The offering is an advanced metering infrastructure that needs to be secured. I guess everybody understands this now. To help our listeners understand, can you explain a little bit what the AMI, so the advanced metering infrastructure, comprises, what the main components are, and what needs protection? Then where do we start protecting these key components?

Zeek Muratovic

Really, what we are focusing on is securing the entire distribution operation, AMI being a part of it. But we also have to look at the substations, then the AMI, and then the edge devices. For AMI on its own, Landis+Gyr has a solution called Command Center. That Command Center is an application that gets all the information fed from all of the meters that are in the field.

Klaus Mochalski

It's a management system for the meters. You have many meters, hundreds of thousands, potentially millions. Then you have the control center, which is a central management system.

Zeek Muratovic

That management system will tell you how much power that meter has been using and manages that. But what the operators, the engineers, analysts, whatever the title may be, who are looking at command center, really don't know is what to look for for security alerts. We have created an application called [AMI Protect] that monitors events in real-time and alerts the SOC or the administrator that there is a potential cyber risk based on the logs that we are seeing from the configuration database, application database.

Klaus Mochalski

It means the data have probably always been there, but it has not been properly utilized for a security monitoring process.

Zeek Muratovic

Yeah. What you will find also is a lot of these applications will give you the necessary data, though it's what you do with that data. When this application was built 13, 14 years ago, security wasn't top of mind for utilities. The way these logs have been generated, are being generated wasn't really... There's no effective way to pull this out and correlate them to let you know that there's some anomaly or something one-off that is currently happening in the environment that could be a potential big risk.

Klaus Mochalski

Right. This actually sounds like a rather achievable task starting at this central location, at an application that provides log data that is relevant to all the smart meters in the field. But you also mentioned that the smart meter infrastructure, the AMI, is part of a bigger infrastructure. How does this security monitoring at the control center at the central management system, tie in with the wider security tools and controls that an infrastructure operator utilizes today?

Zeek Muratovic

It's a part of the big three that I mentioned: the substation, the AMI and the edge. Not all customers will have all three components. So, this solution has been designed to give customers visibility into what they have. Typically, a customer today has substations they can monitor for security risks. They have the head end system, the Command Center for the AMI. Now, the new thing that's coming onto the market are smart meters that are capable of having applications installed on them to give [...] the utility companies more visibility into the meter and what's going on, energy manipulation, etc.

But with that, the issue becomes: Well, if my customers can create apps, are they going to create them secure? Are they going to write secure code? Our edge solution that sits on the actual meters itself can identify threats that could be exposed on those applications that are sitting on the actual meters. But not just meters, we can talk about batteries, solar panels, car chargers, anything that sits on a consumer's home that they use that could manipulate energy. You have this entire space that we should be monitoring to give us a true sense of security.

Because just like your house, if you don't know that your doors are locked or your windows aren't open, you don't know that you're secure. We install alarms in our homes. If somebody opens a door [or a window], there's an alarm. It's a very similar basic, as IT security guys call it, basic visibility just to know what you really have.

Klaus Mochalski

In a sense, and securing all the infrastructure elements is important, or at least looking at the risk of each of the elements and then decide what level of protection they require.

But if I put myself in the shoes of an infrastructure operator, it also seems this sounds quite daunting. There are so many components to secure, and now I get more components. Maybe I started securing my smart grid infrastructure end-to-end, but now I'm looking at my substations that became digitalized. Maybe I'm offering also something like EV charging stations for residential use connected to battery storage systems, connecting to PV systems, and they all connect to the grid.

If you need to – and we all agree, we need to – secure them all, does it mean that the effort for an infrastructure operator just gets multiplied, or are there some efficiency benefits that they can utilize for this end-to-end monitoring that you describe?

Zeek Muratovic

Yeah, well, the solutions are really designed to make their lives easier. There's prioritization. Not everything that we consider as a bad thing is easily exploited. In our world, we have low, medium, high, critical risk, or one-to-five, whatever system you're using. We can pinpoint where you can fix certain things to prevent easy access.

You will always have some misconfigurations here or there. There's always going to be an outdated system, but they're really, really difficult to exploit sometimes. With our solution, we can guide you and help you prioritize what needs to be fixed first. So, you don't get overwhelmed with alert fatigue, all these alerts popping up, fix this, so everything is on fire.

If you probably didn't have visibility into this for the past 10 years, it can be overwhelming. But this is a business decision that has to be made. Do I invest in maybe an additional employee that can help us address this or a consultant, or are we risking being on the front-page news tomorrow? It's a business decision that has to be made. But we can provide that information to our customers. As you know, we do great [security] assessments, and we give them the output of those assessments. Based on that, they can make a very good financial decision, whether a solution like this, an investment in a solution like this exceeds [the cost of] an exposure to a threat.

Klaus Mochalski

Looking at customers again, what's your experience of how they handle the protection chain for their infrastructure? If they add new elements, Is it that they have a team protecting their electrical substations and then a separate team protecting their smart meter infrastructure and yet another team protecting their EV charging fleet, or is this already more integrated than that?

Zeek Muratovic

No, it's very separated, the OT teams and the companies. They need to bring their teams together. It was interesting, I was just with one customer, and we had Manufacturing at this meeting, we had the Security guys, we had the IT guys. And talking about these solutions, back and forth, one team member mentions to the other team member: I didn't know we had a solution. We need it over here! It exists in the same company, but they are not working together to help each other out. They're very segregated in a sense like that. [For] a lot of these bigger companies getting OT and IT to work together is still a little bit of a challenge.

Klaus Mochalski

It's funny if I think that we are still talking about that because we've been talking about the IT-OT convergence, the difficulty bringing the two together out in the field, for many, many years now, and it still seems to be a problem. But also, in a way, it's good news because it also means there is lots to win in terms of operational efficiency with regard to the cybersecurity. And more mature customers today don't really separate IT and OT security anymore. For them, it's just security of their business because at the end, an attack, no matter where it hits, is a threat against their business, and it needs to be treated as such no matter where it hits. It really calls for this integration for efficiency purposes.

But you can also make your security controls more effective in the sense that an attack, especially an OT attack, usually doesn't happen on the OT side. It doesn't usually originate on the OT side. But the IT systems, the firewalls, for instance, probably have seen it hours, sometimes even days before it hit the OT systems. Integrating them and having a common security dashboard and common visibility is certainly a big step forward also in terms of cybersecurity control, efficiency, effectiveness.

Zeek Muratovic

Right. I think what I've seen, the disconnect is that the IT guys will see a security alert, and they are quick to fix it. But the OT guys are like, if the fix goes bad, we're shut off.  

Klaus Mochalski

Robert, don't touch it.  

Zeek Muratovic

Crowdstrike, for example, an update, shut off quite a few manufacturers’ line. The whole distribution can go down. The OT guys are a little bit hesitant on that side. But I'm sure they will find a way sooner or later.

Klaus Mochalski

I think we can tell them that all of the products in the market today are rather mature. They have been around for many, many years. They have been field-tested by many other companies, so you shouldn't be afraid of using this technology anymore.

Looking at some of the customer engagements you recently had, what would be your recommendations for the first steps for customers starting on this OT security – maybe IT/OT convergence – journey?

Zeek Muratovic

Visibility, first of all. Again, you don't know where you are. Do a gap assessment. We help our customers do that all the time. Takes a couple of hours of their time. From there, you know where you stand, you'll know how bad it is. This is the information, as I mentioned earlier, that can be delivered to upper management and say: Hey, this is our current state. These are the risks. We can even go as far down like, these are the certain risks that have been exploited in the previous attacks just down the street from us.

We can provide them with the information necessary to make a good decision, whether they invest in a solution like this or have somebody manage a solution for them, like a managed services provider. There are always options for them there. They're not alone in this battle. We can always guide them to a vendor, or we can do it for them. There are definitely many options out there. A lot of customers that we talk to that are really far ahead in their security journey, they're also willing to help the smaller guys, too. We've done that quite a few times.

If we speak to a customer, we tell them what one of our more advanced customers is doing, and we arrange a phone call together and have the [more mature] customer tell the [less mature] customer, which always means a little bit more than talking just to a vendor. They help guide them as well.

Building those relationships, bringing different companies together into the same room, having a conversations like we do at the Exchange Conference where all our customers come together, share ideas. It's a really good step to get good information. I don't [need to] re-invent a wheel again. Somebody's doing this already. Somebody has a fix. You don't need to do it again. You can just do what they are doing.

Klaus Mochalski

That's a clear advice. Talk to your peers, get a risk assessment in place. You can help there.

Zeek Muratovic

Yes.

Klaus Mochalski

And then get the visibility to make sure that you take the proper and the correct next steps and don't do them blindly.

Zeek Muratovic

Yeah, exactly.

Klaus Mochalski

Very good. Thank you very much for the insightful discussion. See you around here on this podcast.

Zeek Muratovic

Thank you. Bye. Bye.

‍