This is the third blog in our series focused on real-world problems in Operational Technology (OT)/ Industrial Control Systems (ICS) cybersecurity learned from our experiences with clients across the globe. The intent of this series is to cast light on common problems found across client environments - which pose both security and operational concerns.
In this blog, we focus on the security implications of what we call ‘redundant’ Engineering Workstations (EWS) stations in level 2 of the Operational Technology (OT) network – an architecture which we’ve encountered numerous times in client environments across multiple industry verticals.
Due to their role as the controller’s configuration interface, EWS’ are considered primary targets by threat actors that seek to inflict long-term production damage. After an attacker has successfully obtained an initial footprint in the OT network, its enumeration and discovery efforts would typically be geared towards singling out the controllers and their respective EWS’. Once such discovery is accomplished, the attacker is able to manipulate and download code from an EWS to a controller at will.
In the course of our deployments, we have documented a fairly common practice of clients maintaining active EWS licenses on numerous endpoints within levels 2/3 of the network. That is to say that a Windows endpoint that mainly operates as an HMI is also configured to have a working copy of EWS installed. In most cases the reason for that would be to drive more flexibility into process management, enabling control engineers to perform configuration changes from various physical machines rather than be confined to a single EWS machine.
The following figures show a machine named HMI-A1. Examining its timeline reveals that HMI-A1 was also used to upload and download configuration files to controllers.
Figure 1: Asset HMI-A1 featuring a typical HMI baseline
Figure 2: Asset HMI-A1 timeline reveals it utilization as an EWS
While this practice may make workflows more efficient, the operational advantages of using this ‘redundant EWS’ practice comes with a considerable security price tag:
- Expansion of Critical Attack-Surface
While for control engineers increasing the number/availability of EWS’ aides in productivity, for an attacker this practice increases the number/availability of highly desired targets. The more of them that exist in the network, the easier it is for an attacker to discover one and leverage it to impact critical processes.
- Baselining an asset’s legitimate behavior
Besides the direct attack-surface expansion, using redundant EWS’ introduces difficulties in baselining an asset’s legitimate behavior. From an anomaly detection perspective, the best practice is to have each asset confined to well defined tasks. Strictly maintaining this confinement makes it easier to detect and eliminate any anomalous/malicious behavior. Without this confinement, establishing true baseline behavior is much more difficult.
- Network Segmentation
Ideally, OT networks should be segmented by logical asset groups. An asset that features a double role, for example HMI\EWS, breaks this logic and reduces the security efficiency of the segmentation project.
Our recommended best practice is to restrict EWS software to dedicated endpoints only, avoiding redundancy as much as possible. We advise clients to end the practice of “redundant EWS” configuration. Doing so enables both the networking team to proactively enhance the network’s cyber resiliency through sound segmentation, and the security team to closely monitor and guard the endpoints attackers consider as their top targets.