industrial incident response

Cybersecurity incidents on large-scale industrial operations are leading to more comprehensive industrial incident responses, as part of the organization’s cybersecurity arsenal. Creating a strategic plan for how the organization should respond to cybersecurity incidents, and having an effective and tested industrial incident response in place has become a necessity for every industrial company.

The industrial incident response must include proactive elements such as planning, incident prevention, and post-incident analysis/forensics, while reactive elements would focus on detecting and managing an incident once it occurs. The reactive measures include detection, containment, remediation, and recovery and restoration, and are carried out under severe time constraints and great visibility. 

While every organization aims at a smooth, planned industrial incident response resulting in minimal impact to its operations, accomplishing this will require plans and procedures that are in place and tested before an ‘actual’ cyber incident occurs. Incorrect industrial incident response may result in chaotic and haphazard actions that are ineffective or increase damage. 

To deal with the changing cybersecurity landscape, industrial asset owners and operators must adopt new strategies that enhance visibility, logically reverse the trend of separating IT networks from their OT networks, apart from adopting and implementing relevant frameworks from the National Institute of Standards and Technology (NIST), MITRE ATT&CK, ISA99 or other industrial control systems (ICS) standards-based cybersecurity best practices, according to Dino Busalachi, Chief Technology Officer of Velta Technology. 

Digital Transformation / Industry 4.0 initiatives and other uncontrollable events like COVID have created a connectivity rush of access into highly sensitive critical ICS with little to no thought of how to protect the ICS environment. “Blowing holes in firewalls to allow access, allowing RDP, thinking VPN’s were sufficient, 3rd party access from anywhere in the world, etc…,” Busalachi said. “Let’s face it the days of air-gapping of ICS are gone and have been for some time, those mythical barriers simply do not exist. Anyone stating otherwise is stating a falsehood,” he added.

Dino Busalachi, CTO, Velta Technology.

Citing the recent Colonial Pipeline ransomware attack and the Molson Coors attack, Busalachi said that the companies had the opportunity to bring greater visibility into their environment and they punted. 

“I’m sure the call went something like this between IT & OT. IT called up OT and conveyed what was happening to them, probably ask OT how are you doing? What do you think OT told them? My guess is they had a deer in the headlight look and responded with a WTH! How would they know? They have no visibility into their ICS to determine if there are threat signature malware running around in their environment. I mean, after all, OT uses IT technologies like Microsoft for their HMI’s and application ICS servers,” he added.

“Time and time again, organizations can’t effectively respond and understand intrusions because they don’t have the required data,” Ben Miller, Dragos’ vice president for services and R&D, responded. “The community needs to gain visibility within the industrial networks and not just along the perimeter,” he added.

Dick Bussiere, Tenable’s Technical Director for the APAC region also maintained that OT operators need to take a full inventory of all assets, firmware version, patch level, state, configuration and vulnerability positions of everything that is present within the operational technology infrastructure. “Knowing the state, version, configuration, and vulnerability position of all assets within the OT estate is critical, as you cannot defend what you do not know about. A full inventory of communications patterns within the OT environment for the purposes of baselining what is “normal” and identifying suspicious activities must also be performed,” Bussiere informed Industrial Cyber. “This is important because, within an OT environment, communications are predictable and use a finite number of OT-specific protocols. Constant monitoring of asset inventory and network activity is essential as no environment ever remains completely static.” 

Dick Bussiere, Technical Director APAC, Tenable.

The organizational structure of ICS frameworks is usually made up of a large number of control loops, human-machine interfaces (HMIs), and remote diagnostics and maintenance tools built using an array of network protocols on layered network architecture. Unless appropriate security controls are deployed, security vulnerabilities can expose all levels of the ICS network architecture to complexity-induced error, adversaries, and a variety of cyber threats, forcing organizations to include the appropriate industrial incident response in their framework to deal with such exploits.

Adoption of ICS security controls is typically driven by specific industries or verticals that have regulatory requirements where non-compliance can be costly and punitive, Velta’s Busalachi said. “Otherwise, most organizations do just enough to keep off the radar screen and regulatory requirements lag behind technological advancements. Just because the tools exist does not mean companies use them and many who do are just scratching the surface,” he added.

With many sites having a security reference architecture that implements mostly preventative controls, Dragos’ Miller says that over time these controls may erode. “We regularly see overly permissive ‘test’ firewall rules that never are revoked- for instance. And security isn’t a static state but is always advancing as the adversaries find new vulnerabilities and leverage new tactics; we must adopt detection and response measures and not just rely on preventative measures,” he added. 

The degree of security controls taken by an organization is subject to either industry-specific regulations or government mandates, Tenable’s Bussiere said. “Concrete examples include AESCSF in Australia and the Singapore CCoP. And while mandates like these raise the level of security, it should be viewed as the baseline. Despite this, cyberattackers are still successful because there may be gaps in coverage, unenforced or nonexistent security policies, and unknown or poorly secured intersections between OT infrastructure, converged IT/OT environments, and exterior networks,” he added.

There is a need for industrial and manufacturing organizations to make changes to their network architecture, in order to deal with cybersecurity incidents in the evolving landscape. 

“Logically separate the networks,” Velta’s Busalachi said. “Look OT is going to have to step up and own their environment. They need to look at cybersecurity for ICS like their safety. We call it ‘Digital Safety’. Safety is everyone’s responsibility. By changing the vernacular to a term that encompasses OT, like Digital Safety and Process Integrity the role will change,” he added.

He further added that the IDMZ 3.5 using the ISA95 or Purdue model is where this battle ensues and OT will need to become just as proficient with network technologies as IT. “Just because the technologies have converged, it does not mean the IT and OT organization is in alignment. I do not foresee a time where IT will ever be responsible for plant floor operations or production. These functions and roles will always land squarely on OT shoulders. Don’t get me wrong IT has a place, they just cannot lead in the OT space,” Busalachi added.

Tenable’s Bussiere also said that several changes should be considered, including clearly defined policies, roles and responsibilities for those within the plant, and enforcement of these policies, and training for people in the OT world needs to understand the threat surface and attack vectors that constantly evolve. 

He also suggested “careful asset assessment and inventory monitoring that provides deep situational awareness across the entire organization. The employment of network segregation in ‘north and south’ and ‘East-West to help stop attacks from propagating,” he added.

Bussiere also recommended, “where appropriate, the employment of data diode technology and leveraging compensating controls.” Data diode technology is a network communication device that works as a unidirectional security gateway between segmented networks delivering network integrity by preventing intrusion, in addition to network confidentiality by protecting security-sensitive information.

Preventive activities based on cybersecurity risk assessments can help reduce the number of incidents. “Most organizations don’t effectively understand how to measure, manage or communicate cyber risks within industrial environments. Many are just starting their journeys in understanding and securing OT/ICS and haven’t tied them into the organization’s overall risk program,” Dragos’ Miller said. 

“That’s not to say that individuals within the organization may not understand them- they do. But the broader organization itself is still grappling with collectively understanding risks. They can start with activities they already do – like Process Hazard Analysis (PHA) and combine that with threat profiles to get a combined understanding of the impact and how an attack could be executed to generate that impact,” he added.

Industrial organizations must underscore the importance of the risk assessment and management process as being all-inclusive – from the executive suite to the shop floor. “Everybody has a role to play in OT security – it is not just the domain of the IT team. This is especially relevant given the increasing amount of converged IT/OT environments,” Tenable’s Bussiere said. 

“It’s important to note that this is a perpetual process and not a one-off exercise. Additionally, the risk assessment process within an OT environment is a bit different compared to an IT environment, because it must consider the safety of life, limb and property are one of the most important parameters, the physical impact of any threat since cyber-physical assets are being manipulated, and the impact of propagation of an attack from one system or area to another, either digitally or physically,” Bussiere added.

The SolarWinds and Kaseya supply chain attacks have proven that industrial environments are increasingly becoming more interconnected, and include connectivity to third-party firms such as suppliers and systems integrators. “Industrial environments are increasingly becoming more interconnected and this includes connectivity to third party firms such as suppliers and systems integrators,” according to Miller. 

“Some of these third party connections are very controlled while others are not- there’s no one model of how this is done. it’s on the asset owners to trust but verify and it’s on the suppliers to protect their customers,” he added.

Tenable’s Bussiere suggested that organizations must minimize, monitor, and act as part of their industrial incident response. “The OT environment should only contain the bare minimum of software that’s required to operate the processes. This should be tightly enforced at the lower levels of the Purdue model, at Layers 1-3. Doing so helps to mitigate the risk of illegitimate software being present within these mission-critical environments,” he said. 

“Keep up to date on vulnerability information from sources such as ICS CERT. Ensure that a proactive OT security program is in place that provides the right level of visibility, security and control across the converged attack surface so that threats and vulnerabilities can be detected and mitigated before damage occurs,” Bussiere added.

Providing an interesting new angle, Busalachi said that the OT environment needs to develop its supply chain of technology partners that specialize in OT services and technology. “Relying on IT as you can tell is not benefiting OT. IT’s view of the OT environment is narrow and does not take into account OT-specific tenants for operating in an industrial environment, and particularly safety! OT needs to own their space and take a leadership role and quit deferring and deflecting to IT to ‘handle cybersecurity’ requirements,” he added.

“The OT / ICS environment is in constant flux and needs tools to keep up change to improve not only the cybersecurity posture but assist with operational resiliency,” according to Velta’s Busalachi. “The era of periodic assessments (snapshot) will need to come to an end. The complacency of installing a single firewall only controlled by IT, between IT and OT networks is not good enough, it’s a start, but barely moves the needles,” he added.

With an increasing threat from more sophisticated and motivated cyber attackers and the growing integration of ICS into corporate networks, it is imperative for industrial and manufacturing organizations to incorporate appropriate industrial incident response of critical facilities. Doing so will prevent many issues from arising and will allow the facility to respond, if necessary, in a successful and effective manner. While learning from past incidents helps strengthen the system against potential attacks, it is important to have a cybersecurity program that ensures monitoring and remediation are in place and being acted on.

As Dragos’ Miller says, “An organization can have the best incident response plan but if they don’t have the right asset inventories, software inventories, logging, and network threat detection then their response plans immediately grind to a halt.”

Malware, vulnerabilities targeting OT systems surge

Increasing threats of vulnerabilities are steadily rising, particularly in sensitive areas such as OT systems and network devices, putting vital infrastructure at risk, according to data released by Skybox Research

Share on facebook
Share on google
Share on twitter
Share on linkedin
Share on whatsapp


Join over 5,000 Industrial OT & Cyber professionals

Weekly Newsletter direct to your inbox