OT:ICEFALL vulnerabilities raise few eyebrows, though OT cybersecurity defenses must be bolstered

Following the disclosure by Forescout’s Vedere Labs of the presence of ‘OT:ICEFALL vulnerabilities’ that affect devices from ten OT (operational technology) vendors, there is a consensus that the suboptimal vulnerability management surrounding them and the often-false sense of security offered by certifications significantly complicate OT risk management efforts. 

The 56 OT:ICEFALL vulnerabilities cover insecure engineering protocols, weak cryptography or broken authentication schemes, insecure firmware updates, and remote code execution via native functionality. These ‘insecure by design’ security flaws have been found across manufacturers, including Baker Hughes (Bentley Nevada), Emerson, Honeywell, JTEKT, Motorola, Omron, Phoenix Contact, Siemens, and Yokogawa.

The research also notes that many factors complicate OT risk management, including the certification of vulnerable products, lack of CVE assignment, and supply chains propagating vulnerabilities. Once again, the disclosure of the OT:ICEFALL vulnerabilities reveals that one of the biggest issues facing OT security is not so much the presence of unintentional vulnerabilities, but the persistent absence of basic security controls.

Industrial Cyber reached out to experts in the industrial space to determine how these ‘insecure-by-design’ practices crept into OT products, with 74 percent – of the vulnerable product families identified in the OT:ICEFALL vulnerabilities having some sort of certification regarding their suitability for use in critical OT environments.

“What is deemed secure today can be insecure tomorrow,” Marco Ayala, global director for ICS cybersecurity and sector lead at 1898 & Co., part of Burns & McDonnell, told Industrial Cyber. “Practices that have made it into OT products were built with ease and convenience just like you recall the days of surfing the internet using HTTP, FTP, and Telnet sessions. Many act as though this disclosure is new and unprecedented and yet the seasoned ICS security folks know that coding practices in ICS that had scaled from early on were focused primarily on real-time systems functionality, communications availability, integrity, reliability, and overall repeatability.”

So “when we talk about hardcoded credentials, telnet, FTP, Modbus ‘insecure-by-design’ the vast majority of the industry knows this and for over the last decade, the vendors have been working on providing better security concepts and capability of their systems wrapped around the core function,” according to Ayala. So, the insecure-by-design statements should be stated as ‘slow to adapt’ without affecting operational integrity, he added.

“This is not unique only to OT, although it seems that embracing application security methodologies and tools is more common in IT-driven applications; we’re starting to see this becoming more common in the OT space,” Yair Attar, CTO and co-founder at OTORIO, told Industrial Cyber. 

“Is this change happening fast enough? The clear answer is no, as we still see coverage gaps,” Attar said. “Another factor to consider is that certification should extend beyond the initial device security posture to include how it is installed. The context within the environment that the device is in is just as important,” he added.

Richard Robinson, chief executive officer at Cynalytica, told Industrial Cyber that he wouldn’t necessarily say that the ‘insecure-by-design’ crept into OT products. “One could argue that networks crept into OT. We need to remember that most OT products were initially developed and implemented well before the Internet. At that time cybersecurity was not on the mind of or a requirement for designers or operators. These OT (ICS/SCADA) systems were designed and built to be highly efficient, safe, and resilient and were traditionally in isolated physical environments that were not networked to external routable networks,” he added. 

“Authentication, encryption, and many of the other IT security tools and practices that have evolved since the early days of the Internet were not required in OT environments at that time and the introduction of these IT security approaches, in most cases, inhibited or impaired efficient OT processes,” according to Robinson. “Certification and standards have always been the low bar when expecting that a product will be ‘secure.’  In OT environments there are multiple ways that a user, device, or process can be compromised. Not that certification and standards are entirely ineffective, they just have a limit as to what they can help mitigate when it comes to complete OT security,” he added. 

Certification and standards combined with secure monitoring of process data is a key way to better mitigate ‘insecure-by-design’ consequences, Robinson added.

Assessing if the industry expects any federal action to deal with the insecurity by design identified in OT:ICEFALL vulnerabilities relevant in OT environments, Ayala said he does “not expect any federal action or regulatory however I do believe that vendors and certification bodies continue to work together to identify and address insecure-by-design in new systems offerings.”

Regulatory frameworks and federal actions won’t be able to cover everything, Attar said. “They typically are responsive to an event or scenario that has already happened. Choosing a trusted supplier should be considered and looked at as a differentiator among automation vendors. Once automation vendors are financially incentivized to put more emphasis on their software security development life cycle, this will have a significant impact,” he added.

“Regulatory frameworks can play a key role in identifying what aspects of a critical system need protection and visibility, however, in many industries like Energy the use of technical exceptions or technical feasibility exceptions (TFEs) undermines these frameworks and their intent,” Robinson said. “More federal action to try to address the fallout of this type of research is expected. I am not confident that it will change much unless monetary penalties are appropriately implemented and applied. This is a significant political challenge, however.” 

There is continued resistance to regulations due to the perceived costs and due to the capabilities of industry representatives and lobbyists to influence as well as the recent revelations of security vendors influencing government administrators by pushing their solutions through federal action, according to Robinson.

Given the depth of the insecure-by-design practices identified in OT:ICEFALL vulnerabilities, Ayala said that organizations must assess their ICS/OT environments and have an approach such as Consequence-driven Cyber-informed Engineering (CCE) or an ISA/IEC 62443-3-2 assessment approach. This would help enterprises get “a grasp of their installation and the unmitigated risk and mitigated risks that insecure-by-design and other contributing technologies deemed secure today may have to their bottom line of safety and operational prudence,” he added.

Organizations should focus on proactive periodical or ongoing risk management, evidence-based assessments, and understanding of the overall attack surface of their environments, Attar said. “Both the threat landscape and OT network topologies are changing more dynamically than before; most of the vulnerabilities that were discovered are on lower-level devices, meaning there could be peripheral security controls that should help reduce the likelihood of succeeding in exploiting these vulnerabilities,” he added.

According to Attar, moving towards a risk-based approach is a key to managing these security and digital risks and understanding where an organization should focus its resources.

“There is low lying fruit for organizations to take advantage of; leverage the breadth of low-cost, proven ICS security solutions that use the MITRE ATT&CK for ICS framework and the newly introduced MITRE D3FEND,” Robinson said. “This is a great tool for operators to contextualize their OT environments and understand the Tactics and Techniques of Adversaries and have educated and informed conversations with staff, researchers, and vendors about the threats and risks to their environments to ensure that they are being addressed,” he added.

Robinson also said to securely monitor as many communications in the OT environments as possible, as close to the wire where possible (Level 0 – Level 5). “Implement policies and procedures that pivot away from default installations of vendors (standard hygiene). Leverage Zero Trust for devices that are on routable networks is supported by the vendor or can be introduced through an additional vendor,” he added.

Addressing the biggest takeaways from the OT:ICEFALL vulnerabilities that OT environments can use to help turn the needle and enhance their cybersecurity posture, Attar said that awareness is a significant force in ensuring all stakeholders and organizations pay more attention to cybersecurity risks. 

“Organizations should verify their suppliers more carefully, from legal requirements to security checks during FAT or SAT processes,” according to Attar. “Once the automation stack is within the production, organizations should focus on risk management by identifying the risks, assessing the security controls in place, understanding what is exposed, and prioritizing the mitigation plans accordingly on what matters.”

Ayala said that certifications can vary and are definitely a step in the right direction, especially for ICS/OT. “The testing that occurs looks at everything from network stacks, coding practices of the device under test and for example the practices of the vendor. This includes robustness testing and fuzzing for example amongst other means to assess components under test. This has provided many vendors visibility into hardware and code that required attention and remediation of SDLC lifecycle,” he added. 

“I give companies such as Siemens, Emerson, Yokogawa, Honeywell, and Schneider Electric as being early to the table on certifying their products as certifications started to be realized as a value add,” Ayala said. “Cybersecurity is the journey and a constantly moving destination (moving target),” he added.

Robinson pointed to zero trust and said that organizations must “assume the devices on your network are vulnerable and potentially compromised. Never assume a device is configured correctly or securely. Have a business continuity and disaster recovery plan in place. Exercise it before you have an incident. Know who you will need to contact and how. Be able to restore a system to a known default state and test frequently,” he added.

“Understand which assets you have are the most critical for operations, be it connectivity or control,” according to Robinson. “Monitor and truly understand the behavior of these assets and monitor for changes in those behaviors. Do everything you can to ensure the provenance of your data,” he added.

OT operators will always have the CVE Challenge/Race dilemma in OT environments, Robinson said. “Time from CVE disclosure to time to be able to patch will always have a significant timing gap in OT environments. If not monitored at all levels during this gap you will always have a loss of situational awareness and a loss of confidence in operations. There will always be a new CVE. There will always be a new zero-day,” he added.