The necessity of industrial cybersecurity training and certification has grown with the rise in the frequency and sophistication of ICS and OT attacks on critical infrastructure environments. Successful attacks were largely possible due to a lack of awareness and experience, which could have been prevented by using a well-defined readiness game plan, among control engineers and users of ICS (industrial control systems) and OT (operational technology) environments.
In the first of this two-part series, executives discuss with Industrial Cyber whether personnel involved in OT and ICS networks would require intensive and continuous training in the evolving threat landscape. They also look into how often OT and industrial cybersecurity teams undergo training and skills competence, and the amount of training time and resources that need to be built into the OT cybersecurity staff’s schedule.
“I have seen a constant state of change in OT and industrial control systems throughout my career, and there is no sign of that slowing down,” Tim Conway, technical director – ICS and SCADA programs at SANS Institute told Industrial Cyber. “Continuous investment in training is absolutely essential for an organization to effectively manage cybersecurity risk to their operations environments.”
Joel Langill has been directly involved in industrial automation and control for almost four decades as an engineer that designed, operated and maintain advanced automation solutions.
“I do not believe that the system complexity has not changed significantly since the migration from proprietary to COTS technologies in the late 1990’s,” Langill, founder and managing member at Industrial Control System Cyber Security Institute (ICSCSI) said. “This does, however, not remove the requirement for specialized training on the systems deployed and their unique requirements that typically cannot be obtained without some form of vendor guidance,” he added.
Daniel Wallance, associate partner at McKinsey & Company sees an overall cybersecurity talent shortage relative to demand which is even more true for OT cybersecurity. “Therefore in response industrial companies, cybersecurity service providers and OEM manufacturers are increasing their training programs. This includes partnerships with universities and academic research institutions that include specific cybersecurity courses on OT cybersecurity,” he told Industrial Cyber.
In addition, industrial companies are looking at other options for obtaining cybersecurity capabilities they need such as through third-party cybersecurity providers. “Many cybersecurity providers offer OT cybersecurity capabilities including security monitoring, anomaly detection and incident response,” Wallance added. “These can supplement in-house OT cybersecurity capabilities limited by available talent and/or provide surge support such as through incident response support retainer agreements.”
Acknowledging that OT environments evolve slowly, Chris Sistrunk, technical manager at Mandiant ICS Consulting, said that while some ICS vendors are supporting modern technologies like virtual machines and cloud, in reality, many OT asset owners are slow to adopt new technology. “Many still have unsupported technology like Windows XP and Windows 7 as well as unsupported industrial equipment as well (since it’s still working). I would say that sometimes ICS vendors are making systems more complex…which requires more training (not including cybersecurity training),” he added.
“The thing is that the absolute majority of industrial facilities has already evolved to the degree when their operation and safety are both vulnerable to cyberthreats, however automating cybersecurity measures and controls is still a very hard, slow task,” Evgeny Goncharov, head of Industrial Control Systems Cyber Emergency Response Team (ICS CERT) at Kaspersky told Industrial Cyber. “Areas that are not automated have to be covered and backed up by humans – both the OT staff and the cybersecurity professionals. And even the automated measures and processes are highly dependent on the cybersecurity staff knowledge and skills.”
McKinsey sees creative approaches to developing and training OT cybersecurity talent. “Given the already limited supply of cybersecurity talent, industrial organizations are looking towards recruiting and training OT experts in cybersecurity. Experts working at oil & gas, utility, manufacturing companies or at OEM OT providers have OT expertise but not necessarily OT cybersecurity expertise,” Wallance pointed out. “Therefore cybersecurity organizations train these resources in cybersecurity thus creating new avenues of OT cybersecurity talent.”
Examples including cybersecurity training in OT controls and capabilities, threat detection, incident response, vulnerability identification and remediation, OT secure architecture, and network design. “Once trained, it’s essential to then continually refine and develop the skillset through exercises including cybersecurity red-team exercises, incident response simulations, and cybersecurity wargame exercises,” Wallace added.
Langill said he would be comfortable saying that the vast majority of those responsible for designing, delivering, commissioning, and maintaining industrial automation and control systems have no formal training on industrial cybersecurity. “I believe that everyone that ‘touches’ or ‘depends’ on IACS should have some form of training,” he added. The IACS (industrial automation and control systems) involves a collection of networks, control systems, SCADA systems, and other systems deemed to be vulnerable to OT cyberattacks.
“While I can’t speak about the broader ICS cybersecurity community’s training, I can say that heavily regulated sectors like energy tend to have focused approaches to training for personnel,” Conway said. “Organizations with mature cybersecurity programs will also tend to have active training programs and participate in cybersecurity exercises, which are critical to staying ahead of the constantly changing threat landscape.”
“Generally, for the majority of the good professionals they need a big ‘skill upgrade’ training just once for a particular, new knowledge area (such as incident response and digital forensics, or incident detection and threat hunting, or reverse engineering, or penetration testing, a particular approach, such as fuzzing, or a particular cybersecurity product/technology usage) – to get the initial understanding of the major concepts and the future learning directions,” Kaspersky’s Goncharov said. “Then, they need to work hard on the real-world cases to get the real experience and learn things by themselves.”
Sistrunk found it hard to say in general how often do the OT and industrial cybersecurity teams undergo training and skills up-gradation. “But for regulated sectors like Electric, Nuclear, Critical Chemicals, and now Critical Pipelines, OT cybersecurity training is required annually,” he added.
Assessing how much training time and resources are built into every member of the OT cybersecurity staff’s schedule on a continual basis within the organization, Conway highlighted that across the 16 identified U.S. critical infrastructure sectors, there are only a few with required cybersecurity awareness training and an even smaller number with additional dedicated and recurring cybersecurity training.
“Most critical infrastructure sectors do not have unique requirements for training and certification credentials. Therefore, the training approach pursued would be varied, and unique to each organization, which could range from an impressive program at one organization and a nonexistent program at another organization,” Conway added.
McKinsey’s Wallance said that OT cybersecurity training should consist of two components. “The first consists of developing awareness, knowledge, and expertise on OT cybersecurity tailored to individual roles. Thus someone who is a security operations analyst has the knowledge to detect an adverse OT cybersecurity event while an individual focused on forensics has the ability to identify adverse behavior on an OT network,” according to Wallace.
“On a periodic basis, individuals receive refresh training on the latest threats, vulnerabilities, new OT capabilities, and new OT cybersecurity capabilities in order to stay current and maintain awareness,” he added.
“Second consists of training through cyber simulations, wargame exercises, and other crisis response exercises including OT cybersecurity. The purpose here is not only to build engagement around OT cybersecurity but also to build ‘muscle memory’ on the part of the practitioners so that when an event occurs the response is much more seamless,” Wallace said.
Sistrunk pointed that, “Mandiant is not an OT organization, but our OT security consultants and analysts have to take annual general cybersecurity training. Several of our folks actually write and teach ICS/OT security training classes offered to the public through Mandiant’s Learning and Development catalog,” he added.
Next week, do catch up with the second part of the series where executives discuss whether the lack of training was a reason or contributing factor for the recent cybersecurity incidents such as the SolarWinds supply chain attack and the Oldsmar water plant hack. They also look into whether changes in industrial cybersecurity training and certification have to be made given the increased deployment of digital transformation into the OT environments.