Cybersecurity to be baked into product design, development processes, rather than remain afterthought

Global cybersecurity agencies want software manufacturers to take necessary steps to ship products that are secure-by-design and secure-by-default, thereby shifting the balance of cybersecurity risk and revamping their design and development programs. These approaches to product security will help move much of the burden of staying secure to manufacturers and reduce the chances that customers will fall victim to security incidents resulting from misconfigurations, insufficiently fast patching, or many other common issues.

As technology manufacturers make secure-by-design and secure-by-default the focal points of product design and development processes, they must prioritize the integration of product security as a critical prerequisite to features and speed to market. The move works on ensuring that the burden of security does not fall solely on the customer. Software manufacturers should take ownership of the security outcomes of their customer’s purchases and push engineering teams to over time be able to establish a new steady-state rhythm where security is truly designed-in and takes less effort to maintain. 

Software manufacturers must also adopt radical transparency and accountability while working on building the right organizational structure by providing executive-level commitment for software manufacturers to prioritize security as a critical element of product development.  

Industrial Cyber reached out to leading product security experts on how organizations can deploy the principles and approaches for secure-by-design and secure-by-default to ensure that their critical systems are safeguarded and protected. The executives also look into how long it will take to build organization-wide resilience and how feasible the requirements are.

“Organizations starting from scratch should rely on a controls framework and take actions to consistently apply that framework. Such a program needs a holistic strategy for IT and OT,” Megan Samford, vice president and chief product security officer for energy management at Schneider Electric, told Industrial Cyber. “Companies making products and systems should reference IEC 62443 4-1, 4-2, and 3-3 (covering secure development lifecycle, product features, system features, respectively). Asset owners and operators should look for vendors with independent third-party certifications attesting to the security of their products and systems. They may even ask for certification around the deployment itself, which is found in IEC 62443 2-4.”

Samford pointed out that ultimately, this is all about minimizing the attack surface either in the way products/systems are made or the way they are deployed and operated within a customer environment. “To minimize the attack surface in products you need to consider a secure development lifecycle, in addition to security features like secure boot, certificate authority, and event logging. Also, products should come with secure deployment guidance and a hardening guide. Minimizing the attack surface of a system becomes slightly trickier as it requires a threat model of the system, asset inventory, risk assessment and finally secure remote access,” she added.

“It depends on the size and scale of the organization, as well as the state of their IT and OT networks,” according to Samford. “New sites can build in security from the beginning. Nonetheless, most often, they are legacy environments with newer equipment, especially in OT. No matter what, this is a multiyear journey, and you need vendors to go on their journey with you.”

One of the keys to success in this area is around building appropriate processes into procurement cycles for both purchases of new systems as well as renewals of existing subscriptions/maintenance contracts, Paul Griswold, chief product officer for Honeywell OT Cybersecurity, told Industrial Cyber. “As part of this process, companies should leverage product construction details from their vendors (such as software bills of material, or SBOM), evaluate vendor vulnerability disclosure policies and fix release cycles, and put into place continuous monitoring programs to ensure products and systems stay secure over their deployment lifetimes.”

Griswold added that deploying programs of this maturity often requires at least a year, and needs organizational buy-in to establish, maintain and optimize. “Many companies opt to start by adding this level of rigor to the purchase of new solutions, and then expand to cover existing solutions over time.”

Dmitry Raidman, CTO at Cybeats said that it is indeed encouraging to see secure-by-design and secure-by-default principles being emphasized in the context of critical software and hardware. “Organizations can safeguard their critical systems by ensuring that the products they procure have these principles built in rather than afterthought.” 

“Secure-by-default is becoming a baseline expectation, encompassing features such as single sign-on (SSO), multi-factor authentication (MFA), data encryption, strong passwords, secure boot, and TPM, which are crucial for critical software and hardware security,” Raidman told Industrial Cyber. “Many software and device manufacturers have incorporated these concepts into their new products for several years, making secure-by-default capabilities a key differentiator for critical systems.”

However, the challenge lies in dealing with legacy devices that may be 10-15 years old, as these systems often need more processing power to support modern cryptography and compatible hardware, according to Raidman. “In this context, one could argue that more intelligent systems can be more vulnerable, while legacy systems are primitive in their functionality and more secure. Fortunately, we have the Purdue model and air-gapped concepts for managing these technologies. Nonetheless, the new generation of devices being procured must adhere to secure-by-design principles.”

“Building organization-wide resilience can take several months to a few years, depending on factors such as the size and complexity of the organization, the maturity of its security posture, and available resources,” according to Raidman. “The feasibility of meeting secure-by-design and secure-by-default requirements depends on the organization’s commitment to security, including allocating adequate resources, fostering a security-first culture, and collaborating with stakeholders across the organization. By doing so, organizations can significantly reduce the risk of cyberattacks and enhance their overall security posture.”

Leaders in Washington have made it clear – secure-by-design and secure-by-default are the goals, and the software industry has heard the message, Bryan Cowan, a security researcher at Fortress Information Security, told Industrial Cyber. “As we think about those more resilient products coming in the future, we also need to think about how to fix products that are not secure as they stand right now. Tools like SBOMs, give us the means to help users identify problems in their software.” 

Cowan added that “we learned during log4j, SBOMs are a valuable tool in a moment of crisis. If we can’t have secure-by-design and/or secure-by-default software, the next best thing is a roadmap that will help us find where problems exist.”

While the document focuses on placing the onus on vendors to create secure products, it does not address what happens down the line, such as the appropriate implementation of these protective safeguards by asset owners/operators and not turning off these features. The experts address what they see as missing in order to make a more collaborative approach to product security.

Samford said this is correct. “It’s not only about the secure design and shipment of the product, it’s also about how the product is securely configured in the context of the environment, and to what extent the environment itself is secure. To what extent does the introduction of the product change the threat model, the attack surface?” she added.

“Companies must put in place both evaluation criteria and technical programs to ensure that solutions that are purchased and deployed ‘secure by design’ and ‘secure out of the box’ will remain so after deployment,” Griswold said. “This involves continuous monitoring to ensure configurations remain secure, unauthorized programs/code are not injected, and vendor patches are deployed in a timely manner. This requires establishing a solid relationship and regular communication with the vendors and their support/customer success teams, which is not necessarily done by all customers today.”

Raidman said that both vendors and operators share the responsibility for security. “Both parties must collaborate effectively to ensure product security. While vendors must incorporate adequate security measures in their design and implementation, operators also have a significant role in maintaining security.”

For instance, he pointed out that it is crucial that operators promptly apply security patches provided by vendors; otherwise, devices remain vulnerable and unsecured on the network. “Additionally, operators must ensure proper configuration of security features, such as firewalls, as incorrect settings can compromise the system, regardless of the product’s design. Furthermore, operators must avoid inadvertently exposing devices to public networks, as this increases the risk of cyberattacks.”

“Transparency is another critical aspect that both vendors and operators must address. Vendors should be transparent with operators by providing a Software Bill of Materials (SBOM), allowing operators to better understand the components and dependencies within their products,” according to Raidman. “In turn, operators must be responsible for consuming the SBOM to identify risks and implement appropriate mitigating controls. Organizations can significantly improve product security and reduce the risk of cyber threats by fostering a collaborative approach that acknowledges the shared responsibility between vendors and operators and emphasizing transparency through standards like SBOM and VEX.”

One potential challenge for secure-by-default is prioritizing forward-looking security over backward compatibility, Raidman highlighted. “Often, legacy features are included and enabled in products, despite posing risks to product security. A classic example is dropping support for legacy insecure protocols initially designed for serial communication in favor of more secure, modern alternatives.”

“To address this issue, security teams should be empowered to remove insecure features, even if it leads to breaking changes. This may involve replacing many systems and altering architectures that rely on these features,” according to Raidman. “However, it’s crucial that organizations also properly budget for these security improvements, as they can require significant financial resources to implement.”

Cowan said that first and foremost, the White House must adopt policies that encourage the desired secure behaviors. “That starts with strong safe harbor policies that give software makers confidence they can work with security teams and share the ingredients to their products. Safe harbor provisions must include SBOMs and Vulnerability-Exploitability eXchanges, or VEXs, and baseline standards that promote the use of up-to-date software to reduce risks and vulnerabilities across all critical industries – that provide a clear blueprint for suppliers and vendors.” 

The focus should be on security, not liability, according to Cowan. “Strict liability standards tied to cyber performance are a recipe for legal gridlock and a barrier to innovation. Safe harbors are the answer to our nation’s cybersecurity challenges.”

The experts analyze how systems and software product security can be made easy for users to understand and use while remaining effective and efficient. They also look at the mechanisms that must be put in place to ensure that these systems and software manufacturers are continuously monitored and updated.

“I think it comes down to clear product guidance, that isn’t 700 pages long as well as inherit security of the product – demanding from vendors products that have security features already present in them – I reference IEC 62443 4-2, and 3-3 as good examples for product and systems, as well as the Microsoft Seven Qualities of Highly Secure Devices,” Samford said. “We would like to see more interest and market demand from customers in asking for these security features.” 

At a minimum, when somebody is looking for an ‘easy button,’ Samford added that her  answer is “to ask for a min of Security Level 2 for IEC 62443, and honestly, I would like to see more asks for Security Level 3.”

To make systems and software product security easy for users to understand and use while remaining practical and efficient, it is crucial to focus on the user interface (UI) and user experience (UX) design, Raidman said. “Systems should be designed to present information to users in a non-complex manner, allowing them to navigate from one data point to another naturally. For example, when displaying an alert, clicking on it should show all relevant details on one screen. Similarly, if there are 50 alerts of the same type, it’s better to show the alert type and its count rather than repeating the alert 50 times. This approach prevents critical alerts from being hidden at the end of a list due to excessive noise,” he added.

Devices should also use standard protocols and be compatible with general monitoring tools like SIEMs or dashboards for seamless integration, according to Raidman. “When it comes to updates or firmware upgrades, which in critical software should be fully controlled and executed by the operator, these processes should be executed seamlessly and resiliently. For instance, uploading a firmware file should trigger a validation logic to ensure that the firmware is appropriate for the specific device and has proper integrity and signature verification.”

Furthermore, he added that if a system is misconfigured, “the product should display clear warnings or alert messages, such as ‘Telnet protocol is enabled on interface X port 23.’ When a user clicks on the alert, it should direct them to a manual that explains how to disable the insecure feature.”

“We don’t have a choice – we must have both security and ease of use. Improving performance on both sides of that ledger begins with tools that create transparency – creating a greater understanding of the risk while also considering the needs of the users,” Cowan said. “Organizations – especially critical infrastructure companies – are constantly looking to understand the full spectrum of risks they face, but they need more information. Transparency into software from SBOMs, vulnerability advisories like VEX, and attestations on internal security controls make it to have a fuller understanding of risks. A trusted mechanism for sharing of this information between software.”

Based on their experience, the experts throw light on the level of preparedness of software manufacturers to take ownership of the security outcomes of their customer’s purchases and evolve their products accordingly. They also go into whether software manufacturers are on-board or whether they expect some pushback from the joint guidance. 

“I think vendors are on-board, we just want to see a level playing field where if we make the investment, we expect our competitors to make it too,” Samford said. “All OEMs seem to understand that secure products are more serviceable, and ultimately better for the customer, and ultimately for manufacturers – from a brand and quality perspective.”

The difficulty stems from the fact that products are normally the last line of defense, but now they may become the first line of liability, she added.

“While solution vendors are at various places in their maturity in these areas, it would be alarming to see serious pushback against the concepts outlined in the document,” Griswold said. “Some vendors already implement the vast majority of the specifications outlined, especially if they have experience selling into highly regulated industries and/or to multinational governments.” 

He added that those less sophisticated in their secure-by-design processes will be forced to adapt as more customers put in place purchasing requirements in these areas, and then implement continuous evaluation/monitoring programs to ensure vendors remain compliant.

“Based on my experience, a significant number of critical infrastructure, ICS, and OT vendors are already following many principles of secure-by-default or secure-by-design for a while,” Raidman said. “If the requirements focus on new devices and products, I believe that adoption and vendor support will be significant. However, operators need to acknowledge that they may have to allocate budgets and upgrade their aging systems to new models while demanding that security be built-in rather than bolted on. This approach will hopefully create the opportunity and motivate vendors to prioritize security in their products, in order to accelerate selling their newer product lines.”

He added that the pushback may come from vendors if there is an expectation to retrofit legacy devices and solutions that might contain software or hardware that is at the end of support (EOS) or end of life (EOL). “Retrofitting these devices could be resource-intensive and may not be feasible or cost-effective for the vendor. The use of an SBOM could be a great approach to actively monitor and assess the risk of the system components in such cases.”

Additionally, pushback from operators might arise in the form of reluctance to change, especially if they have been running certain systems for over a decade without significant issues by following industry-known security concepts, according to Raidman. “In such cases, it’s essential to educate operators about the evolving threat landscape and the importance of adopting modern security measures to protect their systems and infrastructure. Another driver for adoption by operators could be digital transformation, modern software or devices, and the ROI they will get on such investments. Education and collaboration between the vendors and operators are still key in both of these cases.”

“Last year, industry groups in Washington pushed back against Congress when legislators tried to add language to the National Defense Authorization Act (NDAA) in the last Congress,” according to Cowan. “Now, the software industry is slowly accepting the idea that SBOMs are coming. At the end of the day, we all want to make users safer. We will have an easier time getting there if we can focus on the challenges ahead and not the missteps of the past.”

The experts also look into how organizations can ‘embrace radical transparency and accountability.’ They also discuss the action planning needed to build organizational structures and leadership to achieve these goals.

“As a rule, security should be governed with a set of defined mandates, and everyone should be made accountable to play his/her part. Cybersecurity is a collective play,” Samford said. “With respect to transparency and accountability, organizations need to figure out what their customer’s expectations are in terms of transparency and accountability of the products and services they are providing – we see Software Bill of Materials, third-party security assessments, supply chain, etc.”

She also pointed to the MITRE release of a System of Trust Framework to assess organizations that I highly recommend learning from as it introduces this concept of ‘trust attributes’ – attributes that do or do not instill trust in an organization.

Griswold said that it starts with an investment from both the vendors and the customers. “Customers must require their vendors to be transparent in the construction of their products – especially in terms of the third-party and open-source components used – and insist on technologies like software bills of material to automate the management of vulnerabilities that are disclosed after deployment. In turn, vendors must invest in programs to provide these details on a timely and ongoing basis,” he added.

Organizations can embrace radical transparency and accountability by adhering to regulations and policy changes such as NDAA, ETSI, NERC CIP, IEC 62443, NIST SSDF, and others, which will act as major catalysts for change, Raidman said. “The transparency brought by these policies, such as adopting SBOM, will enable operators to become aware of new vulnerabilities, risks, or potential malicious supply chain attacks almost simultaneously, or even ahead of the vendors. This necessitates vendors to establish a strong DevSecOps foundation and implement continuous Application Security Posture Management (ASPM) as part of their product lifecycle.”

Raidman added that security is not a one-man job. “To build an organizational structure and leadership capable of achieving these goals, organizations must invest in comprehensive security training, adopt security-focused processes, and continuously monitor and improve their security posture. This includes fostering a culture of accountability, promoting cross-functional collaboration, and ensuring security is integral to the decision-making process,” he said.

“If you think about a classic ‘carrot and stick’ approach, right now we’re seeing a lot more stick than carrot. That said, just creating SBOMs is a step towards greater transparency,” Cowan said. “In the first few days after we learned of the log4j vulnerability, everyone wanted SBOMs to see just what might be in their software. Sharing the list of ingredients to a software product is tremendous progress.”

Additionally, Cowan said that the White House’s National Cybersecurity Strategy calls for safe harbor measures that ‘shield from liability companies that securely develop and maintain their software products and services.’ “Companies will need to be transparent to demonstrate they are implementing best practices like those laid out in the NIST Secure Software Development Framework if they expect to be able to protect themselves from multimillion-dollar lawsuits,” he concluded.