According to the FBI, the three most common IT/OT (operational technology) infection vectors are email phishing campaigns (aka BEC or business email compromise), remote desktop protocol (RDP) and software vulnerabilities. While some hackers may intentionally want to bring down your production systems using the second and third vectors, “we’re only in it for the money” is a far better motive for breaking into your IT/OT systems—and what better way to do it than through the phishing emails scattered among the zillions of emails you get every day. Accidentally clicking on one email link can quickly connect you with ransomware—which can not only shut down your business system, but also your production systems if they’re not protected.

Say you can’t afford to pay the ransom? Well, hackers have done their research—not necessarily in the latest and greatest cyber tools to shut down your business, but rather in studying your key personal contacts and the value of your company. Then they determine what you can afford to pay. After all, if they can make a deal that’s affordable, it’s a “win-win” for your company and them. You’re willing to pay and hopefully get the key to the encryption algorithm to unlock your files, and they pocket your money. Oh, and don’t worry, they’ll be back for repeat business unless you find a way to keep them out of your system. This is why the FBI and law enforcement groups discourage paying the ransom as it encourages criminal activity.

“Once inside an organization, ransomware groups and affiliates will often take advantage of the misconfigurations within an organization’s Active Directory security posture,” says Marty Edwards, Tenable deputy CTO of OT/IoT. “Gaining domain privileges provides attackers with the necessary capabilities to distribute their ransomware payloads across the entire network.”

According to the IBM “X-Force Threat Intelligence Index 2022,” ransomware was the number one attack type in 2021, accounting for 21% of all attacks. According to a 2022 SANS Institute survey, 40.8% of OT/industrial control systems (ICS) compromises came down from business IT systems. To safeguard production systems, manufacturers need to isolate OT from IT systems—or at the very least be able to have a quick disconnect when IT systems are infected.

In this article, we look at IT attacks and their potential to spread into OT systems. Unfortunately, OT systems—in and of themselves—are also subject to direct attacks, and we look at methods to remediate potential OT incursions as well as describe some new tools to discover and prevent attacks. As an aside, cyber insurance may be of some help—at least insurers will make sure before underwriting a policy that a potential insured company is up to speed in cybersecurity training and protection—doing its part to minimize cyberattacks.

IT and OT Issues

Nick Creath, Rockwell Automation senior global product manager, has seen the shift in ransomware attackers getting smarter about their targets. “Attackers are doing their research to determine how much ransom to demand.” When hit, manufacturers are not prepared, adds Creath. “We’ve had customers hit by an attack and take a very long time to recover—recovery can be extremely difficult, so manufacturers need to be proactive and plan ahead [to shorten recovery time].” For example, Colonial Pipeline, when its business system was hit by ransomware, shut down the pipeline to prevent the ransomware from hitting its ICS environment. If Colonial had not shut down its pipeline controls, recovery would have taken much longer.

“Smart manufacturing equals connectivity between IT and OT—we all have to think beyond security to resiliency—how can we cut the connections between IT and OT and keep manufacturing?” says Creath.

While the FBI’s findings are quite accurate, says Jesse Newman, IT manager at Concept Systems, a Control System Integrators Association (CSIA) certified member, phishing campaigns have been the most prevalent in non-production networks, though RDP and software vulnerabilities are often the primary weak points in production systems.

But, according to Kevin Romer, principal engineer IT/ICS at E Tech Group, a CSIA member, in the OT/ICS infrastructure there are significantly more vectors used to exploit vulnerabilities due to:

• A lack of IT-savvy staff at the operational sites to install and maintain security standards adapted for OT systems
• A significant installed base of aged PLC and drive control equipment—and panel computers, which are multiple firmware versions behind and no longer able to accept updates
• Engineering stations, operator machine interfaces and historian applications running on older operating systems that are no longer supported and cannot run anti-virus software without special provisions
• Control panels with unmanaged switches—the same switches used in a home network—that still have the default login and password

Simple denial-of-service (DoS) attacks cause a lot of trouble when too much of the plant operation is remote or in the cloud, says Daniel F. Luna, P.E., vice president of engineering at Engineered Energy Solutions (EES), a CSIA member. He describes a remote failure. “I recall just after 9/11, one of our clients had their IT group and domain controllers in the World Trade Center. When it fell, the warehouse in Southern New Jersey could not log in and do work. This also points to non-engineers designing IT systems that work fine under normal conditions but don’t handle partial failures very well.”

Cloud operations are not immune either. At the time of this writing, Microsoft reported sporadic but serious service disruptions to its online Office suite and OneDrive file sharing applications. A hacktivist group claimed responsibility for the distributed denial-of-service (DDoS) attack. According to a BleepingComputer story, the DDoS attack—carried out by a threat actor Microsoft named as Storm-1359—also affected some of Microsoft’s Azure cloud services.

The Human Factor Stretches Across Organizations

Human error is typically at the root of the most prevalent issues facing all organizations, says John Peck, manager of OT security at Gray Solutions, a Gray Company. It’s often in the form of phishing campaigns—whether by email, phone or information left accessible by others, but is also due to other forms of human error such as poor architectural design. Organizations have often implemented the easiest or cheapest option for their environment. It could be things like not implementing high availability or allowing insecure protocols like telnet or http. Often, companies plan to go back and fix these later but rarely do.

Organizations often avoid secure solutions as they do not wish to have the increased burden and/or cost associated with secured systems, adds Peck. Security certainly comes at a cost that is often seen as an overhead and sometimes a pointless expense. This is frequently the viewpoint until a security incident happens, then it becomes a priority.

Third-party vendors are also a common threat vector, says Peck. Vendors pose a risk because organizations often need to grant them access to their networks, systems or sensitive data. This increases the attack surface and introduces potential vulnerabilities. Third-party vendors may not prioritize cybersecurity to the same extent as the primary organization. Organizations have limited control over the security practices and protocols employed by these vendors.

Email and Phishing Campaigns Rely on People’s Mistakes

Inadvertent employee errors, such as clicking on malicious email attachments or divulging passwords, can unintentionally grant access to bad actors, says Jeremy Hulse, executive general manager – cybersecurity, SAGE Group, a CSIA member. “Given the volume of emails employees receive daily and their various tasks, there is pressure to handle the email influx quickly, increasing the risk of a single click causing significant harm to an organization.”

The old adage, “do you want it done quickly, or do you want it done right?” applies to email, says Eric Chambers, senior network/IT specialist at Huffman Engineering, Inc., a CSIA certified member. “Take the time to slow down and digest an email before clicking a link, opening attachments or replying. Management can support both initiatives by offering training and grace. What’s the true cost of ten minutes extra to get through your email compared to the plant shutting down due to ransomware?”

“Education and frequent phishing campaigns by the internal security team to simulate the latest phishing attacks continue to be important,” says Gabriel Sanchez, director of security operations, 1898 & Co. However, these actions alone will have a limited impact over time if they are not integrated into the culture of the organization. Positive reinforcement and simplicity of reporting suspicious emails will help the overall organization combat the ongoing phishing emails.

“One of the most common threat vectors to OT would be the phishing emails that inadvertently allow access into the IT system, but if there is an OT laptop or engineering station that is allowed to [access] both IT and OT networks, the breach has a greater potential for gaining access across the infrastructure,” warns E Tech Group’s Romer. Limiting physical interconnectivity between IT and OT lessens the spread between IT and OT, but unfortunately isn’t always acceptable in management’s quest for Industry 4.0.

“ISA-62443 recommends annual ‘training seminars,’ mandatory for all employees,” says EES’s Luna. “Also, cyber-responsibility starts at the top. Too often I have seen CEOs enter a room, introduce the speaker, and say something like ‘you all need this,’ then leave. If [cybersecurity training] isn’t bought in from the top, it is going to fail.”

Re-emphasizing the necessity of top-down cybersecurity training, Concept System’s Newman says, “Employees will become more engaged in cybersecurity practices and policies when they understand that cybersecurity is a company decision, not an IT directive. Leadership can bridge the gap between the IT department and employees by leading by example, echoing cybersecurity initiatives, communicating employee concerns to the IT department and enforcing written policies.”

Not all social engineering tricks are high-tech like email. Luna suggests a tried-and-true low-tech ruse that attackers still find effective: “Call the switchboard (if you still have one) and try to sell IT services. The reply may be: ‘No thanks. We are using the Crimson IT service. We are covered.’ Now the attacker knows what IT company you use. They can call back and say that they are ‘Harry from Crimson IT. Please do ….’ They’re in.”

E-mail spam filters help a lot, adds Luna. “In addition, every company should use sender policy framework (SPF) protocol to strengthen the integrity of their own email domain. This is a method of assuring that recipients of emails are getting emails from identified and approved senders. Others are filtered out and deleted.”

“Of course, there is still a need for technology (email filtering, sandboxes, etc.), and even more importantly, every company needs well-understood processes that help guide employees when presented with social engineered vishing or smishing attacks,” says Chuck Tommey, digital connectivity executive, Siemens Industry, Inc./Digital Connectivity and Power. “Nothing is better than convincing an employee that they will be backed and in fact celebrated when they respond to a request from the CEO (or someone impersonating the CEO) by following procedures.”

The real issue here is that organizations must create cultures of cybersecurity, adds Tommey. “Everyone needs to understand their personal responsibilities, and that failure to perform can affect the whole organization. This is very similar to the transformation safety programs companies underwent in the 1980s through the 2000s, where we went from single projects or annual trainings to continuous improvement programs with daily emphasis on the importance of safety. This is the model we think is most effective to transform organizational cybersecurity awareness and effectiveness.”

Attack Vectors Beyond Email, RDP and Software Vulnerabilities

Several critical infrastructure cyber incidents have been the result of a general lack of physical security and poor cyber-hygiene, says Siemens’ Tommey. “For example, rock-solid perimeter cyber defenses don’t do much good if we leave buildings unlocked. And simple measures, like limiting remote access only to those who need it, and requiring onsite enabling of that access, can make it much more difficult for a bad actor to gain access and use it for malicious purposes.”

“Honestly, we’ve seen everything from a sticky note with a password on it sitting on a plant floor to machines that have not yet incorporated multifactor authentication or assigned appropriate levels of access,” says Keith Mandachit, P.E., engineering manager at Huffman Engineering. “Though obviously, we’re talking different levels of awareness when it comes to security, each of these is an oversight in terms of security that could allow a hacker to exploit a system. (For a list of potential sources of malware/attacker entry points, see the box, “Common Entry Points for Attackers.”)

Systems with ports that are open to the internet are certainly among the top ways environments are breached, says Gray’s Peck. “This could be things as simple as the company’s SSLVPN solution to someone allowing RDP directly from the internet into their environment. On plant floors, there has also been a common trend for vendors to place devices within the organization’s panels that allow them 24/7 access so that they can assist the organization in the event of an issue. If not controlled and identified carefully, these devices end up as backdoors into the organization’s plant floor environment and often bypass most if not all security controls and provide direct access to the systems the organization is trying to protect the most. These backdoors are often not within the organization’s control to monitor or alert when something has happened or that could be leading to something happening.”

Security incidents at the vendor organization can also bleed over into your organization through these same devices, adds Peck. “PLCs, operator workstations and engineering workstations are often systems that cannot be patched or patched very infrequently and are commonly legacy systems such as Windows 2000 or XP. These become easy targets once a threat actor gets into an environment as there are well-known and documented exploits for most of these.”

When system integrators still see Windows 2000 and XP in facilities, processors are asking for problems—unless perhaps these systems are not connected to anything else in the plant. But if they are connected, they represent a security threat because they are still working with ancient PC-based SMB Version 1 network protocols; current protocols are Version 3.1.1, which have been updated with stronger encryption systems. Turning on SMB 1 protocols in modern servers to communicate with old machines places the servers at a security risk.

“The good news is all the new devices that are available and implemented into the OT infrastructure are Ethernet enabled, meaning they can simply be plugged into the network and they just work—delivering accelerated time to value,” says Rockwell’s Creath. “The bad news is the same. With this efficiency and power comes inherent cyber risk due to the open, unmodified nature of Ethernet.”

OT needs the governance that IT has practiced for decades to ensure a properly segmented network is in place and IT needs to understand that the OT environment is very different from the “carpeted space,” adds Creath. “We’re seeing a key role in the industry aimed at ‘translating’ process requirements into key IT technical implementation strategies. This individual typically sits in OT with a dotted line to IT.”

In addition to this crucial organizational structure, it all begins with asset identification, adds Creath. “You can’t protect what you can’t see, and the majority of our customers have a hybrid installed base of assets from multiple OEMs and of varying ages. Good news is there are a number of solutions available in the market today aimed at facilitating this asset identification requirement, providing rich data contextualization and facilitating protection strategies for the disparate assets.”

In identifying risks to OT, plant engineers need to understand what assets they have and how the assets are interconnected, says Tenable’s Edwards. “It is amazing how many organizations do not have any continuous asset management or visibility solution in place. In today’s threat landscape this is no longer acceptable and you must have visibility into how these systems are configured and what network paths exist to these devices.”

Tools to Thwart Attacks

“The key to detecting and tracking intruders is the combination of tools to provide layered defenses and visibility into networks,” says 1898 & Co.’s Sanchez. Examples include passive network monitoring, endpoint detection and response, firewalls and centralized logging with alerts. Tools today compared to their predecessors from five years ago are different because they are more specialized and automated. Many vendors realized that trying-to-do-everything security was not feasible and, for example, the industry is seeing more tools that specialize in OT.

Intrusion detection systems (IDS) can be implemented to detect threats, says Concept Systems’ Newman. Modern IDS systems will incorporate artificial intelligence (AI) into their threat detection algorithms to provide more accurate results for alerting. If an exploit is detected in a business network, the best option would be to physically disconnect the production network until after the attacker’s method of access has been identified and resolved.

East/west traffic monitoring; AI self-learning of network source, destination, application protocol and behavior tracking; device categorization traffic normalcy checking; and communication baselining are tools today used to detect a new threat or even an unhappy employee who suddenly decides to try to do something malicious, says Matt Smith, network architect IT/ICS, E Tech Group. Years ago, north and south traffic through firewalls and end point antivirus were the primary tools of the trade—although of course still used today, they are not the only tools used to monitor traffic, adds Smith.

Today, a robust security stack consists of next-generation firewalls, a security information and event management (SIEM) tool to help detect, analyze and respond to threats, and security orchestration automation and response (SOAR), which is used to help with the response to security events in an automated manner, says Gray’s Peck.

A properly segmented network with firewall rules allowing only the specific traffic required for business operations is a great place to start, adds Peck. Additionally, disabling unused services, ports and protocols on systems (hardware and software) throughout the network closes unnecessary security gaps adversaries enjoy exploiting. Finally, proper organizational patch management policies and procedures help close many security gaps.

Stopping Attacks Before They Start—Know Your Open Port Exposures

Hackers have a trove of tools at their disposal to find exposed, open ports to the public internet in plant-floor devices, says SAGE Group’s Hulse. To identify open ports before hackers do, plant engineers should implement regular and rigorous testing plans to check for vulnerabilities. Cybersecurity is a constantly evolving field, with new vulnerabilities being discovered regularly. Prompt patching and vulnerability mitigation are crucial.

“Today, it is much harder for a hacker to find open ports on the public internet but, unfortunately, it is still not impossible,” says Siemens’ Tommey. “New equipment comes with security by default, also known as security built-in concepts. So, theoretically, these devices are more secure when taken out of the box. Companies are using open-source tools—like Shodan, nMap and others—to make sure they are not exposed. But, again, security should not rely on any one layer, technique or vendor. Thus, a holistic OT cybersecurity programmatic approach is the best-known methodology to secure OT systems in an ongoing manner.”