You’ll soon be celebrating your control system’s 30-year anniversary, and it’s been, for the most part, a smooth ride. There might have been a situation when dozens of alarms showed up on the shift operator’s supervisory HMI—and you had to call in your maintenance and engineering staff at 3 a.m. only to find a defective sensor, stuck valve and an overheated motor.

Or once, when trying to balance year-end equations, missing ingredients and missing product couldn’t be accounted for, and the books showed many more dollars spent on ingredients that couldn’t be recouped in product. Quality assurance kept finding batches were a little off, but your control system didn’t detect any anomalies.

Article Index:

Since it’s a proprietary system, it has kept intruders out and prevented hacks, right? But because it lacks communication capabilities, unlike higher-level ERP and quality systems, this might explain why you can’t account for slightly out-of-spec products and missing ingredients. The outdated system is incapable of running some advanced control algorithms (like model predictive control) that might have caught ingredient over-usage or too dry a product.

While you might wait until a small motor burns out before you replace it, do you want to take a similar chance with a control system failure? After all, planning, installing and running trials on a new system are a little more time consuming than replacing a motor.


Aging systems: Everybody’s headache and responsibility

Keeping control systems from growing old before their time has many challenges. Oftentimes, it’ll come down to underlying operating systems. Windows XP is no longer supported, and neither is Windows Server 2003, both of which were susceptible to the recent cyberattack called WannaCry.

Bühler Aeroglide, a builder of high-end dryers, roasters, ovens and coolers, works hard to stay current. But what drives food processors to move ahead?

“Processing advantages due to availability and cost reduction in relevant [new] technology,” says Doug Beloskur, product manager automation. Improved process control can drive down energy use, scrap and/or downtime.

Also, updating a system can be spurred by a crisis. For example, Beloskur saw instances of obsolete infrastructure exposed during the WannaCry cyberattack that drove investments in updating systems. But even knowing the issues, some processors still don’t understand the risks involved with older systems.

“The two obvious key issues are lack of available replacement hardware for aging systems and security vulnerabilities for non-supported computer operating systems,” says Kevin Moss, group manager at McEnery Automation, a certified member of the Control System Integrators Association (CSIA).

He says these risks typically don’t convince small to mid-sized food and beverage clients to take action. Usually, these companies put off replacement until a capital project requires significant process changes that are not possible with existing systems.

“It specifically makes sense to upgrade when you have end-of-life components in your system … anything where you cannot go out and buy new parts off the shelf. That presents an inherent risk to your operation,” says Shawn Mott, president and founder, Digitronik Labs, a CSIA member. “We’ve seen customers experience week-long shutdowns because of a legacy component failing and not being able to procure a replacement—and so now, there’s a lot of lead time added on for redesign costs on the fly.”

Mott’s advice: Do a planned obsolescence study, examine existing equipment and have the systems integrator (SI) make recommendations on how best to mitigate risk.

The most logical reason for doing an upgrade/conversion is when a processor has considered its options and the unexpected downtime and costs associated with a not-so-simple conversion, says Steve Pflantz, senior instrumentation & controls engineer and president of the International Society of Automation (ISA).

“With a proper plant maintenance program, you really should never get to that point,” he says. “When a plant no longer has ready access to spare parts, it should have a near-term plan to replace the equipment. The lowest-risk way to do so is to plan the downtime versus unexpected downtime and loss of production.”

A lack of available spare parts—due to running older control systems past their supported lifecycle—puts companies’ production at risk.

“If a client is purchasing spare parts on eBay or any other second-hand source, they are risking their companies’ production on untrusted and unknown sources,” says Chris Wahl, project manager at Stone Technologies, a certified CSIA member.

Second, many older PLC programming applications simply are not supported on modern operating systems, which causes a conflict with the IT department, as it wants to stay current.

“From a software standpoint, we strongly encourage users to stay on a supported version,” says Alicia Millinger, GE Digital product marketing manager, automation software. “Even if a company is not ready to modernize, today’s security requirements make supported software versions imperative. This includes all HMI/SCADA-related software, as an example, as well as the OS.”

This is a basic, but often neglected, way to improve the security of a system. Modern software is built to be secure by design, adds Millinger.

Modern control systems take advantage of better network integration capabilities to provide more process data, alarming, reporting and interfacing to desktop applications. Modern programming systems allow faster development of projects, and increased control system memory allows for more system flexibility, says Wahl. Many older controllers have no memory left to accommodate new projects.

Last, but not least, many companies are finding a generation of maintenance technicians who supported the older system are retiring, and younger engineers are not familiar with the older technologies. For system integrators and engineering houses like CRB, staying current means keeping in touch with automation suppliers.

“We work with the major [automation] system vendors to know what their plans are for system migrations, phasing out of one model to the next,” says CRB’s Pflantz. “This allows us to advise our clients early on as to what they will have to deal with when implementing a project that will require changes or additions to an existing system.”


Role of automation suppliers

Automation suppliers recognize that processors want to use equipment for as long as they can, and they work with both end users and SIs to plan when upgrades are needed and how they can be managed.

“Many of our end-user customers have had systems that have been running successfully for years, and it can sometimes be hard to convince them to update their systems, because they don’t want to interrupt production for any length of time at all,” says Dave Cameron, Bosch Rexroth automation and electrification solutions director of sales.

The automation supplier, a partner member of the CSIA, keeps its parts available for at least 10 years after the parts have been announced as service items only, so often, it’s easier for the user to keep on ordering spare parts. But when these systems reach a critical point, processors must decide whether to retrofit or replace. In many cases, the cost of a new machine leads to a retrofit, says Cameron.

“Typically, we need to ask our users where they are on their journey with automation,” says Aaron Crews, director, global modernization solutions, Emerson Automation Solutions.

There is a cost-benefit ratio that a user needs to take a hard look at and consider. Is keeping the automation system healthy becoming more expensive? Typically, users will notice these situations: Replacement parts are expensive or non-existent. Legacy expertise is expensive and hard to find. Onsite support dwindles as individuals retire. Expenses get higher to support a system with decreasing value.

When food processors opt for replacing old systems with new, Emerson uses a method called Project Certainty, which is an approach for transforming capital project execution. It applies technologies and rethinks work processes to help eliminate cost, reduce complexity and better accommodate changes to maximize capital efficiency. Since a project affects an existing operation, risk reduction is most important. Emerson’s brownfield solutions include checklists to ensure there are no project surprises, software tools to reduce human error in the controls conversion, wiring solutions to assist changeover and reduce downtime, hot cutovers to eliminate downtime and integration solutions at the I/O, controller and HMI levels for phasing the projects and easing the transition for operators, says Crews.

Automation suppliers also work with machine builders that are faced with similar but differing upgrade issues.

“We are always updating products with great new capabilities, and it’s up to us to demonstrate to the machine builder that these new features add value for their end users,” says Bosch’s Cameron.

There’s also another design decision: If an OEM is making a series machine (where several models make up a series), switching to new hardware and software is more complicated than it is for a builder that specializes in custom equipment or small series machines.

Omron’s strategy is to design robust products that can survive the long-term rigors of production environments, says Laura Studwell, industry marketing manager. Many Omron controllers have been on the market, fully supported for more than 25 years. However, when any component is being discontinued, Omron takes a phased approach. Customers are given a one-year “last time buy” notice, yet the company will continue to repair and service that component for seven years.

Omron, a CSIA partner member, will work with machine builders to determine if legacy automation equipment is meeting the OEE specifications desired. If the equipment isn’t, e.g., because of an increase in unplanned stops or defects or increased downtime, then it’s time to consider newer controllers that can help the equipment achieve better OEE scores.

“As plants look to improve the reliability and efficiency of equipment, there is a need to integrate them with modern operations and asset management approaches—for example, we need production counts, machine state, fault codes, etc. to drive OEE metrics for a machine,” says Keith Chambers, director, operations and execution systems, Schneider Electric, software.

At brownfield sites, an extra effort is needed to determine if the existing controller can:

  • Allow connection
  • Provide the needed registers for integration
  • Reliably update the registers to reflect the true state of the machine and its production counts.

If any of these conditions aren’t present, one option is to add a supervisory PLC or data concentrator to connect the existing automation, says Chambers.

Cloud computing and low-cost sensors are newer options for integrating controllers with other brownfield machines. These Internet-of-Things solutions can be ideal for manual or semi-manual operations that have limited local control or in situations where remote plant sites exist and have limited support staff.

Automation suppliers can offer their services to both end-user processors and SIs to gauge the appropriateness of system upgrades.

“Connected Services from Rockwell Automation can collect hardware and software data across multiple sites in weeks,” says John Riess, global marketing lead, integrated architecture. This offering gives assessments of current risks and opportunities, as well as costs of new projects.

Rockwell’s Allen-Bradley PLC-5, long a workhorse in many plants, may be getting a little long in the tooth. However, Reiss says migrating from the PLC-5 to a ControlLogix controller can be done in a matter of minutes, which limits the impact on current operations.

Automation suppliers don’t often come out willy-nilly with hardware upgrades for the sake of upgrading.

“From a hardware perspective, we deal with many issues that drive obsolescence,” says Emerson’s Crews.

For example, end-of-life and regulatory issues (e.g., RoHS, reduction of hazardous substances) with electronic components may often require a redesign of a controller. Modern systems use commercial off-the-shelf (COTS) technologies, and these technologies (e.g., SCSI, PCI, parallel ports) often disappear in 10 years—not to mention hardware architectures moving from 32 bits to 64 bits (including OS platforms) and changing network technologies.


Upgrade, gain transparency, connect with maintenance

The lifespan of an automation control system can be anywhere from 10 to 30 years—that’s a long time, says Allen Casteel, Stellar operations manager - automation & digital PSM. But older systems just don’t have the features that can save money in the long haul, such as energy savings. Modern refrigeration systems have advanced controls for efficiently sequencing compressors and controlling the head pressure at its optimum point.

“If your system is more than 10 years old, you’re probably missing out on some energy-saving benefits,” says Casteel.

The second point Casteel makes is communication improvements. There are great labor-saving benefits to be gained from upgrading a control system that can integrate all the equipment and sensors from the basement to the rooftop. Operators can see the entire plant from one screen and make better choices about control and maintenance. Call it transparency.

“Transparency is improving with the latest generation of automation products, which allows for easier failure identification and rectification,” says Bühler’s Beloskur.

If downtime is a major concern, the latest automation technology can begin to predict failure and allow for the strategic planning of maintenance to ensure the highest possible operational effectiveness.

“A secure remote access solution allows customers to access their production sites in order to troubleshoot and fix problems 24/7 from all over the globe,” says Ute Decker, marketing manager of machine building and automation manufacturing, Belden, a CSIA partner member.

Because of the increased network efficiencies and security available today, users can easily enable remote control and diagnostics of PLCs with minimal IT knowledge. This enables teams to fix problems more efficiently, increasing productive staff time while achieving less downtime.

Many new control systems are designed to integrate with maintenance systems, says Brian Courchesne, Schneider Electric global industry services marketing director. Sensors can monitor the health of components and leverage modern networks to prevent any issues from affecting production or even occurring in the first place. Now, direct real-time measurement of asset reliability and asset reliability risk is possible.

“We work with our customers to shift from reactive to predictive maintenance models by enabling the measurement of operational profitability factors in real time from the asset level,” says Courchesne. “By empowering the workforce with real-time operational profitability data, and process control and real-time reliability risk information, operators will be able to see the impact they and their adjustments are having on the process, and the profitability and reliability of their assets.”

It’s a combination of things that leads to less downtime and more output.

“With our new generation motor, you might get 30 percent more power,” says Bosch’s Cameron. “With new controls, you’ll get better compatibility across a wide range of systems, including IIoT/Industry 4.0 compatibility with IT systems. Connectivity like this can result in less downtime through predictive maintenance, improved operational flexibility and better quality control.”

Today’s fourth-generation HMI bridges the gap between operations and maintenance, says GE’s Millinger.

“When an out-of-spec event takes place in the HMI/SCADA, the automation system can trigger a work process to interface with the CMMS/EAM, secure a work order number, send specific instructions including GIS location to the right person, monitor and log corrective action and close out the work order.”

One part of maintenance hasn’t yet been mentioned, but it’s a critical part of any system—disaster recovery.

“Modern control systems can provide maintenance benefits through implementation of a robust disaster recovery,” says McEnery’s Moss.

Automated and secure backups of programs reduce the possibility of a catastrophic failure resulting from the loss of program revisions or the entire program. Also, backup of configuration files for devices, such as VFDs, specialty modules and networked instruments through automated backup systems/change management systems, allows for quick download to replacement devices on failure.

Speaking of software programs, users who don’t keep current on updates can create unique maintenance problems for themselves and the software developer.

“The main reason for upgrading from a legacy system to the latest software version is compatibility issues that arise,” says Val Dychok, software developer for CAT Squared. “A good example concerning our software platform would be a common situation where some of our customers have an early version of our listeners and mobile software that spans back four to six years. A listener handles incoming messages from handhelds and responds back with a status message and data to be displayed on the handheld.”

If a customer requests the latest mobile software release, it can’t easily be done because of the functionality gap between the listener and the handheld.

“It would require a more complex upgrade,” says Dychok. “It also creates a knowledge gap in the sense that customers don’t know about new functionality that’s been added since their last upgrade and are often blown away by the number of new features when they receive the latest mobile release.”


Old is not new again

Because many legacy automation systems were built as proprietary, standalone systems, upgrading them can be challenging.

“Different components did not communicate with each other,” says Stellar’s Casteel. “Today, people increasingly expect all controls to be integrated and to be controlled from one central location.”

So, why spend the time, trouble and money for an upgrade and new automation equipment? “Without a doubt, newer equipment and software are more reliable, robust and secure,” says CRB’s Pflantz.

Some key reasons are plug-and-play hardware, self-configuring hardware, cybersecurity hardware and software elements, and modular designs for easier replacement of key components, to name a few.

“Technology has advanced, and we are several generations into some systems that have incorporated features brought about to address real operational challenges or issues,” says Pflantz. “And yes, this is just a fancy way to say we learn from our mistakes. We are getting smarter, and so are our control systems, with self-configuration, diagnostics and redundancies that provide continuous operation with barely a blip in the system.”