Process control performance management

New tools help you track, trace and optimize your process.

ABB’s Enterprise IT software includes System 800xA Production Management, which includes optimization and batch management tools and the ability to track information associated with material and control flow across the plant. Source: ABB.

There's no question that more technology tools are available to help food manufacturers satisfy the FDA, FSIS (Food Safety & Inspection Service), USDA and other government regulatory groups concerned with record keeping. But more importantly, these tools can rescue a food manufacturer's entire operation when trouble arises. According to ARC Advisory Group's John Blanchard, director of research, food and pharmaceutical industries, the ability to track and trace ingredients and products can help food manufacturers prevent bad product from getting out the door in the first place, eliminating recalls and the ensuing bad press that can kill a company overnight. In addition, new performance management tools and advanced control techniques can help manufacturers keep plants up and running at maximum efficiency.

Track and trace

Recalls can exact heavy costs. Blanchard tells the story of a chili powder from India that contained Sudan 1, a potentially carcinogenic red dye banned by the FDA. It found its way to more than 45 producers and led to recalls of about 100 different products. In another situation, however, the appropriate tracking and tracing software enabled a major food manufacturer to prevent a bad ingredient from going out the door in finished products, thereby eliminating recalls.

Most automation companies have software products in tune with HACCP, for example, when an out-of-range temperature is encountered in a smoking oven presents a danger to bacteria growth. Source: Siemens.

In October 2002, a Listeria outbreak in eight Northeastern states that caused dozens of illnesses and as many as 10 deaths was attributed to precooked turkey deli meat. After taking samples at Pilgrim Pride's Wampler Foods site in Franconia, PA, the USDA's FSIS found a strain of Listeria from the plant samples. At first, it wasn't considered to be the same strain causing the infections. To be safe, Wampler recalled 27.4 million pounds of meat and temporarily ceased operations while cleaning the plant. On November 13, 2002, Pilgrim Pride's CEO, David Van Hoose, announced that the company "voluntarily initiated the suspension of operations even though no illnesses associated with the Listeria strain in the Northeastern US outbreak have been linked to any Wampler products." Whether the bacteria found at Wampler caused the illnesses and deaths is not the issue here. Had a good tracking and tracing system been in place, it is unlikely all 27.4 million pounds of meat would have had to be recalled.

As it turns out, inaccurate tracking and tracing wasn't just a problem for Wampler. According to the September 2004 issue of the Food Protection Report, USDA's Office of Inspector General (OIG) criticized FSIS over its handling of the recall. The OIG concluded that there were many errors, all due to written forms incorrectly filled out by food purchasers or read improperly by FSIS. For example, FSIS officers failed to reconcile the amount of products purchased by customers with the amount recorded on the company's distribution list and didn't always document follow-up to ensure that customers had located the potentially contaminated foods. This situation screams for electronic record keeping for tracking and tracing.

Danish Crown, a major European pork and beef producer, is building tracking and tracing into a pork processing plant that handles 75,000 pigs a week. This plant will comply with any future international regulations on food safety, animal welfare, ergonomics, and environmental sustainability. Linking the various systems-SAP R/3, process control, plant logistics, warehouse management, building systems control, food safety and QA-into a single integrated system is ABB's Industrial IT Information Manager. The system will also provide Danish Crown with traceability and reporting functions to collect, store, and process data on all the pigs to be slaughtered, butchered, and processed each week.

Although government regulations are forcing the issue, the impetus for tracking and tracing will ultimately be market-driven. "Industry-driven standards around full traceability in the food supply chain may soon outstrip government regulations (as represented by the Bioterrorism act) as well as expand commonly used programs such as HACCP," says Sean Robinson, MES business development manager, JMP Engineering Inc.

The grocery industry in Canada has proposed a standard for ingredient traceability that goes to a level of detail not required by any legislative body (see www.can-trace.org); and the intention is to make compliance with the standard mandatory over a period of time. Going beyond any similar initiative, this industry group is even publishing a data model for what would be an appropriate database within a processor's operations, and for exchange of data between supply chain links. Wal-Mart is eyeballing this effort with great interest and has suggested that it may adopt the standard for suppliers.

"One of the most challenging aspects of the evolving standard is that it will absolutely require food processors to provide operation-by-operation ingredient (and eventually quality) traceability, which has always been perceived as costly and difficult to accomplish," adds Robinson. "As a manufacturing systems specialist, I see this differently."

Governments and customers want food processors to easily correlate the following data elements against finished goods lots: raw material characteristics (as established by certificate of analysis or post-receipt tests), raw material consumption, post production test/quality data, key process variables (particularly those that affect quality or safety), environmental data that may affect quality or safety, and proof of process reliability. "If treated as a compliance issue," says Robinson, "it certainly can seem burdensome to provide this [level of tracing]. However, if we look at what a processor needs to attack common operational pains such as yields, efficiency, inventory variances etc., we see that they essentially need access to the same data as the customer or regulator." Without a system inventory, variances or lack of sources can prevent unforeseen problems.

According to John Slater, PhD, vice president, manufacturing planning at Logility, supply chain management systems give producers the ability to look for other sources when one supplier's ingredient is bad or defective. This means the formulation doesn't have to change. However, when exact replacement ingredients can't be found right away, formulations can easily be changed without having a major impact on the product.

One technology that will augment supply chain management is RFID. According to Lynn Stephenson, corporate engineer for Brach's candies, some of Brach's customers are thinking seriously about implementing RFID in the near future. Wal-Mart, of course, has been the major proponent of this technology which can satisfy many of the needs for tracking and tracing food products.

Still on paper?

According to David Fisher, Citect business development manager, many food manufacturers are still using pencil and paper to track their products, but this doesn't scale well when a problem arises. The cost of the software is not the primary concern. Because the application of the technology is still fairly unfamiliar to food manufacturers, CEOs are leery about investing in it.

Tools like FMC FoodTech’s NumeriCAL finite difference thermal process modeling software lets engineers set up thermal or cooking applications that provide data acceptable to the FDA and the USDA. Source: FMC FoodTech.

Indeed, a major issue is the learning curve. Sam Vail is responsible for the computer-integrated manufacturing environment at Nestlé's PowerBar plant in Boise, ID. Vail explains the dilemma at most facilities: "With limited engineering manpower available, we'd like to spend less time learning software and more time applying it to the problem."

Fisher says that while manufacturers are good at tracking logistics from a distribution perspective, they are not particularly good at tracking the product's manufacturing. But according to Claus Abildgren, Wonderware's product marketing manager-production management, food manufacturers want to expand focus for data collection and analysis to a production context, beyond process control.

What data should manufacturers collect? According to Thomas Lundqvist, LINK product line manager, FMC FoodTech, "The answer involves keeping records of equipment settings and human intervention with the process. Having belt speeds, equipment temperatures and product data (like core temperatures) available in one system can be critical when it comes to better understanding why a product is undercooked, for example. Data collection solutions should be capable of complying with 21 CFR Part 11. In addition, built-in security methods like usernames and passwords should be used and records of who did what should be kept (event logs)."

To get better at tracking the process, says Abildgren, manufacturers need to improve operational capabilities by understanding actual production status/history (production context to process control). Food manufacturers need to determine the correlation between process parameters and production performance. In other words, they must understand the impact of malfunctions in the process control system on business performance. Also, they need to understand in what areas of the process these improvements will have the most significant impact.

McCall Farms uses Beckhoff’s TwinCAT architecture to keep its process lines running. TwinCAT runs in the kernel mode on Windows computers. If Windows crashes, TwinCAT processes keep on running. Source: Beckhoff Automation.

Performance management

According to Blanchard, performance management software enables manufacturers to see the results of process improvements on their business performance. This software, often called downtime software, has been applied to automobile production lines and is now finding use in food packaging lines. But it's not limited to packaging. Operational performance management software collects data from the plant floor for real-time as well as historic downtime and performance analysis and reporting.

According to Abildgren, "This means understanding the overall equipment effectiveness (OEE) and determining root causes for production losses, in real time. Using today's IT technology, even the impact of micro-stops on bottling lines can be analyzed and corrective action taken to increase performance. For many of our customers, this has provided significant return on investments with limited investments."

Campbell Soup has already discovered the benefits of this software. Francis Tan, principal engineer, says it enables Campbell to monitor production equipment to collect data for monitoring downtime, or as he says, "We prefer to use the word uptime."

Some experts may include asset management software under the umbrella of performance management because it clearly helps achieve better performance. However, asset management includes assigning equipment to process needs, in addition to predictive and preventive maintenance. The advantage to predictive maintenance is being aware that a stoppage is about to occur. Campbell Soup's Tan reports that one of Campbell's plants uses an advanced vibration monitoring system, that when coupled with predictive maintenance software, predicts possible equipment failures before they occur.

Yet another way to keep tabs on the plant floor, according to Mike Sardinha, president of CAT2, is through the use of electronic scoreboards (some call them scorecards). "Using electronic scoreboards on the plant floor to communicate plant performance is particularly effective," says Sardinha. "Scoreboards provide instant response to workers on their performance and, used in conjunction with RF sensors, provide feedback in time for proactive control."

Stability-a hardware issue?

When asked what technology trends are really important right now, Nestlé's Sam Vail replied emphatically, "Stability, stability, stability." He's grateful to Cisco, Intel, HP, Microsoft and Wonderware for plenty of acquisition, processing and analytical power, but notes, "We still do not have the workstation, OS or application stability that is required in an uptime-intensive environment."

Programmable Automation Controller (PAC) was a term invented by ARC Advisory Group to denote a compact, industrial-strength controller capable of providing PID, motion, and PLC functions in the same box, which consists of an open architecture. Source: National Instruments.

There are several approaches to fixing this problem, and most of them center on hardware; that is, finding hardware to survive nasty, sloppy food processing environments. One approach Campbell Soup uses is thin clients. Without hard drives, these smaller (typically Windows CE-based) devices are just more physically reliable than a PC. Maintenance is important as well. Francis Tan said that in the past it could take several hours to replace a full-blown PC on the plant floor and get it up and running. With thin client-based hardware, the application resides on the server; it takes little time to plug in a new machine, connect it and have it operational. With the application residing on the server, there is always hot backup with failover at that level as well, adding to several layers of redundancy that just didn't exist before. (Failover in this context refers to a redundant server that takes over (transparently) if the primary server fails, either due to hardware or software failure.)

FMC FoodTech's Lundqvist suggests the use of Windows CE-based interfaces as they become available. The smaller OS allows the use of flash cards instead of hard drives. Since there are no moving parts, this is a more robust solution, allowing food processors to have PC functionality in their interfaces as robust as traditional HMIs.

Hardware systems that are compact and built for the environment seem to be the solution. "When making improvements in a food and beverage plant," says Rahul Kulkarni, National Instruments product manager-industrial data acquisition and control, "engineers try to identify and remove bottlenecks in the process that consume additional time." Examples include long settling time for PID loops, machine downtime caused by excessive vibration, and lack of alarming when faults are detected. To remove these bottlenecks, Kulkarni says engineers are replacing traditional PLCs with more intelligent programmable automation controllers (PACs) in areas that need performance improvements. The open architecture of PACs enables them to seamlessly integrate with existing PLCs or DCSs via standard fieldbuses. These diskless controllers have enough performance for vibration monitoring applications and are versatile enough to handle motion control and PID.

When stability is an issue for building food processing machines, the use of a Windows OS with critical process equipment requires a purely industrial approach. McCall Farms, a producer of canned fruits and vegetables, has recently changed much of its automated cooking and packaging equipment to a Beckhoff TwinCAT system. According to Jason Durant, electrical engineer, much of the older control software was becoming obsolete and ran on early versions of Windows. The system was producing too many errors and lockups, causing loss of product. "We upgraded our existing HMI program, which directly links into TwinCAT via an OPC server."

Recently, McCall Farms avoided considerable expense from potentially wasted product by using TwinCAT, which runs in the Microsoft Windows kernel, and is independent of other OS processes. In the past, when a whitebox computer in the cook room locked up, Durant was unable to take immediate corrective action. But because TwinCAT runs in the kernel mode, the process was able to complete without problem, resulting in zero loss of product.

Tight money, open systems

Money is always an issue when making technical improvements in a low-margin industry. "The big initiative is increasing capacity without spending significant capital," says Jennifer Hiltz, industry lead-food and beverage, ASECO Integrated Systems. "This implies maximizing availability, performance and quality (i.e., overall equipment effectiveness (OEE))." Key ingredients to maximizing OEE include hot standby controllers, robust programming, good trouble-shooting and diagnostic techniques, and good instrumentation, data collection and analysis.

According to Charley Rastle, solutions manager at Rockwell Automation, the drive to reduce costs and meet regulatory requirements requires open access to information. Food manufacturing control systems must share data and ultimately produce usable and actionable information. By using open, information-enabled network architectures, users gain access to the information they need to make informed decisions about their facility and collect the data to meet regulatory demands.

One implementation that exemplifies this open network-based architecture is Gallo Glass Company's plant (part of E & J Gallo) in Modesto, CA. Based on GE Fanuc's Proficy iFIX shop floor information system, the plant saves $5 million annually while increasing production by 5%, reducing defects by 25%, and cutting downtime by 25%. Says John Gallo, vice president, "The vision is one of an information-centric culture-where every employee can combine trade craft and science in every bottle we make. Our process is complex with many interacting variables. This new technology has simplified the learning curve for new employees, making them more effective and proactive in their jobs." The system serves multiple users-from operators on the lines to maintenance managers to senior managers who need quick, graphical reports via the Web.

One caveat to all this openness is, of course, security. According to Todd Stauffer, manager, product marketing, process automation systems, Siemens, controlling access in computer networks is a significant issue. Many manufacturers are implementing standard operating procedures to ensure that only authorized individuals gain access to the process automation system and data. This type of policy typically includes user administration based on Windows security (such as password expiration and lockout after a number of retries) and controlling access to project data stored on the hard drive. The methods for keeping processes secure are very similar to those used in the pharmaceutical industry, for instance, to meet FDA regulation 21 CFR Part 11. Many manufacturers are evaluating biometrics, card readers and other special access control mechanisms. To further prevent unauthorized physical access to the automation system, key assets, including controllers, PCs, servers and engineering workstations, are now physically isolated and protected in locked rooms.

Advanced control

Advanced process control (APC) in the process industries usually conjures up PID, model-predictive control, fuzzy logic, etc. In the food and beverage industry, some advanced control techniques are in place in oils and extracts. Advanced control techniques have also been used in beer and other applications where the cost of losing a batch may be considerable. Wonderware's Abildgren says, "A new area where we are starting to see interest is in the FDA's Process Analyzer Technology (PAT)."

More widely known in the pharmaceutical industry for advanced analysis of active pharmaceutical ingredients, PAT involves sophisticated multivariate analysis. FDA defines PAT as "a system for designing, analyzing and controlling manufacturing through timely measurements of critical quality and performance attributes of raw and in-process materials and processes with the goal of ensuring final product quality." As food manufacturers introduce more active ingredients in food products, PAT is being used in new product development processes and even in commercial production.

According to Abildgren, "Another area where we continue to see increased adoption within food manufacturers is statistical process control (SPC)...to ensure that temperature in the pasteurization or fermentation process was within process specifications." Another SPC application, says Northwest Analytical's Jeffery Cawley, VP market development, is weight and portion control, both for regulatory compliance (Class II and III citations are huge hidden costs) and product giveaway. In a bakery application where each mixer dumps 1,200 lbs. of dough into a pouring machine that dispenses one pound (454 gm) per loaf, SPC/SQC software was able to detect changes in dough texture from one batch to the next. Further investigation showed that flour and water proportions were not conforming to the recipe in one mixer, causing wider variation in weight, mostly on overfills. Getting the weight variability to a normal level saved the bakery approximately $200,000 per year.

A model-based predictive control solution based on Pavilion Technologies' Process Perfecter APC software was used at Fonterra Co-Operative Group's Kauri K2 milk powder plant in New Zealand. The plant has two evaporators and one spray dryer with two falling film evaporators feeding a Niro atomizing spray dryer processing 6.5 to 9 tons per hour. The objective was to reduce solids variability by 50%. Results showed better than the objective: variation was reduced by 73% for Evaporator 1 and 68% for Evaporator 2. The application of model-based control increased production rates by 5-15%, improved product quality 50%, and increased energy efficiency by 5-12%. Payoff: better than 60% ROI.

The way a manufacturer can benefit from using SPC techniques is based on how it views process upsets and deals with them. According to Cawley, manufacturers can either remove process upsets or adjust the process to react to the upsets. Each method has its pluses and minuses. Obviously, advanced control techniques will not be a panacea for all food processing applications, but some of the more basic tools aren't that expensive, and they just might save time, money, product and equipment.

If you've been waiting to update your control technologies just to satisfy the government, you might want to rethink your goals in terms of maximizing your profits. Manufacturers that have already updated now have the edge. Their supply chains are under control, their processes are optimized, and their records are available at a moment's notice to track and trace ingredients and products.

For more information:

John Blanchard, ARC Advisory Group,
781-471-1000,
www.arcweb.com

Dennis McKinley, ABB,
919-870-1915,
www.abb.us

Sean Robinson, JMP Engineering,
905-631-6660 ext. 223,
www.jmpeng.com

John Slater, Logility,
800-762-5207,
www.logility.com

David Fisher, Citect,
770.521.7511,
www.citect.com

David Hancock, ACP,
678-990-0945 ext. 203,
www.thinmanager.com

Thomas Lundqvist, FMC FoodTech,
312 861 6000,
www.fmcfoodtech.com

Claus Abildgren, Wonderware,
949-727-3200,
www.wonderware.com

Mike Sardinha, CAT2,
501-328-9178,
www.catsquared.com

Rahul Kulkarni, National Instruments,
888 280-7645,
www.ni.com

Shane Novacek, Beckhoff Automation, 952-808-6515,
www.beckhoffautomation.com