The about-face is driven partially by the untapped return from such initiatives. But other factors also are at play, not the least of which are the looming enforcement of standards for electronic records and time-stamped signatures in the six-year-old FDA regulation 21 CFR Part 11 and the more demanding requirements for verifiable records included in the Bioterrorism Preparedness and Response Act. Compliance costs for 21 CFR 11 by pharmaceutical companies alone are pegged at $5 billion, double their spending for Y2K compliance.
Even more stringent record keeping is mandated under the Bioterrorism Act, and steep financial penalties loom for companies that cannot quickly produce those records for inspectors. Paper records are not precluded, though as a practical matter large organizations will have to rely on electronic systems to meet the Act’s timeliness-of-response requirements. Migration to centralized databases also is occurring in other countries. The European Union will require all food companies, regardless of size, to maintain processing and product data electronically by January 2005.
Perverse, but nice
“In a strange way, the Bioterrorism Act is a positive force in improving the way resources are managed in the processing industry—materials, equipment and people being the resources,” suggests Paul Moylan, marketing manager for the manufacturing and process solutions unit of Rockwell Automation. “Thought leaders in the food industry recognize there is no enforcement agency ready to shake them down because they are not in compliance with the act or 21 CFR 11, but those leaders aren’t going to risk their brands and their companies’ names because they failed to take action.” Top-down integration is still years away, but companies need to be devising a roadmap for executing that integration, beginning with traceability and recall records.While FDA is charged with enforcement of the act’s provisions, Moylan says it’s unclear who will actually evaluate processing facilities, given the large number of food plants and the paucity of FDA inspectors. The point may be moot, given the interest among retail customers. Oracle’s Friedman notes Wal-Mart is insisting on “one up, one down” traceability of food suppliers’ products, with those unable to demonstrate total responsiveness likely to be shown the door.
Two years ago, Rockwell acquired Propack Data Corp., a specialist in enterprise production management systems for pharmaceutical and biotechnology companies. That led to the introduction of Regulatory Compliance Services, which helps those companies address computer-system validation and other compliance issues under 21 CFR 11. “Since then we’ve added experts in food compliance to develop remediation plans for this industry,” Moylan explains. “In a perverse way, the Bioterrorism Act also is forcing the industry to adopt more sophisticated ways of automating their tracking processes.”
The software and systems integration arm of Rockwell must pay more than lip service to the open-architecture ideal. Food processors need complete accessability to their own data if they are to relay information quickly to suppliers and customers. Rockwell and other suppliers of PLCs and other automation hardware have embraced openness as the only way to meet client needs. “The more recent the (controller) offering, the easier it is to connect to the new system, despite hardware manufacturers’ effort to dig their heels in and maintain closed systems,” says Moylan.
“There are open systems, and then there are truly open systems,” he adds. “There are vendors who say they have open architecture, but they have their own versions of XML or OPC, and when you try to integrate them, it turns out it’s not as open as you thought it was.”
Open architecture was a prerequisite in the company’s work with Fonterra Co-Operative Group Ltd., the New Zealand processor that dominates the global market for dried dairy ingredients. The Southern Hemisphere’s largest contingent of Allen-Bradley PLCs is found in Fonterra’s 10 plants. Data from a total of 230 controllers is minded for storage in a Rockwell RS Sql industrial transaction database. But Fonterra also makes extensive use of Pavilion Technologies Inc.’s advanced process control system, a component Fonterra management credits for more than $9 million in cost savings in the last two years alone. Seamless integration wasn’t a nice-to-have feature: it was absolutely essential.
The relationship between Fonterra and Pavilion extends over more than a decade, according to Ian Steele, the dairy cooperative’s former engineering manager who now serves as vice president of strategic development at Austin, Texas-based Pavilion. Automation without integration was never a consideration, he says. Instead, senior management insisted from the outset that global-compliant standards and open architecture be specified in every control and at every interface. The mantra became “automate and integrate,” and ROI for each initiative was calculated, with much of it made possible by the information flowing to operators, who were able to make better decisions that improved corporate profitability.
“We managed the integration of everything above the HMI ourselves, and the standards in the formal delivery contracts were very specific in terms of architecture and system configuration,” recalls Steele. “A solid, growth-oriented, strategic architecture is committed to at senior levels in a company, and that means addressing capacity and connectivity issues up front. Otherwise, companies tend to be cost driven. Needed elements of the enabling architecture are then closed off, and you’re later faced with a costly retrofit.”
Product variability is the enemy of plant efficiency and corporate profitability. Reducing variability to improve consistency and, ultimately, boost throughput and capacity has been the focus at Fonterra. A 43 percent improvement in moisture error from set point in drying and evaporation operations has been realized since 1990, resulting in more product that is in spec, less product giveaway and maximized value from raw materials. Variables that affect drying, such as outside air humidity, are compensated for by closed-loop systems, freeing operators to focus on decisions “that only can be done by the human,” he says: decisions that impact profitability, not the process and set points.
Increased efficiency and higher-value products have paid off for Fonterra, which is the processing arm of the New Zealand dairy industry. The country accounted for 31 percent of international dairy trade last year, up from 19 percent in 1990. By comparison, the U.S. has 4 percent share and production costs that are more than double New Zealand’s, according to the International Farm Comparison Network.
The process-analysis link
Plant-wide integration is not on the agenda at many processing facilities; instead, product-line integration is today’s challenge. A SCADA system can tie together the controls, but that leaves the database needs of a process analysis system as yet another project. To simplify matters, FMC FoodTech has bundled both in an advanced control system called LINK.LINK grew out of a desire to better tie the Stein cooking and Frigoscandia freezing lines acquired by FMC a few years ago, explains Thomas Lundqvist, manager of controls and automation products at FMC. Poultry and meat processors might have 10 individual components from those equipment suppliers on a line. Rather than leave them with the integration challenges of tying together PLCs in ovens, conveyors, coating machines and freezers to achieve line speed synchronization and recipe management, FMC decided to offer out-of-the-box integrated controls.
Discussions with customers convinced FMC that it had to go further, though, and include a process analysis module. This data collection and storing capacity will support a statistical process control package and other report generators to help identify the causes of product inconsistency and identify yield-improvement opportunities, Lundqvist says. “Our customers’ customers are requiring a lot more product data and product uniformity, and the ability to feed back quality data to operators in real time helps address that need.”
It also addresses the traceability demands the processors must meet, not simply in the U.S. but globally. “In a way, we are becoming a systems integrator, though we don’t want to be the people to tie the whole plant together,” he says. LINK was designed to easily tie in with systems from Rockwell, Siemens and plant-wide architectures like Wonderware’s FactorySuite A2.
FactorySuite A2 is the first Wonderware industrial application server to be built on its new ArchestrA platform, a Windows-based operating system that features reusable coding to speed implementation and reduce system-engineering costs. Adoption of the MicroSoft platform simplifies data exchange between the shop floor and the front office, where Windows-based applications are common. That was the case at Tillamook Creamery, one of the first installations in the food industry.
Systems integration was handled by Progressive Software Solutions, which also is the first certified ArchestrA integrator. Tillamook had a Windows-based ERP, explains Tim Fief, a founder of Progressive, and the compatibility between ERP and MES gives the cheese maker’s front office much greater visibility to inventory levels, production costs and other aspects of processing than it otherwise would have.
“The application server has SPC and other aspects of the business system associated with it, but it also has traceability, and our food customers are very interested in looking at the genealogy of their products now,” says Fief. Centralizing the data is the only way to effectively manage it, and the application server satisfies that need.
This approach allows project managers and auditors to determine who did what to each product and at what time, adds Yves Dufort, Wonderware’s director of strategic systems integration. However, it is the prerogative of each end user to determine the level of security that is built into the system. Open access might be allowed for some functions, name and password for secure inputs, or verifiable security for critical actions where a supervisor must also log on with name and password.
“A software product will not make a system compliant and verifiable; that requires a process,” points out Dufort. “We deliver tools that can be validated as compliant with 21 CFR Part 11 or meet the record-keeping requirements of the Bioterrorism Act, but it is up to the systems integrators to engineer systems that allow firms to accomplish those objectives.”
To satisfy both 21 CFR 11 and the Bioterrorism Act, systems must juggle the need for security and the demand for visibility. Standardization of shipping codes also is needed if food processors are going to communicate effectively with their retail customers. Speaking at the Oracle forum, John Norris, former deputy commissioner of the FDA, suggested that, with the myriad proprietary coding systems used by processors, their suppliers, and retailers, “Traceability is all but impossible.”
“The retailer-manufacturer interface is horrible in this country,” Norris maintains. “The weakest link has the most exposure to you,” and just as mass merchandisers are insisting processors have “bulletproof” recall and security systems in place, processors must be prepared to drop retail accounts who refuse to cooperate with their efforts to standardize case- and pallet-coding.
Outsourcing arrangements further expand the need for visibility to secure systems. “The intellectual property owners and the people who make products increasingly are different entities,” points out Rockwell’s Moylan. “That expands the visibility and integration issues beyond the four walls of the plant. If a contract manufacturer is making my product, how do I know he’s using the right recipes and executing the process properly? The solution is portal technology. We’re still one or two years away, but that’s where we see the industry going.”
If the industry is to get there, even higher levels of systems integration will need to be in place. Any automation plans that fail to consider the demands for both openness and security is a recipe for integration disaster.
For more information:
Angela Siem, FMC FoodTech, 419-626-4143, angela.siem@fmcti.com
Ian Steele, Pavilion Technologies Inc., 512-438-1400, isteele@pav.com
Tim Fief, Progressive Software
Solutions, 541-924-1741
Paul Moylan, Rockwell Automation,
919-465-1741, pemoylan@software.rockwell.com
Yves Dufort, Wonderware Strategic
Integration Group, yves.dufort@walshautomation.com
Henrik Sahl, ABB, 011-45-4550-4310, henrik.sahl@dk.abb.com
