Acceptance Testing of Control Systems – Part # 1

In the long list of Project Accomplishments, Acceptance Testing of Control Systems and Emergency Shutdown Systems for New Plants had been one of the most enriching and satisfying endeavors for the author. As a young Engineer, alacrity to inspect finer details of complex control system architecture unravels technology in layers of design. With time and experience, the finer details give way to checking of system performance and process design implementation while mentoring young engineers to explore finer details!

OEMs have now got used to devising methods and checklists for clients after gaining experiences from success and failures during Acceptance Testing at Shop Floor and Site Acceptance Testing in actual plant conditions. During a Factory Acceptance Testing of a Control System for a Petrochemical Plant with Invista and Kvaerner U.K. that lasted more than 35 days, an important lesson was learned by a prominent OEM that how communication gap in understanding complex loops from Licensor can also jeopardize Acceptance Testing at Factory Premises. Taking the cue from JIT, OEMs are also making precious shop floor as Fast Moving Arena, where any delay or hiccup can derail future shop floor usage plans. Hence Acceptance Testing has gained significance in terms of Project Requirement, Technical Specifications, Implementation of Licensor Design, Client Requirement of Reliability and Maintenance and more importantly cost overloading on both Client’s and Supplier’s side should a delay or re-work in Control System Design post Testing affect project costs and schedules. The Write-up tries to explore Best Practices besides “Typical FAT Procedure” that have become norm in this “Fast Moving” Activity

1.      Feel it in Real Time: GA/IA Drawings are approved and Client lands up to inspect 150 Cabinets only to find that “it doesn’t seem so” or “Maintenance Aspect which couldn’t be fathomed on paper”. The solution lies in creating typical cabinets GA/IA of each type and getting it modified as a Master Specimen.  The client feels it and approves the drawing with Pictures and Video of Master Cabinets. This can be immediately arranged after MR is approved, so that fabrication activity are commenced.

2.      Investing in TPIs: It takes a Day and not a fortune to train TPI by Client before actual testing can commence. They are trained to inspect finer details. So a Client’s own checklist on verifying Bill of Material can be taken up by TPI (Remember Young Engineer) while you dedicate more time on System Performance and Process Design Implementation. 10% sampling on 100% TPI verification can generate great results in terms of the utility of time and resources.

3.      Obtain Details on System Capabilities regarding Function Blocks: During a FAT with a Boiler OEM, the system capabilities got stretched due to the overdose of Timers, RS Flip Flops, and other Holding Registers. A handshake on function block capabilities before system design by a control System Manufacturer with Detailed Engineering Contractor would not have led to encountering system limitation during FAT. More Economically Unviable for all than Embarrassing.

4.      Graphics or Virtual Reality: Operations can be demanding at times, which indeed is a Good Thing. Interface graphics though getting standardized due to the evolution of ISA guidelines remain a subjective matter. If graphic design is moderated during FAT with Operations and OEM Graphic Team, CG requirements will not only put stress on FAT resources but also deviates Instrumentation Team in managing expectations of their Operation Counterparts. A solution lies in sending your best graphic designer at Client Site with all the tools on a Server Machine that finalize 95% optimization of graphics and sign off for minor adjustments during FAT. Say during Interlock Testing.

5.      Burn Test in Severe Conditions: Electronics Burn Test during Pre-FAT by OEM is a good measure of endurance of a Control System. However Ambient Temperatures during such testing doesn’t get recorded. In actual site conditions and especially during new Project sites, the strength of Control System Electronics gets tested under severe ambient conditions. A 30 Days Burn Test can be reduced to 20 Days schedule with 50% of Time seeing 25% severity in Ambient Conditions. For Example, if Control System Ambience requirement is 24 Deg C, 10 Days Testing can be carried out at say  30 Deg C. New Control System fresh from Factory Acceptance Testing can more reliably withstand the testing of the newly constructed site with say failing Air Conditioning and less than satisfactory Access Control.

6.      Cybersecurity and Firewall Effectiveness: IP-based Client Server Architectures and CAT 6 replacing proprietary bus have rendered control systems exposed to fragilities of modern day IT hardware. Hence it remains a good idea to get Independent Security Audit and Testing done through Organization’s IT team mostly responsible for security policy framework. Though IT security deals with security of data while ICS Security deals with Safety of Process Plant, the first line of defense can be tested by IT Team while End Point Security solutions can be tested through ICS Instrumentation Group. This saves enormous time in synchronizing IT policy implementation especially on ICS Gateways at the site.

7.      Understanding Cultural Preferences: In a world congregating clients, licensors, detail engineering contractors, EPCC, Vendors, Testing Platform becomes a melting pot, it is essential that SOP of Communication and Matrix is devised before everyone hits the shop floor. For countries where English use is not widespread, having a set of technical translators yield rich dividends. The client plays a pivotal role in coordination which should not be left to the Vendor OEM alone. So any escalation goes through Client who decides whether Escalation of conflict is necessary in the first place. Thankfully there are no rules for this but understanding culture and preferences by the client of stakeholders in Testing, is, therefore, useful for successful culmination of System Testing.

To be continued…