Overview
Inoperable controls cabinet replaced and recommissioned in eight weeks vs. the competition’s eight-month lead time.
Successful MKVe recommissioning with no failed starts or trips after initial startup.
Root Cause Analysis (RCA) leading theory: battery charger malfunction, combined with other factors.
Challenge
A few days after the start of a planned major inspection on their 7EA gas turbine, the maintenance team at AES’ Harding Street Power Station began troubleshooting issues with the battery charger on the Mark Ve (MK5e) control system. Within minutes of powering up the charger, smoke was seen emanating from the turbine controls compartment and a fire alarm was sounding near the MK5e controls cabinet.
Minutes later, the team extinguished the control cabinet’s fire. The hazard left the MK5e inoperable and covered in the fire extinguisher’s chemical residue.
The AES team shifted to recovery mode.
How could they “restore or replace” the control system so that it would operate safely, reliably, and without drastically lengthening the outage?
Solution
The AES team reached out to several vendors, including AP4 Group, to request quotes for recovery or replacement. The scope of nearly all of these options was lengthy, around eight months in some cases, due to long lead times on hardware procurement. All except for one provider - AP4.
The AES team received our experts on site, and within three days we submitted a firm proposal consisting of:
Replacement in-kind of the MK5e,
Four-week procurement cycle, and
Four-week installation and re-commissioning.
Eight weeks. Perhaps, an ambitious timeline, but one that the AP4 team was confident with due to our extensive experience and in-house expertise. AES would go on to award the work to AP4.
The key factor that enabled our winning strategy is the synergy between our ability to build, factory acceptance test (FAT), and deliver an identical MK5e panel and the experience of our field service team to complete the recommission under severe time and quality constraints.
A four-week commissioning schedule is not unheard of for a brand-new installation, where everything is in pristine condition. Our challenge was to recover from the fire, where surprises showed up at every turn, and deliver a reliable, functioning unit to our customer’s satisfaction.
How were we so confident that we could meet the timeline? The AP4 Team has successfully completed 75+ turbine-control retrofits with various OEMs. This unparalleled know-how allows us to face any control challenge with the “been there, done that” confidence and due care for safety, detail, and precision.
Results
We honored the eight-week commitment. The new cabinet was re-wired and loop checked, all the tasks in our commissioning schedule were “checked and yellow-lined” and, most important: with no “punch list items”.
As of 11 months after the commissioning, the unit remained operational without any related issues, including 100% starting reliability and zero trips due to the MK5e retrofit.
ROOT CAUSE ANALYSIS
The team noted that the battery charger responsible for the fire had been problematic since late 2023. A third-party vendor performs quarterly checks on all battery banks at the facility and had replaced a couple of parts on the battery charger prior to this event and had noted some abnormal activity.
The RCA’s leading theory:
The charger failed to charge the batteries and DC voltage cycled between low and high values.
These oscillations caused heat to build up in the power supply until it caught fire. A lower probability, alternate possibility, was that AC could have moved onto the DC via the failing battery charger or an external source from the field.
Other related issues were determined as follows:
A preventative maintenance strategy to clean the MK5e hardware and I/O did not exist
No DACA converter as an alternative MK5e power supply to the battery bank
Jumpers existed on both the MK5e and the field – which complicates troubleshooting
Available Fire Extinguishers were ABC instead of Class-C CO2, which are the correct type for electrical fires.
The RCA issued the following recommendations:
Perform thermography scans of existing MK5e to identify any hot spots.
Replace the original battery chargers. Suggested to have High/Low DC Voltage Alarms, Ground Detection, and Common Alarm.
Setup Voltage Monitoring in EGD & PI and Graphics for all MK5e panels.
Consider adding H2 Monitoring to all Plant Battery Locations
Where there is risk of electrical fire, provide Class-C CO2 fire extinguishers.
