Real-time edge IIoT monitoring powers predictive maintenance for utility circuit breakers

“We chose NI’s Single-Board RIO hardware because its performance, reliability, and price point per unit made it economically viable for our customer to deploy monitoring nodes to every single circuit breaker in their service area. They have tens of thousands of them across a large geographic region.”​

David Patrick, Vice President, MSO Technologies

Project Summary

An electrical utility provider managing tens of thousands of circuit breakers faced critical operational blind spots because of its reactive maintenance approach. They had no means of acquiring or analyzing the day-to-day operational data needed to implement preventative maintenance strategies and improve overall operational efficiency.

System Features & Components

• An asset health monitoring system capable of operating in extreme ambient conditions across geographically extensive service areas

• Comprehensive circuit breaker event data acquisition, analysis, and alarming

• Dual form factor design: portable units for scheduled maintenance and permanently installed systems for continuous monitoring

• Cost-optimized price point per unit to make widespread condition-based monitoring economically viable across tens of thousands of circuit breakers

Outcomes

The collaboration delivered transformational results with projected savings of $400 million over 20 years and a six-year payback period. Operational efficiency improved through fewer reactive maintenance events, enhanced service reliability, better data reporting, and fewer and shorter service outages. The electrical utility provider transformed its reactive, rigid service schedule into a data-driven, condition-based maintenance approach.

Technology at-a-glance

Portable Unit

• NI CompactRIO-9053

• NI LabVIEW Real-Time & FPGA

• MSO RTOS daemon software

• Lens Client™ software for laptop-based configuration and analysis

Installed System

• NI Single-Board RIO-9651 (SOM)

• Circaflex 580 for I/O and connectivity

• NI LabVIEW Real-Time & FPGA

• RTOS daemon (MSO)

• Lens Enterprise™ Server for centralized data management

Critical Grid Infrastructure

Circuit breakers are critical parts of the electrical infrastructure, serving as crucial power delivery devices that continuously transfer electricity across the network. They respond instantly to current levels outside predetermined thresholds, making them vital system management devices that protect grid infrastructure and enable operators to intervene when circuit breaker maintenance is needed.

Proactive Asset Monitoring

To mitigate the risks associated with circuit breaker maintenance, one utility provider knew it needed more data insights to reduce unexpected service outages and circuit breaker events.

Because circuit breakers in the electrical grid are expected to have a long operating life and high availability, this utility provider needed a solution that would help them use their service and maintenance crews optimally while mitigating the risk of unnecessary site visits.

Their existing solution for circuit breaker maintenance was to evaluate and diagnose issues during planned maintenance events or other onsite crew operations, which meant they couldn't capture data from unanticipated breaker events during typical day-to-day operations.

Traditional maintenance approaches can result in operational blind spots, preventing comprehensive equipment performance tracking and leaving utility providers vulnerable to unexpected failures. A lack of data insights made it difficult to proactively stabilize the workload for service personnel and to recommend preventative maintenance services for their assets.

To meet their grid availability and cost-reduction goals, this utility provider needed a solution that would enable them to proactively manage their infrastructure assets.

Building in Predictive Maintenance

The deterioration that precedes and causes most circuit breaker failures is detectable before an actual failure. The ideal solution to safeguard the utility provider's infrastructure and workforce is to install smart electrical grid technologies to measure and analyze all circuit breaker events.

They needed a continuous asset health monitoring system that was reliable enough to monitor circuit breaker equipment continuously and robust enough to handle extreme weather and conditions across their extensive service area. With tens of thousands of circuit breakers across their service area, they needed to optimize the price point per installed monitoring system to make widespread condition monitoring economically viable.

For this utility provider, evaluation and diagnosis of circuit breaker issues were limited to planned maintenance events or onsite operations, leaving them without visibility into normal and fault-condition breaker operations. This lack of insight made workload stabilization difficult for service personnel and prevented effective preventive maintenance.

To maximize asset availability and manage service crews' workload, the utility provider needed to continuously monitor circuit breakers for activity outside normal operating thresholds. Data insights into circuit breaker activity would enable the utility provider to proactively prioritize and schedule maintenance based on the urgency and criticality of the issues identified.

This electrical utility’s regulatory requirements mandated annual testing of every circuit breaker in operation. Compliance with regulations and specifications is critical for public safety and ensuring reliable access to electricity. To ensure regulatory compliance, a circuit breaker must respond promptly and adequately to electrical events. These circuit breakers then provide performance data to regulatory bodies that characterize their response.

Because circuit breaker operations can occur at any time, compliance for a particular circuit breaker can be confirmed by observing how the breaker responds to an electrical event. With a solution for remote, continuous data logging and insights, this utility provider could dramatically reduce compliance costs without ever sending a maintenance crew to the site.

Seeking expert guidance, this utility provider turned to MSO Technologies for a solution that could continuously monitor their assets, provide data insights to predict potential circuit breaker failures, and ensure regulatory compliance.

The utility provider wanted two form factors of the same system to provide the insights they needed while scaling out and deploying permanent monitoring solutions across tens of thousands of circuit breakers.

Table 1. Descriptions of two form factors of circuit breaker monitoring systems

One form factor would be a portable unit for assessing circuit breaker health during typical, reactive, or programmed maintenance events. The data gathered onsite would then be used immediately to identify issues and help personnel return the asset to service. The second form factor would be a permanently installed system that continuously monitors, captures breaker events, and publishes data to a remote server for immediate analysis and notification, if warranted.

Connecting Circuit Breakers to the IIoT

MSO Technologies is an expert in delivering operational technologies and solutions to the energy industry that help seamlessly connect assets and business systems to the Industrial Internet of Things (IIoT).

MSO turned to Cyth Systems for help because of its success in delivering highly responsive, deployed measurement systems in extreme and challenging environments. Cyth designed and built the field signal interface for high-speed data acquisition. MSO’s software managed event trigger detection, data aggregation, and data publishing for subsequent analysis. 

To address all the circuit breaker event measurement use cases for the utility provider, MSO and Cyth defined requirements for two physical form factors of a system running the same software stack.

1. Circuit Breaker Analyzer (CBA)

   1. A portable unit that utility maintenance teams take onsite to perform necessary measurements during scheduled maintenance events

   2. Using Lens Client™ software on a laptop to configure event thresholds and consume pre-trigger and post-trigger data published by the CBA for immediate analysis, charting, and upload to the central Lens Enterprise™ Server.

   3. NI CompactRIO hardware in a portable, ruggedized enclosure

   4. Standard connector enables connectivity with any brand of circuit breaker
      

1. Circuit Breaker Online Monitor (CBOLM)

   1. Permanently installed enterprise IIoT solution that interfaces with both the centralized server and clients to the server

   2. Continuously measures key circuit breaker health indicators and selectively logs data based on pre-determined thresholds

   3. sbRIO platform was chosen for its compatibility with the RIO software stack, smaller physical and energy footprints, cost point at volumes of scale, and measurement performance in harsh and fluctuating outdoor environments

   4. Datalogging is triggered by circuit breaker events

   5. Pre-trigger and post-trigger data are logged to the Lens Enterprise™ Server

   6. Standard connector enables connectivity with any brand of circuit breaker

The CBA and the CBOLM are both based on the NI RIO platform, enabling re-use of the same software stacks and codebases across both system types.

MSO developed the solution for interfacing between the utility provider’s IT infrastructure and the NI RIO-based data acquisition and analysis systems. They created a daemon to run on the NI Linux Real-Time Operating System, responsible for event threshold detection, securely logging event data, and publishing to the remote server.

Cyth developed a custom CircaFlex daughterboard to mate with the NI sbRIO-9651. The CircaFlex board enabled Cyth to incorporate the signal conditioning and connectivity needed to ensure data integrity and system reliability.

The data acquired by the CBA and CBOLM is eventually pushed to a central database. The cRIO-9053 in the CBA unit acquires data, which is then passed to a PC running MSO’s Lens Client™ software, which analyzes the data, charts it for user review, and transfers it to the central database.

The NI sbRIO-9651 in the CBOLM continuously reads the signal trace inputs. When a circuit breaker event occurs, the sbRIO triggers data logging to capture and send the data immediately before and after the event via MQTT to the central server. The published data is inserted into the central database, analyzed, and notifications to maintenance personnel are made, if required. This data is then available for on-demand and remote viewing, so maintenance experts can retroactively view and analyze circuit breaker behavior and events to decide on the optimal path to resolution.

MSO’s code compatibility with the RIO hardware on both the CBA and CBOLM, combined with Cyth’s field-tested software architecture, helped accelerate development of both solutions and will help ensure system sustainability well into the future.

The implementation of MSO’s circuit breaker monitoring solution delivered significant advantages across multiple operational dimensions:

• Hardware Customization with Software Flexibility: Commercial-off-the-shelf (COTS) hardware customized through software down to the transistor level provided exceptional measurement quality without sacrificing reliability

• Code Portability: Existing cRIO code was seamlessly transitioned to the sbRIO platform, reducing development time and validation requirements

• Optimized Connectivity: The 320-pin connector of the sbRIO-9651 system-on-module (SOM) provided superior integration capabilities for long-term deployment

• Scalable Architecture: This “semi-custom” approach delivered an ideal balance of performance and cost-effectiveness for large-scale deployment across thousands of units

• Comprehensive Integration: NI software libraries enabled rapid I/O integration and communications protocol support, while Real-Time Operating System capabilities ensured optimal, reliable performance

  
  

Ultimately, MSO’s enterprise software solution enabled the utility provider to:

• Constantly acquire and analyze key indicators in circuit breaker events

• Incorporate numerous custom trigger definitions to enable the detection and reporting of asset health and maintenance concerns continuously

• Design in safeguards, like short-term onboard storage, to help protect against data loss when a system gets disconnected from the IIoT

• Deliver accurate and timely alerts through real-time analysis and reporting capabilities

Improved Operational Efficiency

Over the expected 20-year operating life of the CBA and CBOLM systems, MSO’s end-customer, the utility provider, stands to save ~$400 million, with a nominal payback period of six years. A large portion of cost savings can be attributed to increased service reliability, enhancing grid stability and performance.

The installation of CBOLM units across the utility service area is expected to:

• extend the life of their existing circuit breaker population by one to three years

• dramatically reduce the outages experienced by customers

• reduce the need for unplanned, reactive maintenance events

Performing maintenance at the first sign of abnormal, or out-of-tolerance, circuit breaker behavior can help reduce further damage or deterioration. CBOLM systems provide an unprecedented level of data insights into circuit breaker health, enabling the utility provider’s maintenance teams to prioritize the assets with the greatest need and intervene before a damaging or catastrophic failure occurs. These need-based insights and maintenance prioritization are expected to increase average circuit breaker life by 1-3 years.

The availability of proactive, data-driven, condition-based service recommendations fundamentally reshaped service scheduling, enabling the utility provider to prioritize maintenance based on actual equipment condition rather than rigid, predetermined intervals. Reducing unplanned maintenance has enabled this utility provider to lower asset maintenance costs and, most importantly, limit risks to maintenance personnel by reducing unplanned maintenance events.

Because the CBOLM system captures and logs data whenever a circuit breaker event occurs, it dramatically reduces the number of maintenance events needed to ensure regulatory compliance. Maintenance teams can access circuit breaker operating data via the centralized server and report on breaker regulatory compliance without traveling to the site.

This enhanced approach to monitoring and managing assets will deliver substantial, long-term cost savings for this utility provider through improved operational efficiency, enhanced regulatory compliance, and a designed-in approach to proactive maintenance.

Through transforming complex equipment monitoring needs into actionable intelligence, MSO and Cyth Systems empowered the utility provider to set new industry standards for predictive maintenance and grid management.