How to Build an Embedded Datalogger | Cyth Systems

Introduction

Traditional dataloggers often fall short in field deployments and complex industrial environments where measurement flexibility, real-time processing, and ruggedization are critical. Cyth’s approach to designing an embedded datalogger addresses these limitations by leveraging the NI CompactRIO platform and LabVIEW to create a portable and intelligent automated data acquisition system.

• Challenge: Closed-design dataloggers can lock you into fixed configuration and lack the flexibility for expanded measurement coverage, powerful inline processing, and fully customizable data formatting for recall and analysis.

• Why We Built This: Our approach to building an embedded datalogger provides flexible performance for a wide range of IIoT and field applications, meaning you don’t need to conform to closed tools. Here are some core tenets of the design:

  
  

  • Headless Operation: Runs independently without requiring a connected host machine or data server.

  • Intelligent Processing: Combines real-time processing and FPGA-linked I/O modules for flexible triggering, synchronized measurements, and datalogging logic controls.

  • Data Access and  Communication: Solid-state storage and built-in Ethernet networking capabilities for remote monitoring and data retrieval

Section 1 - Why Use an Embedded Datalogger?

Aggregating measurement data from the field

While measurement automation systems often start on the bench or in the lab, they frequently need to be deployed to collect data and test field systems. The ability to combine I/O and interface alongside compute and storage can be critical to validating how a machine or product performs in its intended environment. Collecting this field builds test and performance insights for use cases that cannot be mimicked in simulation environments or controlled laboratory settings.

Embedded dataloggers provide several key advantages:

• Deployment flexibility – Embedded dataloggers can operate is harsh conditions without external dependencies other than a power source. Deploy in remote, mobile, and space-constrained environments where traditional setups are impractical or impossible.

• Appliance-Level autonomy – Without the need for dedicated host computers, embedded dataloggers remove overhead and points of failure for critical datalogging operations. Systems can be designed to start logging immediately upon power-up and can be programmed with custom processing and logging conditions for specific applications.

• Real-time performance – Real-time operating systems and FPGA-based processing deliver microsecond-level precision for time-critical operations outside the bounds of CPU-only benchtop systems.

• Reliability – Industrial-grade components and security layers at the top of the stack enable long-term deployments in field environments where maintenance access may be limited.

Section 2 - Common Applications

Dataloggers can provide performance insights in both monitoring and test applications. Here are some examples we’ve come across:

Industrial Monitoring

• Continuous vibration and RPM monitoring on rotating equipment and trend tracking for predicative maintenance

• Power quality monitoring energy-intensive production facilities and data centers

• Environmental monitoring in harsh field conditions with multi-measurement sensing and configurable alarm thresholds.

  
  

Mobile Design Validation Test (DVT)

• Vehicle testing programs integrating dynamic performance measurements, CAN bus telemetry, and GPS location.

• Aerospace flight test systems with high-speed data acquisition and avionics control.

• Field research data collection for heavy machinery

• Temporary monitoring installations, inclusive of machine vision

IIoT Integration

• Edge computing nodes performing AI inference on machine data and local decision making

• Standalone intelligent, mixed-signal sensors with wireless connectivity

• Test station health monitoring and factory 4.0 intelligence

Section 3 - Datalogger System Features

While the specific behavior of the datalogger ideally matches the application requirements, we have identified some common capabilities that make embedded dataloggers powerful and scalable tools.

• Flexible Triggering – configure trigger sources, thresholding, and data synchronization

  
  

• Multi-channel synchronization – Interleave multichannel with timestamp alignment for post-analysis and correlation tracking

  
  

• Advanced Data Handling – Real-time pre-processing and filtering to log conditionally for smart storage utilization to protect system-level signal-to-noise

  
  

• Data Formatting – Design for downstream compatibility and recall with metadata and plugins

  
  

• Integrated Diagnostics – Heartbeat monitoring for health verification, automated error detections, and remote troubleshooting support.

Section 4 - How Cyth Designed an Embedded Datalogger

Cyth has developed a number of dataloggers over the years and while each has unique requirements which must be met, we come back to a common core of flexible functionality built on the NI CompactRIO platform and LabVIEW.

Below is a more detailed overview of how we design and optimize embedded dataloggers:

Core Requirements, I/O Coverage, and Sensor Selection

While many monitoring and test systems begin with an understanding of what needs to be measured, mapping these requirements to an implementation often becomes a challenge.

CompactRIO supports a wide variety of analog, digital, signal-conditioned, and interface I/O in their C Series modules. The modular nature of the platform makes adding channels or completely new measurements rather straightforward, meaning you can standardize on the platform  and have confidence in continued viability even through changing requirements.

Connectivity and Ruggedization

When designing for connectivity, there’s a number of related topics that arise.

1. Physical connectivity – How will the sensors and interfaces connect to the datalogger? Consider strain relief, cables, ruggedized connectors for durability and signal integrity requirements.

2. Data access and retrieval – How will data consumers access the data during and/or after the datalogging interval? Consider onboard storage limit, peripheral ports, network and NFC interfaces, and UIs.

3. System updates and troubleshooting – How do we push any firmware, measurement code, and security updates to the system? Consider overall software stack, Docker, CI/CD tools

4. Portability, mounting, ergonomics – Will this embedded datalogger need to be fully portable, or will it ever need to be mounted in place? Consider mechanical design for shock & vibration, enclosures, overall ruggedization.

The CompactRIO controller provides -40 °C to 70 °C operating temperature, IP20 rating, and robust shock and vibration specs in a passively cooled form factor, making it a versatile platform for a wide range of deployment environments. For some applications, additional ratings necessitate further enclosure design and ruggedization for NEMA Class 1, Division 2 protection.

The controller also has designed-in mounting points, accessories, and downloadable schematics helping design for these additional protections or more straightforward mounting.

Datalogging Software

While the I/O, connectivity, and mechanical design are certainly important, the software is the key to usability and data insight. To put it another way, the software is the key to unlocking a datalogger’s true potential.

While working with a closed software system can have some short term benefits, we have found that they often reach their limits well before the entire job is done. For this reason, Cyth recommends a more open approach to software development, centered around LabVIEW, TDMS, and design patterns proven over countless projects.

Through the NI RIO platform, LabVIEW unlocks access to a rich ecosystem of I/O, advanced control & monitoring schemes, and custom data processing algorithms, all while operating heedlessly.

For applications not requiring headless operation or real-time performance, CompactDAQ and FlexLogger offer a range of data acquisition, configuration, and logging capabilities.

Some of the descriptions below may benefit from a basic understand of the CompactRIO platform.

Core Architecture

While there are many design patterns available, Cyth typically uses a Queued Message Handler (QMH) design pattern orchestrated through queues and notifiers to drive an advanced state machine. Here are some common requirements:

• Continuously register state change conditions through both I/O and logic

• In-state data processing and logging

• Multi-loop synchronization (control and processing)

• Notifier scheme to wait on new data before initiating a process

• Operator download mode via Data Streaming API over Ethernet

FPGA Triggering and Processing

One of the most powerful features of the RIO platform is access to an FPGA directly linked to both I/O and headless CPU processing. The approachability of FPGA programming in LabVIEW opens doors for high-performance, purpose-built embedded datalogger features.

Simpler triggering schemes can also be setup and run using the DAQmx API and digital lines available through the controller and modules.

Embedded dataloggers can also utilize FPGA fabric to process data before a logging step to either optimize dataset sizes and simplify post-processing. Common use cases are:

• Filtering

• Peak detection

• Transient analysis

• Timebase processing

Data Handling & TDMS

The key consideration here is ease of data recall and integration with downstream processing tools. Engineering teams must consider data formats, metadata, plugins, and existing processing IP and dashboard. While CompactRIO controllers can run AI inference and perform local decision making, this topic is out of scope for the application note.

Section 5 - Tying it all Together

The overall usability of embedded dataloggers can benefit from system-level software, deployment packages, and troubleshooting tools.

In terms of deployment control, a CompactRIO target runs headless off a real-time executable (.rtexe), that can be configured to launch on boot. IF changes need to be made to the code, the executable can be re-built and pushed over the network (with credentials) or through a boot drive. Peripherals ports on the controller support embedded UIs with controls, indicators, and graphs which can be designed to include operator and troubleshooting diagnostics.

Alternative Approaches

CompactDAQ, FieldDAQ, and FlexLogger are all tools that could also be used for distributed datalogging.

Conclusion

Cyth’s approach to embedded datalogger design has proven to help engineering teams build and deploy high-performance, distributed measurement systems. Uniting the flexibility of LabVIEW with the modular reliability of CompactRIO creates solutions that are capable of advanced triggering, inline data preprocessing, data storage, and networking all with the potential for headless operation. These features are critical for a wide range of validation testing, industrial monitoring, and IIOT-ready systems requiring advanced system performance in the field.