Author: Joe Spinozzi Weblink: Click Here Publication TestStand Advanced Architecture Series Date: 2008-12-18 NI TestStand's step type architecture allows developers to create innovative and unique custom step types that meet the needs of your particular environment.  This document explains custom step type development and provides a set of best practices to help you get started making better custom step types faster.

TestStand includes numerous built-in step types to create steps in a test sequence. Some step types call code modules, specify how the step executes, and specify how the step evaluates data. Other step types include self-contained behavior, such as the Message Popup, Call Executable, FTP Files, and other steps.

You can use the architecture of the built-in step types to create innovative and unique custom step types that meet the needs of your particular environment.

Unfortunately this freedom can also lead developers to create custom step types that are hard to share, that conflict with other step types, that are difficult to upgrade, or that might cause type management issues in the future. In addition, the default structure of step properties can be challenging to understand fully at first, which can make it difficult to decide how to architect and implement a custom step type. Getting started with custom steps can be a steep learning curve.

This document explains custom step type development and provides a set of best practices to help you get started making better custom step types faster.

Anatomy of a Step Type in TestStand

To construct your own step type in TestStand, it is fundamentally important to understand how existing step types operate.

You might be surprised to know that the diverse step types in the Insertion Palette are created in the same way from a single customizable object in TestStand, as shown in Figure 1. As a developer, you can use this same model to create your own custom step types.

Figure 1 –Step Type Structure Used to Create Insertion Palette Step Types

When you select a step type from the Insertion Palette and place it in a sequence, TestStand create a unique instance of that step type as a step. The newly created step includes many unchangeable properties of the step type that cause the step to behave and appear in a specific way.

Each step includes a collection of values called property objects that store settings and measurements unique to that step during edit time and run time. When you create a new step, TestStand sets the value of some of these property objects to initially define the basic properties for the type of step you create. The designer of the step type defines the default properties and settings of the step type. Step type properties are the first element of the step type structure.

Because each step is a unique instance of a step type, some default step settings might need to change. The configurable step settings allow steps that share the same step type to behave differently. For example, you can configure Message Popup steps with different messages, fonts, colors, and images to make each step unique from other Message Popup steps.

Each step type also includes a collection of Disable Options, which are the second element of the step type structure. You might need to disable some settings to prevent changes to the settings so you can ensure that the step operates as intended and that all steps of the same step type share a particular setting. For example, the Numeric Limit Test step uses the Status Expression to determine if the step passed or failed. Because the step depends heavily on that expression, TestStand disables editing the expression.

When you modify a custom step type, you should update some features in all instances of that step type. For example, if you modify the default Step Description property, you should update all the steps of that type . However, Because the Comment property for each step is generally unique to that step, you do not need to update it in each step when you modify it. Table 1 includes an example of the built-in step properties that are shared across all step type instances.

* Not a Type Property dialog tab

Table 1. Step Type Properties Dialog Box for the Pass/Fail Test Step

Some of the built-in properties are not visible in the Types window or in the Property Browser panel. Enable the Show Hidden Properties option on the Preferences tab of the Station Options dialog box to view these types of properties. By default, only properties in the Result container are visible. The Result container by default includes the following properties to store in memory, which TestStand uses to populate the ResultList during the execution of a sequence:

• Error — Property object that contains the error information for the execution
• Status — String that contains the status of the execution
• ReportText — String that contains text to display in the report
• Common — Property object that contains results common to all sequences on the station

You can add properties to the Result container that you want to keep in memory during the execution of sequence.

The step type structure also offers several categories of code module adapters, called substeps, that allow the step to invoke code modules, which provide the step its functionality at edit time and at run time. The step type structure includes the following categories of substeps, each with different fundamental abilities and characteristics: Edit, Pre-Step, Post-Step, and Custom. The first three substeps have a specific behavior that govern how and when they execute. Custom substep designers manage substep behavior dynamically.

The Edit substep is the most unique of the substeps and is probably the first one you encounter when using a step type. It is unique because it is the only one designed to be used during sequence editing and not at run time. You can use the Edit substep to interact with sequence editors to collect information about how the step should operate by using the context menu of each step or the Step Type Edit tab of the Step Setting pane. For example, when you create an instance of the Statement step type and a Sequence Call step type, as shown in Figure 2, notice that the Statement step includes an Edit Expression context menu item and the Sequence Call includes an Edit context menu item, which is disabled, to indicate that the Statement step type includes an Edit substep but the Sequence Call step type does not.

Figure 2 – Steps that Include an Edit Substep Have Edit Context Menu Items

The Edit substep launches a graphical user interface (GUI) in which you can modify the variables or settings of that step instance during development time. The modifications can persist in the sequence file. An Edit substep usually calls a code module that executes a user interface. In the Statement Step example in Figure 2, when you select Edit Expression menu item, TestStand launches interface in which you can create an expression, which TestStand stores and then evaluates when the step executes. In TestStand 4.0 and later, you make changes to built-in step types on the Step Type Edit tab of the Step Settings pane, as shown in Figure 3. In previous versions of TestStand, you make changes to step types in a separate dialog box, as shown in Figure 4. A step type can include multiple Edit substeps to provide more than one editing interface.

Currently, you cannot embed the editing interface for custom step types into the Step Settings pane. Instead, as shown in Figure 3, each Edit substep appears as a button in the Step Settings pane. You can right-click and select the Edit Menu item from the context menu or click the Edit button in the Step Settings pane to modify custom step types.

Figure 3 – The Edit Substep Launches a GUI in Step Setting Pane or Provides an Edit Button to Access the GU.

Figure 4 – Editing Interface for Custom Step Types, Similar to TestStand 3.5 Editing Interface

The next three substep categories operate at run time when the step executes. Figure 1 shows these three substeps in relationship to each another. When the step executes, the Pre-Step substeps executes first, then the code module executes, then the Post-Step substeps execute last. If more than one Pre-Step or Post-Step substeps exist, they execute in the order they appear on the Substep tab of the Step Type Property dialog box. Figure 5 shows an example of the Multiple Numeric Limit Test step, which includes a Post-Step substep. After the code module runs, the Post-Step executes to evaluate the numeric results against numeric limits.

Figure 5 - Post and Edit substeps of the Multiple Numeric Limit Test Step Type

Figure 6 shows the four built-in Test step types and an Action step type. The design of each step type controls how the step behaves. You can configure these step types to call code from LabVIEW, C DLLs, ActiveX, or other programming environments.

Figure 6 - Built-in Steps for Calling Code Modules

The Action step type calls a code module, in which you can specify the input values and determine what to do with the output data. The other four step types share the same behavior, but you can also use them to analyze the resulting data using string or numeric comparisons. These step types provide the foundation for common test systems.

You can use the Types Window, as shown in Figure 7, to study the design of built-in and other step types, as shown in Figure 8.

Figure 7 - Types Window Provides Access to Step Type Properties

Figure 8 –Step Type Properties Dialog Box

Creating custom steps can make test development easier, can help individuals or groups share code modules, and can more carefully control the execution of steps.

Best Practices for Creating Custom Step Types

Creating custom step types requires a number of tasks, especially when you create step types from scratch. A good course of action when creating custom step types is to plan the design, implement the design, and then examine the run-time behavior and usability of the step type.

Planning Custom Step Types

• Consider modifying a step type or creating a new custom step type in the following situations:
• Built-in step types do not meet your needs.
• Test specifications include common, repetitive actions that must occur before or after code modules, such as setting up the step or analyzing the results.
• You want to simplify the user experience for configuring steps with the help of a user interface.
• You want to expert a common action for other groups, companies, or customers.

Developing a custom step type is a significant investment that you should compare to the value the step will provide by reviewing the return on investment (ROI). Use the built-in step types when they meet your needs so you can complete the project sooner and with fewer hours of effort. A well-designed step can make sequence development faster, can reduce debugging efforts, can allow developers to share standardized code, and can achieve consistency among multiple test stations and separate groups. However, custom step types can require a significant amount of time to plan, program, debug, deploy, and maintain.

Using Templates Instead of Custom Step Types

You can create step templates by developing and configuring steps in a sequence and then dragging and dropping those steps into the Templates List of the Insertion Palette. TestStand stores a copy of the step instance as a template you can reuse to quickly create new sequences by dragging and dropping the step template into a new sequence.

The difference between a step template and a custom step type is that you can create a template only from existing step types and the template includes only the same capabilities of the original step type. You cannot modify or redesign the step template to the extent you can a custom step type. For steps you add to the Templates List, you can configure only the settings enabled by the developer of the original step type. In contrast, you can use a new custom step type to create a completely new step with entirely new behavior. Also, changes to a step template affect only future instances of the step and do not change existing instances of that step. Changes to a step type affect all instances of the step type.

One benefit of using a step template is to avoid customizing step settings more than once if you reuse the same step in the same way. For example, if you set an IVI Power Supply to 5V and then to 3.3V and plan to do that many times throughout the entire sequence, creating two step templates after you initially use and configure the steps can save time. However, if you wanted to make a step first configure the power supply and turn it on or off, creating a custom step type is a better approach.

In summary, use step templates to repeatedly insert an existing step type with the same preferences and configurations. Use a custom step type to achieve more customization, to change behavior of steps, or to create new behavior.

Planning the Functional Scope of Custom Step Types

One of the first things to consider when creating a custom step type is the scope and capability of the step. For example, consider creating a step based on the Numeric Limit Test step type to check more than one set of limits. You must decide how many limits the custom step will check. The existing step type checks one set of limits, but the custom version could check two or more if you design it that way. Creating a flexible step can make custom steps more valuable and useful and can limit the support and modifications needed in the future. However, it’s also wise to keep custom step types as simple as possible or to create more than one step to accomplish a set of goals. As a custom step type becomes more complex, the ROI can diminish.

Storing Custom Steps

Another consideration is how to use the custom steps you create. National Instruments recommends creating all step types in a type palette file, not within a sequence file, because TestStand searches the type palette files for step type updates when you load a sequence file. TestStand also helps you manage the reuse of steps by keeping a copy of each step type used inside a sequence file. If you deploy the sequence file without the type palette file, the sequence file still contains a copy of the step type. Refer to the Best Practices for NI TestStand Type Management document for more about type management. .

Determining Code Variables

As soon as you determine the functional scope of the custom step type, brainstorm all the possible variables the step might need and how to organize those variables. In the previous Numeric Limit Test step example, if you limit the overall scope to two sets of limits (such as Customer and Internal Limits), perhaps two containers to handle the data are sufficient. However, if you decide not to restrict the number of limits or you decide that more limits might be a good idea for the future but not at the current time, it might be wise to create an array of containers. Working with an array variable is more complex than working with a container, so you need to decide if the extra work now for a feature you might not use until a future time is a wise investment. Selecting the data storage and I/O can be a critical part of the preparation and design of custom step types because a wisely designed step can be easier to modify and maintain.

Using Expressions to Maximize Flexibility

Use the Edit substep to provide a user interface to modify settings and variables for the steps you create. Using expressions is a flexible way for users to interact with data, but working with the Expression control can require a larger investment than working with string or numeric controls. However, modifying a custom step type to add expressions later requires more effort than designing it with expressions the first time. Figure 4 shows the difference between older versions of TestStand and TestStand 4.0 and later, which use the Expression controls. To make your custom steps more consistent with built-in TestStand steps, consider using Expression controls.

Figure 9 – Expressions are more Flexible than Fixed Values for Specifying Settings

Creating Useful Descriptions

If you use the sequence editing landscape wisely, you can create partially self-documenting sequences. However, when you work on a sequence, you can use only a few fields to describe how the sequence works, and the description field is not user-editable. Therefore, as a custom step type designer, create useful descriptions and leave the StepName and Step Comments fields for other purposes. For example, the Step Description in the lower example in Figure 10 is more descriptive and provides better documentation in less space.

Figure 10- Step Description Examples

Use Disable Properties with Care

When you release a step you develop, you can protect the behavior of the step by disabling the editing of certain settings. For example, the built-in For step disables most of the settings because they do not apply to the function of the step. However, preventing users from editing step settings can limit flexibility, so disable settings carefully.

Use Pre- or Post-Step Substeps Instead of Default Module

Implement the code module for the basic operations inherent to the step type as a Pre- or Post-Step substep instead of as a default module step. Use the default module setting only when each instance of a step can call a different code module. The default module setting exists separately on every instance of the step, and TestStand does not update existing step instances by default when you change the setting on the step type. However, substeps exist only in the step type, and all instances of the step refer back to the substeps. Changes to substeps automatically affect all existing instances of the step type.

Best Practices for Implementing Custom Step Types

Set Variables without Using ContextReferences

Code modules modify the step variables during the edit phase or during execution time. You can use the Context Reference of the step and use the appropriate PropertyObject.SetVal method of the TestStand Engine API to dynamically manipulate variables. You can also use the Specify Module tabs and dialog boxes with a code module to use the tools of the module adapter to retrieve and modify variable values without programming. Using the Specify Module tab or dialog box is less likely to introduce errors and require troubleshooting. figures 11 and 12 compare the two methods of modifying step variables.

Figure 11 – Editing Variables from within LabVIEW using the Sequence Context Requires Adding Some or All These Methods to Every Code Module for Each Variable Used

Figure 12 - The Module Adapter Reads or Write Step Variables to the Inputs and Outputs of Code Modules and Checks for Errors

Unload Module Option while Editing

When you edit the code module of a custom step type, TestStand loads the code module into memory when you run the step. Once the code module is in TestStand’s memory, you cannot edit the code module until TestStand unloads the code module. Therefore, during development and testing, unload the code module as soon as the step or execution completes. After development completes, reset the Unload Option back to the default value so the step does not load and unload the code module more often, which can slow execution.

Use Multiple Edit Substeps

You can use multiple Edit substeps when the editing interface becomes complicated to break the interface down into more than one window. Notice that the Multiple Numeric Limit Test step includes two tabs for editing the source of the numeric data and the limit conditions for each data source. However, because you cannot embed the editing interface for custom step types into the Step Settings pane, remember that using multiple Edit substeps requires users to navigate through multiple dialog boxes to configure the step. Consider creating a single Edit substep and using multiple tabs to help organize and simplify the configuration.

Make Steps Modal to TestStand

Always make Edit substeps and other user interface code modules modal to TestStand because when TestStand calls Edit substeps, it disables the sequence editor. If code modules are not modal, the TestStand window can hide the code modules. Users might think the sequence editor crashed and might try to terminate TestStand.

LabVIEW includes several settings that control the appearance of a VI called from TestStand. Figures 13 and 14 show how to make a LabVIEW VI modal to TestStand. Other programming languages include internal methods for accomplishing the same behavior.

Figure 13 – Settings for Modal Behavior on the Module Tab of the Step Settings Pane

Figure 14 – Code Required for Modal Behavior in LabVIEW VI

Provide Visual Feedback

TestStand waits for code in Pre- or Post-substeps to execute before continuing. If code modules for those steps operate slowly or silently, TestStand might seem unresponsive. If you anticipate this situation, provide an indicator, such as a spinning icon or a progress bar, to communicate to users the status of the executing code. You can also change the cursor or use UI messages, such as UIMsg_ProgressPercent, to update an indicator.

Provide an Abort Option

If a step executes run-time code that takes a significant amount of time, include an Abort button or other input to programmatically terminate TestStand so users can abort or exit the code and the TestStand internal termination or abort management system can clean up steps, close other windows, and record results properly. If that code is modal, it might be impossible to use the built-in abort and terminate options within TestStand menus and user interfaces.

Use the Termination Monitor

Code modules you develop should include and periodically poll a termination monitor to gracefully handle when users terminate or abort sequence execution using the built-in options in TestStand or in the TestStand API.

Handling Errors with TestStand

TestStand handles errors code modules return and manages the execution of the sequence based on those errors. Use the code module adapter to return the error to the Error Out container of each step in TestStand. Additionally, rewrite error messages in your code modules to include helpful information to users, such as " Unable to establish communication with the motor" instead of " VISA Error in myfile.vi".

Best Practices for Modifying or Updating Custom Step Types

Add a New Property with a Different Type

If you need to change the type of a property, create another property with the new type and retain the property with the previous type. If you change the type of a property, TestStand replaces the value of the property in step instances with the default value of the property. In addition to creating a new property with the new type. add logic to the step type to handle cases where a step uses the old property and the new one. For example, when TestStand implemented limit values as expressions, two new Boolean properties were added to specify to use the old numeric limit property. The UseLowExpr and UseHighExpr properties determine if the step evaluates the old numeric limits or the new expression limits, as shown in Figure 15.

Figure 15. Numeric Limit Test Properties

Do Not Rename Properties in Custom Step Types

Do not rename properties in step types because TestStand renames the property in all step instances and replaces the value with the default value for the property. Renaming a property can also affect any Edit substeps unless you update the code in the Edit substep to reference the new name of the property.

Conclusion

Working with custom step types can be challenging and can have powerful rewards. Review the instructions and tutorials in Chapter 13, Custom Step Types, of the NI TestStand Reference Manual and in this document before you create your own custom step types.

About the Author

Joe Spinozzi – Cyth Systems LLC

Joe Spinozzi is the co-founder and director of operations at Cyth Systems. With 12 years of experience helping dozens of companies develop products and test systems, he now leads a team of engineers to plan, architect, and complete projects across many industries. Operating from San Diego, California, Cyth Systems is responsible for test systems large and small currently operating in the United States, Europe, and Mexico.