Real-time: Convert Hierarchy to Array

What Does This Article Cover?

The payload provided by an Intelligence Hub Connection Input might have a hierarchical structure. In some cases, it is beneficial to process the payload as an array. This article describes solutions to achieve this.

General design considerations

Depending on the Intelligence Hub Connection Input and source system it might be beneficial to use the include metadata option to assign the value to the generic attribute name "value" and obtain the source's timestamp.
Determine if JSONata, JavaScript, or Pipeline loop stages are best for converting the hierarchy.
Use an Intelligence Hub Model to define a schema if necessary.
The overall data pipeline solution might involve polling at a frequency or calling a pipeline externally.
The resulting data payload can be written to a MQTT broker, database, data lake, or data warehouse, or made available by the Intelligence Hub REST Data Server.

Convert OPC UA Hierarchy to an array using JSONata

These considerations pertain to an OPC UA source like KEPServerEX for example.

Depending on the OPC UA tag hierarchy configuration an Intelligence Hub OPC UA Connection Input might return a complex hierarchical JSON object.
It might be desirable to process the data as a JSON array.
The Intelligence Hub Pipeline JSONata stage can be used to do this.

Changing a data payload from looking like this.

{
    "Line01": {
        "Motor01": {
            "Amps": {
                "value": 98.42495904316564,
                "_tagName": "2/1/[default]/Assets/Line01/Motor01/Amps",
                "_tagQuality": "Bad",
                "_tagTimestamp": 1762799054913
            },
            "Speed": {
                "value": 96,
                "_tagName": "2/1/[default]/Assets/Line01/Motor01/Speed",
                "_tagQuality": "Bad",
                "_tagTimestamp": 1762799054913
            }
        }
    },
    "Line02": {
        "Motor02": {
            "Amps": {
                "value": 99.80155319986743,
                "_tagName": "2/1/[default]/Assets/Line02/Motor02/Amps",
                "_tagQuality": "Bad",
                "_tagTimestamp": 1762799054913
            },
            "Speed": {
                "value": 98,
                "_tagName": "2/1/[default]/Assets/Line02/Motor02/Speed",
                "_tagQuality": "Bad",
                "_tagTimestamp": 1762799054913
            }
        }
    }
}

To an array that looks like this.

{
    [
        {
            "value": 99.65827234427421,
            "path": "Line01/Motor01/Amps",
            "time": 1762799453775
        },
        {
            "value": 77,
            "path": "Line01/Motor01/Speed",
            "time": 1762799453775
        },
        {
            "value": 102.46419553603752,
            "path": "Assets/Line02/Motor02/Amps",
            "time": 1762799453775
        },
        {
            "value": 70,
            "path": "Assets/Line02/Motor02/Speed",
            "time": 1762799453775
        }
    ]
}

By using this code in the JSONata stage.

event.value.**[value and _tagName and _tagTimestamp].{
  "value": value,
  "path": $replace(_tagName, /^2\/1\/\[default\]\//, ""),
  "time": _tagTimestamp
}

Alternatively, Intelligence Hub Pipeline loop stages can be used to provide a solution. An example of this approach is available in the project file [here].

Convert Sparkplug Hierarchy to an array using JavaScript

These considerations pertain to a Sparkplug source like Cirrus Link MQTT Distributor for example.

Depending on the Sparkplug hierarchy configuration an Intelligence Hub Sparkplug Connection Input might return a complex hierarchical JSON object.
It might be desirable to process the data as a JSON array.
The Intelligence Hub Pipeline JavaScript stage can be used to do this.

Changing a data payload from looking like this.

{
        "_group": "DairyGroup",
        "_edgeNode": "PackagingArea",
        "_device": "DairyDevice",
        "value": {
            "Overview": {
                "Motor 1": {
                    "Amps": 100.8266323324433,
                    "HOA": 98.73531511261912
            }
        }
    }
}

To an array that looks like this.

{
    [

        {
            "value": 101.23964666359804,
            "attribute": "Amps",
            "path": "DairyGroup/PackagingArea/DairyDevice/Overview/Motor 1"
        },

        {
            "value": 101.23964666359804,
            "attribute": "Amps",
            "path": "DairyGroup/PackagingArea/DairyDevice/Overview/Motor 1"
        }
    ]

}

Using this JavaScript

// incoming payload
let payload = event.value;
// base path from top-level metadata
let basePath = `${payload._group}/${payload._edgeNode}/${payload._device}`;
// recursive function to flatten all metrics
function flatten(obj, path = basePath) {
  let result = [];
  for (let [key, val] of Object.entries(obj)) {
    let currentPath = `${path}/${key}`;
    if (typeof val === "object" && val !== null) {
      // if it's an object, recurse deeper
      result = result.concat(flatten(val, currentPath));
    } else {
      // if it's a primitive value, capture it
      result.push({
        value: val,
        attribute: key,
        path: path
      });
    }
  }
  return result;
}
// perform the flattening
let flattened = flatten(payload.value);
stage.setValue(flattened);

Alternatively, Intelligence Hub Pipeline loop stages can be used to provide a solution. An example of this approach is available in the project file [here].

Summary

Use these considerations to convert a hierarchical JSON structure to an array using the JSONata Pipeline Stage with Intelligence Hub. Implement an approach above and align your pipeline’s schedule, modeling, and outputs with your downstream requirements.