How-To: Monitor Intelligence Hub Component Health
Introduction
Monitoring Connections and Pipelines can be crucial in manufacturing and Industrial IoT architecture. This guide provides a step-by-step process to monitor Connections and Pipelines and transmit their state data to your preferred output. In this guide, we will send the status data to MQTT so you can view the output via our UNS Client. However, you can also use this guide to send the payloads to another connection you've configured within the HighByte Intelligence Hub, access status data in an instance using our REST data server, or output the data through a REST output to third-party tools like Splunk or Datadog.
Step-by Step Guide
- Select and output for your flow
- Determine the destination where you want to send the status of a connection or pipeline too.
The sample project will use 'Status/' for a MQTT topic path. If you emulate this in your own project, the name of your pipeline and/or connection will appear in MQTT
- Determine the destination where you want to send the status of a connection or pipeline too.
- Navigate to Pipelines and Utilize the Pipeline Wizard
- Select 'New Pipeline'
- Name your pipeline and select 'Build Flow'. This is the Intelligence Hub pipeline wizard.
- Reference the component to be monitored as a Source
- Under references choose the 'System' Type.
- Next, for System Type select 'Input'
- From here you are able to reference your Connections or Pipelines. You can expand out the references for a Connection or Pipeline to reference specific metrics.
- Reference your Target
- In the reference panel select 'Output' for Type.
- From here select the output you want to send your data to.
Example Output Data
Pipeline Data
{
"StringData": {
"name": "Example_Pipeline",
"lastError": "",
"state": "Good",
"triggerStatus": [
{
"name": "Example_Flow",
"lastError": "",
"pipelineName": "Example_Pipeline",
"state": "Good",
"healthy": true
}
],
"stageStatus": [
{
"name": "Targets",
"lastError": "",
"pipelineName": "Example_Pipeline",
"state": "Good",
"healthy": true
}
],
"statistics": {
"idle": true,
"totalRuns": 129,
"totalErrors": 0,
"queuedWrites": 0,
"lastCompleteTime": "2024-10-15T15:36:47.683Z",
"executingSamples": {
"meanMS": 4,
"minMS": 1,
"maxMS": 12,
"stdDevMS": 1,
"count": 129
},
"waitingSamples": {
"meanMS": 0,
"minMS": 0,
"maxMS": 4,
"stdDevMS": 0,
"count": 129
},
"stages": [
{
"stageName": "Targets",
"totalRuns": 129,
"totalErrors": 0,
"totalWarnings": 0,
"lastCompleteTime": "2024-10-15T15:36:47.683Z",
"executingSamples": {
"meanMS": 4,
"minMS": 1,
"maxMS": 12,
"stdDevMS": 1,
"count": 129
}
}
],
"triggers": [
{
"triggerName": "Example_Flow",
"totalRuns": 129,
"totalErrors": 0,
"lastCompleteTime": "2024-10-15T15:36:47.680Z",
"executingSamples": {
"meanMS": 987,
"minMS": 11,
"maxMS": 1008,
"stdDevMS": 86,
"count": 129
}
}
]
},
"healthy": true
}
}
Connection Data
{
"_name": "connectionStatus",
"_model": "ConnectionStatus",
"_timestamp": 1697488915351,
"name": "MQTT",
"lastError": "",
"status": "Good",
"statistics": {
"pendingWrites": 0
},
"healthy": true
}
Pipeline Data
{
"Pipelines": {
"Pipeline_Example": {
"name": "Pipeline_Example",
"lastError": "",
"state": "Good",
"stageStatus": [
{
"name": "Breakup",
"lastError": "",
"pipelineName": "Pipeline_Example",
"state": "Good",
"healthy": true
},
{
"name": "InputsOutputsFromInstance",
"lastError": "",
"pipelineName": "Pipeline_Example",
"state": "Good",
"healthy": true
},
{
"name": "Transform",
"lastError": "",
"pipelineName": "Pipeline_Example",
"state": "Good",
"healthy": true
},
{
"name": "Write",
"lastError": "",
"pipelineName": "Pipeline_Example",
"state": "Good",
"healthy": true
},
{
"name": "WriteNew",
"lastError": "",
"pipelineName": "Pipeline_Example",
"state": "Good",
"healthy": true
},
{
"name": "WriteNew_1",
"lastError": "",
"pipelineName": "Pipeline_Example",
"state": "Good",
"healthy": true
},
{
"name": "Transform_1",
"lastError": "",
"pipelineName": "Pipeline_Example",
"state": "Good",
"healthy": true
},
{
"name": "Breakup_1",
"lastError": "",
"pipelineName": "Pipeline_Example",
"state": "Good",
"healthy": true
}
],
"statistics": {
"idle": false,
"totalRuns": 3776,
"totalErrors": 0,
"queuedWrites": 1,
"lastCompleteTime": "2023-10-24T15:05:43.001Z",
"executingSamples": {
"meanMS": 114,
"minMS": 91,
"maxMS": 152,
"stdDevMS": 17
},
"waitingSamples": {
"meanMS": 0,
"minMS": 0,
"maxMS": 0,
"stdDevMS": 0
}
},
"healthy": true
}
}
}
Utilizing the REST Data Server to Obtain Component Health
You can utilize the REST data server to access component Health data.
Use the /data/v1/instances/{instanceName}/Value endpoint with to get a response body based on the configuration of your instance.
Utilizing the intelligencehub-events.log with an External Log Consumer
The Intelligence Hub captures errors in a JSON format within the intelligencehub-events.log. For example, if a particular flow encounters an error, this will be documented in the log. By default Intelligence Hub does not store system metadata for flows and connections in the log. To send system metadata to the log, navigate to your reference panel, set the 'Type' to 'system', 'System Type' to 'Output', and drag the 'Info' reference to your flow target.
By forwarding system metadata to the events log, third-party tools can access and analyze this log. This allows for monitoring the status of your connections and flows, regardless of whether they are in an error state. Therefore, this could provide a holistic view of the instance's health.
It's important to note that within your settings, under the 'Logging' section, there's an option to modify the 'Max File Size'. Adjusting this ensures that your log consumer retrieves all the necessary data from the file before any old data gets overwritten by new entries.
Attached Project
We've attached a project that demonstrates this process. Copy the code block below to your clipboard. To import the project, within the Intelligence Hub go to Manage -> Project. Set the Full Project flag to off and change Import Type to 'JSON'. Paste the copied code block and click 'Import'. Upon loading the project in make sure your MQTT broker is enabled by navigating to 'Settings'. If you need to adjust the MQTT Broker ports in settings, please also adjust the ports in the 'MQTT' connection that was imported with this project.
Within this project you will notice that there is a model with three attributes that is being used in an instance called 'Example_Instance'. This instance is used in the 'Example_Pipeline' that sends the instance's data to MQTT.
'Status_Pipeline' is a Pipeline that uses the system references of the 'Example_Flow' and all system connections as a source and flows it to MQTT. If you go to the UNS Client and connect to MQTT, you will see a Topic called 'Status' that contains state date on the 'Example_Flow' and any other connection set up in your project.
The project also has a "Model_Health_Instance" which models some of the data from the System. Component reference. In the "Model_Health_Pipeline" the Flow trigger is utilized the 'While True' Mode. The 'While True' expression is looking at the Status attribute to change from "Good". This will enable the flow to trigger when the status changes from good.
For demonstration purposes, data is sent to the local MQTT broker, with Model_Health_Instance monitoring an MQTT connection (our broker) that has store-and-forward enabled. To test this, turn off the MQTT Broker in Settings. The Status attribute will change from 'Good' to 'Bad.' Although the broker output will not immediately display the data (as it is offline), store-and-forward will cache the writes in the SQLite database located in your runtime folder (or StoreForwardData directory if configured). Once the connection is re-established, the cached writes will automatically be pushed out.
{
"_timestamp": 1723835864894,
"Component_Name": "MQTT",
"Status": "Bad",
"LastError": "Connect operation failed on uri tcp://localhost:1885. Cause: Connection refused: getsockopt."
}
Instead of outputting to MQTT you could this payload or a transformed version of it out of REST output to an observability platform.
Conclusion
Monitoring Connections, Flows and Pipelines via MQTT is an efficient method to track and report statuses. The process outlined in this article can be adapted to various outputs and can be an integral part of your digital transformation strategy.