Introduction
HighByte Intelligence Hub is well-suited to connect to many different data technologies to either read or write data payloads. However, there may be times when it is desirable to write directly from one hub to another. While there is no writing mechanism specifically for the Intelligence Hub, its flexibility allows it to send and receive payloads by several mechanisms. The simplest, in many cases, is using REST transfers over HTTP, but depending on needs, plenty of other mechanisms are available. REST should be considered a first choice and a best practice unless it is unsuitable for some reason.
Best Practice: REST
The Rest Data Server may be enabled on the recipient Intelligence Hub. A pipeline will be built to receive delivered payloads and treat them accordingly. If multiple types of data should be delivered, then multiple pipelines may be built - each accessed with a unique HTTP endpoint - or a single pipeline may be built that uses switch stages and model validation stages to sort payloads and deliver them as necessary to subsequent destinations or pipelines. The receiving hub will use an API Trigger.
The sender hub will write to a REST output using a Post message to encapsulate the event.
Strengths
-
Simple to set up and use
- Only requires a single network port (8885 by default) that serves many purposes
Weaknesses
-
Relatively low performance
Alternative Methods
If REST is not suitable for a situation, other methods are available. This set of options is not exhaustive. Oftentimes, a specific problem can have a very specific ideal solution. It is still valuable to consider whether a simple solution "works" even if it is not "optimal" to avoid unnecessary engineering overhead.
Webhook
Webhook works very similarly to REST. A receiving pipeline is set up in a receiving hub, and a sending hub uses a REST Post to deliver the event to the receiving hub. A receiving pipeline will use an Event Trigger to read the webhook events as they are delivered. Those events may then be sorted in the pipeline similarly to REST.
Webhook pipelines are not required to deliver a message back to the sender, which makes them simpler to set up and quicker to perform, but if multiple webhook pipelines are necessary and the events cannot or should not be sorted in a pipeline, a unique incoming port is necessary per webhook connection.
Strengths
-
Simpler receiving pipelines
- Better HTTP performance for delivery
Weaknesses
-
May need multiple ports per webhook connection (9001 by default)
MQTT
MQTT is a very performant mechanism to transfer payloads. It relies on another technology stack to receive, store, and hand off event payloads. The sender hub will use an MQTT connection to write to a determined topic or structure, and then the receiving hub will subscribe to that topic with its own MQTT connection. This MQTT connection supports the Event Trigger.
If MQTT is already available regardless, or the data is valuable to more than one consumer, or if the performance for many small, high-frequency writes, then MQTT makes sense and can provide a lot of value. By this mechanism, MQTT is a very common platform for building a Unified Name Space. However, if delivery frequency can be mitigated another way or there is only one data consumer, the additional deployment may not be beneficial.
The additional deployment could be mitigated by using HighByte Intelligence Hub's built-in MQTT broker.
Strengths
-
High performance
- Distributes to many consumers
Weaknesses
-
Requires an additional 'hop'
- Requires an additional technology deployment and stack