Skip to content
  • There are no suggestions because the search field is empty.

Files: Overview

This article introduces the starter solutions that pertain to files.

What Does This Article Cover?

  • How HighByte Intelligence Hub is used with files 
  • Which Intelligence Hub Connections are used with files
  • Typical design patterns you'll see in file-related solutions

 

Files and Intelligence Hub

At HighByte we observe that due to legacy equipment in some environments valuable OT data is contained in files.  Intelligence Hub can be used to compose solutions that read data from files, create files, move files, add context and metadata for files and more.  

 

Intelligence Hub Connections for Files 

The Intelligence Hub File Connector supports reading and writing files locally on the file system, via SMB (Server Message Block), FTP, and SFTP.  In addition, Intelligence Hub includes dedicated Connections for Parquet Files and CSV Files. 

 

Typical Intelligence Hub Solutions  

The following captures some of the Intelligence Hub solutions related to files that we observe.  Starter solutions are available for the scenarios below.  Once ingested into in a pipeline, Parse stages are available to convert XML and JavaScript into events.

Solution Description
Convert the format of a file Intelligence Hub can be used to create a solution that converts a file to a common format.  For example, convert a pipe delimited file without headers to a conventional CSV file.
Add context to a file  Adding context to a file can be valuable for identifying the contents of a file.  For example, Intelligence Hub can be used to move a file and add a descriptive filename or add metadata for the file when writing the file to a data lake like Amazon S3.
Process top file A starter solution for processing a file written to a directory.
Process many files  A starter solution related to processing many files that were written to a directory.
Transfer files via MQTT Use MQTT to move files from one directory to another.

 

Summary

Starter solutions accelerate time-to-value by giving you a ready-to-run baseline for file scenarios.  Use them to learn patterns and then tailor them for your environment, scale, and operability needs. 

 

Additional Resources