Reference Tool: Docker Configuration

What Does This Article Cover?

• What is a Container and Container Image?
• What HighByte provide
• Example commands
• Notes for running the Container
• Makeup of reference image
• Deployment considerations
• Other related material

What is a Container and Container Image?

Container

A Container is a sandboxed environment for running applications decoupled from the host OS. As a result, the Container is portable to any applicable host.

Container Image

The Container Image defines the makeup of the Container and where it can be run, since it will depend on the host's kernel architecture. For example, a Container can be expected to run on x64 Linux. On ARM64 Linux, the x64 kernel will need to be emulated. In that case, you are often better off rebuilding the Container Image targeting the relevant kernel, which may additionally require modifications to the dependencies specified in the image. Container images can be loaded into a local repository, or to your organization's private repository to use across your team, which are then used to generate the containers.

What HighByte provides

HighByte supports container deployments and provides a Docker image supporting x64 Alpine Linux including OpenJDK, for convenience. Users are welcome to download the standard Intelligence Hub package and build container images tailored to meet their unique deployment and security requirements. Requirements may include things like dependencies for running on AArch64 (ARM64) architecture, incorporating third-party JavaScript libraries, navigating network security requirements, managing volumes, etc.

Example commands:

Following are some commands that may be useful when working with the provided Container Image. Note that the Docker container runtime engine should be installed.

Download the Docker Image by logging into the HighByte portal. Open up your command line application and navigate to the folder where the image was downloaded. Run the following command to load the .tar file into the local Docker image repository: The rest of this article references the image file "HighByte-Intelligence-Hub-4.0.0_Docker_Build-2024.9.11.456.tar" as an example.

> docker load -i "HighByte-Intelligence-Hub-4.0.0_Docker_Build-2024.9.11.456.tar"

You can run the following to see a list of images stored locally:

> docker image ls

Then create and run a Container based on the image with a flag for specifying the port to access the Intelligence Hub from the host:

> docker run -p 45255:45245 highbyte:4.0.0

Then browse to the specified port 45255 in a browser on your host:

Anatomy of docker run command:

Additional flags can be added to access other network services, specify name for the Container, specify name for the persistent storage, etc.

> docker run -p 45255:45245 -p 1895:1885 -p 8895:8885 --name highbyte400 --mount source=hb_vol_400,target=/usr/local/highbyte/appData highbyte:4.0.0

Breakdown of the above command:

> docker run \
-p {external port for intelligence hub}:{internal port specified in intelligencehub-settings.json included in the image} \
-p {external port for MQTT server}:{internal port specified in intelligencehub-settings.json included in the image} \
-p {external port for REST DATA server}:{internal port specified in intelligencehub-settings.json included in the image} \
--name {choose name for the container} \
--mount source={optional name for persistent volume on host}, target={the container directory mapped to the persistent volume} \
{name of the loaded image}:{image tag}

Other handy commands for interacting with the Container:

Shell access - bash is not available

> docker exec -it highbyte400 sh

• as root

> docker exec -it -u root highbyte400 sh

check running/all containers

> docker ps
> docker ps -a

/appData - Project configuration files

The appData directory is set to /usr/local/highbyte/appData. The Image creates a volume for this directory automatically to persist data but is assigned a random name and is not mapped to a Host directory. Consider mounting this folder with the -v or --mount flags if you need to transfer files from the Host or specify a name for the volume if you need to transfer files between Containers.

/storeForwardData - Store and Forward

The storeForwardData directory is also set to /usr/local/highbyte/appData for simplicity.   This could be change if it is desired for the storeForwardData to be stored in a different folder.  This would be done by changing its value in the intelligencehub-settings.json

MQTT server

This service is not enabled by default. Enable via the Settings page in the UI or load a modified intelligencehub-settings.json file.

REST Data API server

This service is not enabled by default. Enable via the Settings page in the UI or load a modified intelligencehub-settings.json file.

Consider overriding the intelligencehub-settings.json if needed to specify different values for the above properties:

A modified intelligencehub-settings.json file can be bind mounted to overwrite the intelligencehub-settings.json embedded into the Image. The 'readonly' parameter is optional but can be useful so the Container is expected to be the same each time.

> docker run --name hb400modified_settings -p 45255:45245 --mount type=bind,source=C:\HighByte\modified-settings.json,target=/usr/local/highbyte/runtime/intelligencehub-settings.json,readonly highbyte:4.0.0

Alternatively, bind-mount a modified intelligencehub-settings.json file and then assign it via the environment variable:

> docker run --name hb400modified_settings -e HIGHBYTE_SETTINGS_FILE=/usr/local/highbyte/mod/modified-settings.json -p 45255:45245 --mount type=bind,source=C:\HighByte\modified-settings.json,target=/usr/local/highbyte/mod/modified-settings.json,readonly highbyte:4.0.0

Alternatively, bind-mount a modified intelligencehub-settings.json file and then override the entrypoint with a java start command pointing to the new settings file:

> docker run --name hb400modified -p 45255:45245 --mount type=bind,source=C:\HighByte\modified-settings.json,target=/usr/local/highbyte/mod/modified-settings.json,readonly highbyte:4.0.0 java -cp "/usr/local/highbyte/runtime/intelligencehub-runtime.jar:/usr/local/highbyte/runtime/lib/*" -Dhighbyte.settingsFile=/usr/local/highbyte/mod/modified-settings.json com.highbyte.intelligencehub.runtime.Main start

Note on bind-mounting, when bind-mounting a host directory to a container directory, any existing contents of the container directory will be overwritten.

Alternatively, some prep can be done before starting the container to load the modified intelligencehub-settings.json file into a mounted Managed Volume instead of bind-mounting to a host path:

> docker create --name hb400modified -p 45255:45245 -v managed_vol400:/usr/local/highbyte/appData -e HIGHBYTE_SETTINGS_FILE=/usr/local/highbyte/appData/modified-settings.json highbyte:4.0.0
> docker cp modified-settings.json hb400modified:/usr/local/highbyte/appData
> docker start hb400modified

Example overriding the entry point:

> docker create --name hb400modified -p 45255:45245 --mount source=managed_vol400,target=/usr/local/highbyte/appData highbyte:4.0.0 java -cp "/usr/local/highbyte/runtime/intelligencehub-runtime.jar:/usr/local/highbyte/runtime/lib/*" -Dhighbyte.settingsFile=/usr/local/highbyte/appData/modified-settings.json com.highbyte.intelligencehub.runtime.Main start
> docker cp modified-settings.json hb400modified:/usr/local/highbyte/appData
> docker start hb400modified

Reference: Application Settings

Environment variables - optional settings for the Container

• Environment Variables - deployment specific variables can be passed into the container environment

> docker run -e PUBLIC_Site=Portland -e PUBLIC_Area=Brewery -p 45245:45245 --name highbyte highbyte:4.0.0

• Java runtime memory - specify max heap memory of the Java runtime. Default is 25% of system memory. The following command sets the max heap memory to 4GB.

> docker run -e JAVA_TOOL_OPTIONS="-Xmx4g" -p 45245:45245 --name highbyte highbyte:4.0.0

• Container resources - specify the resource allocation for the Container, i.e. CPU, memory. The following command sets the CPU to 50%, memory to 8GB, and memory-swap to 512MB:

> docker run --cpus=0.5 --memory=2g --memory-swap=512m -p 45245:45245 --name highbyte highbyte:4.0.0

File IO - working directory is /usr/local/highbyte/runtime. This means that when setting up the File or CSV Connection, the immediate folder is this working directory. When writing files to the mapped volume /usr/local/highbyte/appData, either specify the absolute path, or the relative path "../appData".

 

User permissions - the user in the Container is 'highbyte' and has access to the /usr/local/highbyte directory.

Consider bind mounting relevant files to a directory besides /usr/local/highbyte/io to separate access from the /appData files.

Upload files from Host

• When using Volume mount - write in files with global read/write permissions by passing in a tarred file for Intelligence Hub to access:
• as user highbyte

> tar -cf - SOURCE_DIRECTORY | docker cp -a - highbyte400:/home/highbyte/tar

• as root

> tar -cf - SOURCE_DIRECTORY | docker cp - highbyte400:/home/highbyte/tar

• /expression-imports - This can be useful for importing third-party JS libraries into the /appData/expression-imports folder, since npm isn't included in the Image. Consider mounting to a Docker managed volume for /expression-imports, especially on Windows, for disk IO speed. The container will have to be restarted to restart the application runtime for the JS libraries to be recognized. Another method is to use Docker cp directly, and then optionally change the file ownership afterwards:

> docker cp file_to_import highbyte400:/home/highbyte/tar
> docker exec -u root
> chmod 777 /home/highbyte/tar/file_to_import

• When using Bind-mount - Files can be copied directly into the mapped host file directory with global read/write permissions for Intelligence Hub to access. The readonly parameter at the end is optional to prevent changes made within the Container from affecting the host files. This can be useful for config/setting files.

docker run --name hb400 -p 45255:45245 --mount type=bind,source=C:/HighByte/highbyte400,target=/usr/local/highbyte/appData,readonly highbyte:4.0.0

• Consider leaving the /appData as a Docker managed Volume instead of bind-mounting for disk IO speed. If needed to extract files, for example pulling the log file, the following can be used:

  > docker cp highbyte400:/home/highbyte/appData/intelligencehub-events.log file_exported.log

Log file

Can be viewed through the browser UI and the file is located in the /appData directory.

Versioning - Image Tags

Assign tags to the images. This is useful when managing multiple versions of Intelligence Hub.

Here we see that 4.0.0 image tag gets assigned 4.0.0.456, and then a new 4.0.0 image is loaded.

    > docker image tag highbyte:4.0.0 highbyte:4.0.0.456

Networking

• Host to Container - use the assigned external port assignments from when the Container was created, i.e. ports 8556 and 1890 for-p 8556:8555 -p 1890:1885

• Container to Host - host.docker.internal - to access services running on the host machine from the container

• Container to Container - find the IP address and use the internally mapped port assignment(s).

  • Find the ip address on the default bridge network

  > docker network inspect bridge
  

  

Consider creating a user defined bridge network to group Containers that will interact with the Intelligence Hub. In this case, Containers can be referenced by their hostname, network alias, or Container name instead of having to find the IP address.

> docker network create new_network

Add Containers to the new network: Here we've added the --hostname and --network-alias parameters: Don't include underscores in the hostnames as they are not valid

> docker run --hostname highbyte400-hostname --name highbyte400-container --network new_network -p 45255:45245 highbyte:latest
> docker run --hostname postgres-hostname --name postgresql-container --network-alias postgres-alias --network new_network -p 5435:5432 -e POSTGRES_PASSWORD=mysecretpassword postgres

Certificates

Consider setting a hostname to fix relationship with self-signed certificates

Override entry point commands - In the event that a custom .jar file must be run in the container; one method is to bind-mount the file and then pass the java commands to start the runtime.

> docker run --name hb400modjar -p 45255:45245 --mount type=bind,source=C:\HighByte\jar\intelligencehub-runtime.jar,target=/usr/local/highbyte/modjar/intelligencehub-runtime.jar highbyte:4.0.0 java -cp "/usr/local/highbyte/modjar/intelligencehub-runtime.jar:/usr/local/highbyte/runtime/lib/*" com.highbyte.intelligencehub.runtime.Main start

Further consideration - Consider creating a docker-compose.yml file for storing the various settings across many Containers.

Makeup of reference image

FROM alpine
RUN apk add openjdk11

# Allow changing the username at build time, default to highbyte
ARG USERNAME=highbyte
RUN adduser -D $USERNAME

# Create and chown before we drop root
RUN mkdir -p /usr/local/highbyte && chown highbyte:highbyte /usr/local/highbyte

# Don't run as root
USER $USERNAME

# Backend Runtime Application
COPY --chown=highbyte:highbyte ./runtime /usr/local/highbyte/runtime

# Runtime Configuration file
COPY --chown=highbyte:highbyte ./intelligencehub-settings.json /usr/local/highbyte/runtime

# Frontend Configuration GUI
COPY --chown=highbyte:highbyte ./configuration /usr/local/highbyte/configuration

EXPOSE 45245

WORKDIR /usr/local/highbyte/runtime

ENTRYPOINT [                                      \
    "java",                                       \
    "-cp",                                        \
    "intelligencehub-runtime.jar:lib/*",  \
    "com.highbyte.intelligencehub.runtime.Main"   \
]

CMD ["start"] # options: ["start", "stop"]

# Persist the users appData between container restarts
VOLUME ["/usr/local/highbyte/appData"]

Other related material:

*User Guide-Docker