What Does This Article Cover?
An Intelligence Hub solution can be created to subscribe to AVEVA PI System to obtain metadata for Asset Framework assets and attributes. The solution might be used to create metadata tables to related to a narrow values table. The following describes how this solution can be created.
Design Assumptions
The following are some of the design considerations.
- This design approach should be used to create an infrequently updated metadata table in a data warehouse or data lake to complement values table. This approach assumes that the metadata table and the value tables will be linked via an identifier. In this case the unique key is attribute ID and asset ID.
- This design approach uses the Intelligence Hub PI Connection Asset Metadata Input that returns attribute configuration per asset including PI Point mappings to attributes. The Asset Metadata Connection Input queries PI Asset Framework's database and can be a long duration query if there are many assets returned by the query.
- This design approach will handle new PI assets and new attributes assuming the query defined in the Connection Input will obtain the new assets. For example, the reference solution uses a PI AF template in the query so as new assets are added in PI using the template they will be returned by the query.
Overall Solution Configuration (Attribute Metadata)
The following describes the Intelligence Hub configuration objects.
- Obtain data from PI using the PI Connection Asset Metadata Input. Include attributes should be on. Include Children should typically be turned off.
- Model the desired output schema using an Intelligence Hub Model.
- Create a Pipeline with Flow stage configured with the desired polling frequency. It will typically be infrequent.
- Configure the Connection Input as the Reference of the Flow stage.
- Configure the Connection Output for the respective data store.
Pipeline Configuration (Attribute Metadata)
The following describes the Intelligence Hub Pipeline design.
- Breakup the incoming array.
- Store the element ID for each asset in metadata.
- Filter to focus on only the attribute object.
- Breakup the resulting array.
- Flatten the point object.
- Map the payload to the predefined schema using the Intelligence Hub Model.
- Use the OnChange stage to only send payloads with changed data for each attribute ID.
- Buffer transactions using a strategy that aligns with the solution requirements for latency. It may be beneficial to buffer transactions for a cloud data warehouse, like Snowflake for example.
- Consider the use of the Buffer Key based on use case.
- Create the desired file format for the data lake if applicable.
- Write to the data lake configuring the desired file name format.
Other Considerations
The following should be considered related to this design
- The PI Connection Asset Metadata Input query is relatively slow and may not be capable of returning values for hundreds of PI Assets.
- Consider whether static values for attributes should be stored in a metadata table or a values table.
Solution Video
The following video captures a reference solution.
Additional Resources