Skip to content
  • There are no suggestions because the search field is empty.

Add a DateTime or Timestamp to file name

Reference on some consideration points for adding timestamp on the output name when writing to file

What Does This Article Cover?

Some options for adding a timestamp value to the filename when writing. The options vary depending on using a predefined File Output with a Write Stage and using a Write New Stage.

  • Prerequisites

    • Working with File Output and Write Stage
    • Familiar with System.Internal.DateTime
    • JS methods .getTime() and .replace(
    • regex used with .replace()
    • Familiar with Unix time (aka epoch) and ISO8601 formats
    • Working with Write New Stage
    • Dynamic Outputs

Why:

{%raw%}The built-in {{System.Internal.DateTime}} evaluates in ISO8601 format which has characters that may not be desired or not be supported for filename use by the host operating system (e.g. ':' not supported in filenames on Windows OS)

Results in:

Option 1: Assign a unix timestamp in the Write New Stage

The handlebar syntax can be added to the File Name directly by converting the System.Internal.DateTime to unix timestamp. The .getTime() method is added to the end to convert it into unix timestamp that only has integers, since System.Internal.DateTime evaluates as ISO8601 format, which includes ':' and many systems don't support having a ':' in the filename.

This method is simple to add, but is an advanced feature as you need to be familiar with how JS expressions can be executed within {{}} syntax in the text fields of the Write New Stage.

FileName{{System.Internal.DateTime.getTime()}}

Option 2: Assign an ISO8601 format using event.metadata

This option provides a clear step in the Pipeline with a Transform Stage dedicated to generating the timestamp and assigning it to event.metadata to be used later on. This makes it clear how the timestamp is being generated at first glance.
 

The user creating this expression needs to be familiar with JS and how the regex expression works that's passed to the .replace() method, but even users who aren't familiar with JS can understand the function of this Stage by viewing the event data in the replay/debug.

 Example file output:
 
 
Use a pipeline to populate the metadata with the desired date format:
 
 
Transform - "SetMetadataDateTime"
//assign the built-in system date object
const date = System.Internal.DateTime;

//call method for the ISO string
const filenameSafeDateTimeISO = date.toISOString().replace(/[:.-]/g, "_");

//call method for the Unix timestamp value
const filenameSafeDateTimeUnix = date.getTime();

//assign values to metadata
stage.setMetadata("dt_iso",filenameSafeDateTimeISO); 
stage.setMetadata("dt_unix",filenameSafeDateTimeUnix);
Write New - "WriteToFile"
 

Option 3: Outputting year/month/day/hour using event.metadata

Here's another example similar to Option 2, but just with a different formatting in case you want the timestamp to be used as a tree structure like is often done in data lake storages, like with how our write to AWS S3 or Azure Blob includes a toggle option to add this prefix.

Example output

Use a pipeline to populate the metadata with the desired date format:

Transform - set event.metadata

//assign the built-in system date object
const date = System.Internal.DateTime;
//you can also use the following:
//const date = new Date();

// Extract parts, in UTC
const year = date.getUTCFullYear();
const month = String(date.getUTCMonth() + 1).padStart(2, '0'); // months are 0-based
const day = String(date.getUTCDate()).padStart(2, '0');
const hour = String(date.getUTCHours()).padStart(2, '0');

const formatted = `${year}/${month}/${day}/${hour}`;
stage.setMetadata("dt",formatted); // 2025/08/12/10
Write New
 
 

Notes:

Using the {{System.Internal.DateTime}} as a Dynamic Output in the File Name field of the File Connection >> Output is not supported. Predefined Connection>>Outputs like this are used with the Write Stage, therefore, it is suggested to use the Write New Stage instead as covered in the options above.
 
Results in: