What Does This Article Cover?
- Manually parse CSV text to JSON format
- Example
- Considerations
- Other related material
Decoding text byte code
The CSV connector expects expects UTF-8 encoded, character delimitted text files. There may be a scenario where CSV text data can't be ingested through the CSV connector. For example, the file is not encoded in UTF-8, or the data is sourced from a connection that doesn't natively convert CSV data. The following covers an alternative example of converting CSV text data into a JSON format.
Example
Shown here, the source data is a CSV formatted string. *Note the .value is used since metadata is enabled on the input file data and .utf8String references the text data * 
Then converted to JSON format: 
The following expression can be pasted into a Custom Condition where the Source is a File connector Input for CSV formatted text data.
// Define a function CSVToJSON which converts CSV data to JSON format
var CSVToJSON = (data, delimiter = ',', eol = '\n') => {
// Extract titles (header) from the CSV data and split them by the specified delimiter
var titles = data.slice(0, data.indexOf(eol)).split(delimiter);
// Return the rest of the data (excluding the header), split by end-of-line delimiter,
// then map each line to a JSON object using the extracted titles
return data
.slice(data.indexOf(eol) + eol.length)
.split(eol)
.map(v => {
// Split the current line by the delimiter to get the values
var values = v.split(delimiter);
// Reduce the titles and values into a single JSON object
return titles.reduce(
// For each title, assign the corresponding value from the line to the JSON object
(obj, title, index) => ((obj[title] = values[index]), obj),
{}
);
});
};
//initialize the variable with text from the source data. Note that .value is used since metadata is enabled on the input.
var payload = this.currentValue.value.utf8String
//call the function and pass the CSV text, tab delimiter, and end-of-line carriage-return and newline characters
CSVToJSON(payload,'\t','\r\n')
Considerations
In the example here, a tab - '\t' is passed in as the 2nd argument into the CSVToJSON() function call to override the default comma, and the Windows style CRLF "\r\n" is passed in as the 3rd argument to override the default '\n' end of line character.
If, in your case, these are different as well, they can be specified when calling the function as shown in the example above.
CSV - Delimiter - The default delimiter set by the function is a comma. CSV stands for comma-separated values, and traditionally may have only used a comma as the delimiter. As shown in the example provided, other characters like a tab may be used instead.
End of line character - The default end-of-line character is '\n' and the character-delimiter is a ','. Files out of a Windows system typically use Carriage Return (CR) - '\r' and Line Feed (LF) - '\n' for end of line characters. Unix systems typically have just a newline - '\r' character.
Validate the outcome - Make sure to understand what the function is doing to transform the CSV text and to review that the outcome is what you expect. For example, if you expect the end of line characters within the value content, then the provided function must be modified.
Feel free to reach out to us if you have any questions.