There may be occasions when you do not know the name of an input file when setting up the graph and so you need to get a list of file names from another file. This file could be created by earlier components in your graph.

Let's say that a UniversalDataWriter had written the following file names into a file called "FilesToProcess.csv".

We can get the reader to read this list of files using the UniversalDataReader. We have to set up the File URL in a special way to get this to process correctly.

Our Graph

The image below shows a simple graph.

The Data Generator simple generates a filename (FilesToProcess.csv) that gets sent to the output port. The Read File(s) component gets this name and then opens up that file. It then reads each of the 3 files and loads them in before passing them to the next component, which is Thrash.

Metadata

You will need to have a metadata definition containing a string value with the FileName. You will also have a Customer metadata record to pass on the read customer data from the files.