"Metadata" is one of those words that programmers use like salt and pepper. I like to think of Metadata simply as a list of fields. Let's quickly discuss why Clover needs Metadata and then we will see why it is so important.

Reading Data

When data is read in using a Reader component, it has to be passed to another component for processing. Data is passed, record at a time, along an Edge. If you are taking the serious approach to reading this then you will know already that an Edge is the connecting line between two components.

Each Edge has to be associated with a Metadata definition so it knows what fields of data are being passed from one component to the next. Another way of looking at it is that one component's data spits its data out along the edge and the component at the other end receives that same data. The Metadata simply describes the fields of data that are involved.

Transforming Data

Let's say you do the following simple transformation

Here you can see that you need two Metadata definitions. One for the input format that comes from the file and another for the format of the output file.

To help visualize all this, take a look at the Metadata definition below. In the next section we are going to define Metadata for our customer.csv processing.

It is beyond the scope of this document to go into more detail that we need about them advanced options in a Metadata definition, but if you want to find out more, try here.