Write the config

Finally, define a config.json file. There must be exactly one config file per delivery, located in the root directory. It tells the pipeline which files to read and what entity type they define, names the data’s sources and provenance, and specifies the column mapping. The mapping is only needed when the column names were customized.

Here’s an example configuration:

{
  "includeInputSubdirs": true,
  "inputFiles": {
    "atividade_fisica_adultos.csv": {
      "provenance": "MINISTERIO_SAUDE_BR",
      "format": "variablePerRow",
      "columnMappings": {
        "variable": "sv",
        "entity": "municipio",
        "date": "ano",
        "value": "valor"
      }
    }
  },
  "groupStatVarsByProperty": true,
  "sources": {
    "dados.saude.gov.br": {
      "url": "https://dados.saude.gov.br",
      "provenances": {
        "MINISTERIO_SAUDE_BR": "https://dados.saude.gov.br/dataset/atividade-fisica-adultos"
      }
    }
  }
}

The config has four main fields: includeInputSubdirs, inputFiles, groupStatVarsByProperty, and sources.

includeInputSubdirs (optional)

Whether the pipeline should consider files in subdirectories of the input folder. When true, files can be referenced using relative paths within those subdirectories. If omitted, the default is false.

inputFiles (required)

Defines the input files the pipeline will process. Each CSV is one entry, keyed by its filename — so if files live in subdirectories of the input folder, specify their paths relative to it. You can also use the * wildcard to select multiple files matching a name pattern.

The parameters for each CSV:

Parameter	When	Meaning
format	required	Always variablePerRow.
provenance	optional	Identifies the specific dataset the data came from. Each provenance is a dataset published by a given source and must match a provenance defined under sources.
importType	required for entities	The type of data in the file. When the data defines new entities, use entity; otherwise the default is variables.
entityType	required for variables	The type of the entities the observations belong to. All entities in one file must be the same type. For files that define custom entities, use rowEntityType.
rowEntityType	required for entities	The DCID of the entity type — new or existing — associated with the custom entities defined in the file.
columnMappings	required for custom columns	Fill this in when the CSV column headers don’t follow the standard names.

Using the CSV from the formatting step, the mapping looks like:

"columnMappings": {
  "variable": "sv",
  "entity": "municipio",
  "date": "ano",
  "value": "valor"
}

groupStatVarsByProperty (optional)

When true, groups variables in the Statistical Variable Explorer by the values of custom properties. For example, if a gender property is defined with the values male and female, the variables are grouped accordingly.

sources (required)

Defines the sources and provenances associated with the input data. Each named source contains a mapping between provenances and their URLs.

Parameter	Meaning
url	The URL of the source.
provenances	A set of NAME: URL pairs identifying the specific datasets obtained from that source. The provenance name must match the value used in the provenance field under inputFiles.

Final file layout

After the steps above, you should end up with a structure similar to this:

meus_dados/
├── config.json
├── vars1.mcf
├── dados1.csv
├── dados2.csv
└── outros_dados/
    ├── vars2.mcf
    ├── dados3.csv
    └── dados4.csv

The number, names, and organization of the CSV and MCF files inside the folder are up to you. The config.json, however, must live in the root directory.

With the files in place, validate and deliver the dataset.