Manage Flat Files in Composer

Composer can visualize data from flat files including CSV, JSON, and TSV files.

• The maximum supported file size is 500 MB
• Data from the flat file is stored in a PostgreSQL database

Before you can establish a connection from Composer to your flat file storage, a connector server needs to be installed and configured. See Manage Connectors and Connector Servers for general instructions.

After the connector has been set up, create a data source configuration and upload your file or files to that source. See:

• Define a Data Source Configuration
• Upload a New File
• Edit an Existing File
• API Endpoints

Upload a New File

1. Log in as a user with the Administer Sources or Create New Data Sources privilege.

2. Create or edit an existing data source.

3. Select Add to add a new data entity, then select From File.

   

4. Add a unique Data Entity Name, then select Upload New File. The File Upload work area opens.

   

5. Enter File Details, such as a unique Display Name, and optional Description.

6. Use the Browse button to select a file to upload.

7. After you have selected a file, Composer may autofill the Single Quote Char. and Field Delimiter fields. Adjust if needed.

8. If needed, change the No. of records to display from 10 to up to 1,000 records.

9. Enable or disable the checkbox Column Headers in first row to match your file layout. If no column headers are in your data, Composer assigns numerical column headings, field_1, field_2, and so on.

10. Optionally, select Preview to preview your data.

11. Enable or disable the checkbox Use Only File Structure .

12. Select Save to close the File Upload work area and continue creating or updating your data entity for this source.

Edit an Existing File

1. Log in as a user with the Administer Sources or Create new Data Sources privilege.

2. Edit an existing data source.

   

3. Select the data entity with the file you want to edit, then select Edit File. The File Upload work area opens.

4. Browse for a new file.

   • If you are replacing the existing file, you can upload a file with any data file structure. Enable the Replace checkbox in Upload Settings.

   • If you are adding data to the existing file, your new file must use the same data file structure as the existing file. Disable the Replace checkbox in Upload Settings.

   

5. Select Preview to preview your data.

   • The Current Data tab shows the data of your existing file.

   • The New Data shows the replaced or amended data preview.

6. Select Save to Save your changes or Cancel to discard your changes. The File Upload work area closes.

API Endpoints

1. Log in as a user with the Administer Sources or Create new Data Sources privilege.

2. Edit an existing data source.

   

3. Select the data entity with the file you want to edit, then select API Endpoints. The API Endpoints dialog box opens.

   The dialog offers convenient example cURL requests but the APIs can be leveraged from your preferred development platform.

4. Copy and modify the example cURL requests to include your own Composer credentials, replacing the placeholders for username and password. Select Close to close the dialog.

   

Composer Feature Support

Flat file support for specific Composer features is shown in the following table.

Key: Y - Supported; N - Not Supported; N/A - not applicable