Supported file types

Panoply supports the following file types for the File Upload data source:

• .tar and .gzip for archive
• .json for JSON
• .csv and .tsv for character-delimited files - See our sample CSV file
• .xlsx for Microsoft Excel
• .txt for plain text

Aside from these various file types, Panoply also supports the uploading of files of up to 100MB in size.

Note on the JSONs

Data collection succeeds only if your JSON file contains a single object or record. For example, sample data like this is ingested correctly:

{
 "first_name": "John",
 "last_name": "Doe",
 "age": 34,
 "gender": "Male"
}

But it will fail if your file contains more than one record or an array of objects like these:

[{
 "first_name": "John",
 "last_name": "Doe",
 "age": 34,
 "gender": "Male"
},

{
 "first_name": "Amelia",
 "last_name": "Doe",
 "age": 28,
 "gender": "Female"
},

{
 "first_name": "Claudia",
 "last_name": "Doe",
 "age": 32,
 "gender": "Female"
}]

For the data collection to succeed using an array of objects, you need to revise your formatting by using the JSON Lines text format (.jsonl).

Ensure that each object or record occupies a single line (similar to CSV files) and is delimited or separated by a newline.

{"first_name": "John", "last_name": "Doe", "age": 34, "gender": "Male"}
{"first_name": "Amelia", "last_name": "Doe", "age": 28, "gender": "Female"}
{"first_name": "Claudia", "last_name": "Doe", "age": 32, "gender": "Female"}

 Note on the archives

The data collection will fail if you try to upload an archive file (such as .tar or .gzip) that includes unsupported file types or other archive files. To resolve this issue, follow these steps:

1. Review your archive.
2. Remove unsupported files.
3. Upload the modified archive file.
4. Start the data collection.